Echoes Across the Screen: A Comparative Analysis of Phonetic Convergence in Face-to-Face and Video-Mediated Interactions

Abstract

The phenomenon of phonetic convergence—the tendency for interlocutors to become acoustically more similar to one another during interaction—is a well-documented feature of human speech production, underpinned by Communication Accommodation Theory and episodic theories of lexical access. However, the vast majority of research regarding phonetic accommodation has been conducted in face-to-face (FtF) laboratory settings or via high-fidelity audio-only channels. The rapid proliferation of video-conferencing technology (e.g., Zoom, Microsoft Teams) introduces a novel sociolinguistic environment characterized by visual latency, audio compression, and a lack of shared physical space. This study investigates whether the magnitude and nature of phonetic convergence differ between FtF and Video-Mediated (VM) interactions. Sixty dyads participated in a cooperative map task in either an in-person or a remote video condition. Acoustic analysis focused on vowel space dispersion (F1 and F2 frequencies) and speech rate entrainment. Results from linear mixed-effects models indicate that while temporal convergence (speech rate) remains robust across both modalities, spectral convergence (vowel formants) is significantly attenuated in VM interactions. These findings suggest that the technological mediation of the video call acts as a sensory filter that disrupts the subtle acoustic-phonetic feedback loops necessary for fine-grained spectral accommodation, carrying significant implications for sociolinguistic theory in the digital age.

Introduction

Human interaction is rarely a static exchange of information; it is a dynamic process of mutual adjustment. Among the most subtle yet pervasive of these adjustments is phonetic convergence, a phenomenon where speakers inadvertently (or sometimes strategically) alter the acoustic characteristics of their speech to match that of their interlocutor. This behavior, often subsumed under the broader framework of Communication Accommodation Theory (CAT) proposed by Giles (1973), serves as a mechanism to reduce social distance, facilitate comprehension, and signal group solidarity. For decades, sociolinguists and psycholinguists have mapped the contours of this phenomenon, establishing that speakers align on various dimensions, including voice onset time (VOT), vowel formants, pitch, and speech rate (Pardo, 2006; Babel, 2012).

However, the landscape of human communication has undergone a seismic shift. The ubiquity of computer-mediated communication (CMC), particularly video-conferencing platforms, has moved a significant portion of daily interaction from shared physical spaces to virtual environments. This shift raises critical questions regarding the robustness of speech accommodation mechanisms. Does the "Zoom fatigue" associated with cognitive overload, combined with audio compression algorithms and network latency, dampen the cognitive resources available for accommodation? Or, conversely, does the "headphones effect"—where voices are piped directly into the ear—enhance attention to the acoustic signal, thereby increasing convergence?

This article presents an experimental study comparing phonetic convergence in traditional Face-to-Face (FtF) settings versus Video-Mediated (VM) interactions. We posit that while the intent to accommodate remains constant, the channel constraints of VM interaction alter the acoustic targets available to the speaker, resulting in distinct patterns of convergence.

Theoretical Framework

Two primary theoretical frameworks inform the predictions of this study: the exemplar-based models of speech production and Communication Accommodation Theory (CAT).

Exemplar theory (Goldinger, 1998) suggests that speech perception and production are tightly linked through episodic memory. When a speaker hears an interlocutor, the specific acoustic details of that utterance are stored as traces. Subsequent production activates these traces, biasing the speaker’s output toward the recently perceived tokens. If VM environments degrade the acoustic fidelity of the input signal—through frequency cut-offs or compression artifacts—the exemplar traces may be less rich, potentially inhibiting precise spectral matching.

Conversely, CAT emphasizes the social motivation behind convergence. Giles, Coupland, and Coupland (1991) argue that convergence is a strategy to gain social approval. In VM settings, the lack of shared physical context (co-presence) might create a greater psychological distance. According to Media Richness Theory (Daft & Lengel, 1986), video conferencing is leaner than FtF interaction. If convergence acts as a "social glue," speakers in leaner media might theoretically over-compensate to bridge the digital divide, or alternatively, fail to converge due to the diminished salience of social cues.

The Impact of Transmission Quality

Technical constraints in CMC cannot be overlooked. Video conferencing software utilizes codecs (e.g., Opus, SILK) that prioritize intelligibility over spectral fidelity. High-frequency information is often attenuated, and temporal latency (jitter) can disrupt the rhythmic turn-taking essential for entrainment (Lev-Ari & Keysar, 2010). If phonetic convergence relies on the precise calibration of articulation, the "noisy" channel of a video call may disrupt the feedback loop required for alignment.

Therefore, we propose two hypotheses:

• H1 (The Robustness Hypothesis): Temporal measures (speech rate), which are resistant to spectral degradation, will show equal convergence in both FtF and VM conditions.
• H2 (The Degradation Hypothesis): Spectral measures (vowel formants), which rely on fine-grained acoustic detail, will show significantly reduced convergence in VM conditions compared to FtF.

Methodology

Participants

One hundred and twenty participants (60 dyads) were recruited from a large North American university community. All participants were native speakers of American English, aged 18–35 (M = 22.4, SD = 3.1), with no self-reported history of speech or hearing disorders. Dyads were same-gender pairs to minimize physiological differences in fundamental frequency and vocal tract length that can complicate acoustic distance calculations. Pairs were unacquainted prior to the experiment. The study was approved by the Institutional Review Board, and informed consent was obtained.

Experimental Design

The study employed a between-subjects design. Dyads were randomly assigned to one of two conditions:

1. Face-to-Face (FtF): Participants sat across from each other at a table in a sound-attenuated room, separated by a low visual barrier that allowed eye contact but obscured the task materials.
2. Video-Mediated (VM): Participants were seated in separate sound-attenuated rooms. They communicated via a low-latency video conferencing setup (custom WebRTC interface) using high-fidelity headsets. Crucially, to ensure acoustic analysis was not compromised by network compression in the recording , local audio was recorded directly at the source (44.1 kHz, 16-bit) before transmission. However, the audio heard by the partner was subject to standard Opus codec compression (approx. 32 kbps) to simulate typical video call quality.

Task and Procedure

To elicit spontaneous yet controlled speech, we utilized the "Diapix" task (Baker & Hazan, 2011). Each participant was given a cartoon image; the two images were almost identical but contained 10 discrete differences (e.g., the color of a flower, the presence of a dog). Participants were instructed to converse to find the differences without showing their pictures to one another. This task necessitates collaboration, repetition of keywords, and negotiation of meaning—ideal conditions for convergence.

The session lasted approximately 20 minutes. Following the interaction, participants completed a brief post-task survey regarding their perceived rapport with their partner and the difficulty of the task.

Data Processing and Acoustic Analysis

Audio recordings were segmented into inter-pausal units (IPUs) and force-aligned using the Montreal Forced Aligner (McAuliffe et al., 2017). We focused on two primary acoustic metrics: Vowel Space Density (Spectral) and Speech Rate (Temporal).

Spectral Analysis: Vowel Formants

We extracted the first two formants (F1 and F2) from the midpoint of the stressed vowels /i/, /u/, /a/, and /æ/. To normalize for anatomical differences, formant values were Lobanov-normalized. Convergence was operationalized as the change in Euclidean distance between speakers' vowel spaces from the first quarter to the last quarter of the interaction.

The spectral distance between speakers A and B for a given vowel was calculated as:

Global spectral distance was the mean of across all four vowels.

Temporal Analysis: Speech Rate

Speech rate was calculated as syllables per second (syl/s), excluding pauses greater than 200ms. Convergence was defined as the reduction in the absolute difference between Speaker A's and Speaker B's speech rates over time.

Results

Data were analyzed using Linear Mixed-Effects Models (LMER) in R using the lme4 package. The dependent variable was the Dyadic Distance (absolute difference between partners) for the acoustic feature. Fixed effects included Time (Early vs. Late interaction), Condition (FtF vs. VM), and their interaction. Random intercepts were included for Dyad and Word.

Spectral Convergence (Vowel Space)

Figure 1 (described below) visualizes the change in vowel space distance. A significant main effect of Time was observed (), indicating that, overall, speakers became more similar in their vowel production as the task progressed. However, a significant interaction between Time and Condition was found ().

Post-hoc analysis revealed that while FtF dyads showed substantial convergence (a large reduction in distance), VM dyads showed a negligible reduction in spectral distance. This supports H2: spectral convergence is inhibited in video-mediated environments.

Temporal Convergence (Speech Rate)

For speech rate, the LMER revealed a significant main effect of Time (), showing that partners synchronized their speech rates over the course of the conversation. Crucially, the interaction between Time and Condition was not significant ().

Table 1 summarizes the model outputs. The lack of interaction suggests that speech rate convergence occurs to the same degree regardless of whether the interlocutors are face-to-face or communicating via video.

Table 1: Fixed Effects Estimates for Spectral and Temporal Convergence Models

Parameter	Estimate ()	Std. Error	t-value	p-value
Model A: Spectral Distance (Vowels)
(Intercept)	2.45	0.12	20.41	<.001
Time (Late)	-0.42	0.11	-3.81	<.001
Condition (VM)	0.15	0.16	0.93	.352
Time × Condition (VM)	0.31	0.14	2.21	.028*
Model B: Temporal Distance (Speech Rate)
(Intercept)	0.85	0.05	17.00	<.001
Time (Late)	-0.15	0.04	-3.75	<.001
Condition (VM)	0.08	0.07	1.14	.256
Time × Condition (VM)	0.02	0.06	0.33	.640

Discussion

The results of this study provide a nuanced answer to the question of how technology shapes phonetic accommodation. We found a clear dissociation between spectral and temporal convergence. While speakers in video calls matched each other's pacing just as effectively as those meeting in person, they failed to converge significantly on vowel quality. These findings have several theoretical and practical implications.

The Spectral Filter of Technology

The attenuation of spectral convergence in the VM condition supports the exemplar-theoretic view that robust acoustic input is necessary for phonetic alignment (Goldinger, 1998). In a face-to-face setting, the rich, uncompressed acoustic signal allows the listener to map the interlocutor's vowel space with high fidelity. In the VM condition, despite high-quality headsets, the intervening transmission codecs likely obscured the subtle formant cues required for imitation.

It is also possible that the lack of spectral convergence reflects a disruption in the "automatic" nature of accommodation. Pardo (2006) notes that convergence is often subconscious. If the cognitive load of processing video (interpreting pixelated facial expressions, managing latency) is high, the cognitive resources usually allocated to phonetic monitoring may be diverted. This aligns with the "Cognitive Load Theory" applied to CMC, suggesting that the brain is too busy managing the medium to manage the minutiae of the message's form.

Temporal Entrainment as a Communicative Necessity

In contrast, the robustness of speech rate convergence suggests that temporal alignment serves a different, perhaps more fundamental, communicative function. Street (1984) posited that speech rate convergence is critical for turn-taking management. In video-mediated interaction, where latency poses a constant threat of talking over one another, synchronizing speech rate may become more critical. Speakers effectively lock into a rhythm to predict turn-endings, compensating for the lack of physical cues. This resilience of temporal convergence indicates that while technology may distort timbre , it cannot easily suppress the human drive for rhythm .

Sociolinguistic Implications

From a sociolinguistic perspective, if vowel convergence is a primary marker of social closeness (Babel, 2012), the lack thereof in VM settings implies that video calls may be structurally ill-suited for building deep implicit rapport. While explicit information is exchanged effectively, the implicit "we are the same" signal sent through formant alignment is lost in transmission. This supports the notion that CMC, despite visual availability, remains a "lean" medium for social solidarity compared to physical co-presence.

Conclusion

As our communicative landscape becomes increasingly digitized, understanding the physiological and acoustic impacts of these tools is paramount. This study demonstrates that phonetic convergence is not a unitary phenomenon; it is sensitive to the modality of interaction. While the temporal scaffolding of conversation remains intact in video calls, the spectral nuances that color our social interactions are filtered out. We accommodate to the machine as much as we accommodate to the human.

Future research should investigate whether higher fidelity audio (lossless transmission) restores spectral convergence, or if the psychological barrier of the screen is the true inhibitor. Additionally, exploring how these dynamics play out in immersive Virtual Reality (VR) could offer further insights into the future of embodied digital interaction.

References