Introduction

By far the most common kind of language use is conversation [1]. In conversation, utterances are produced spontaneously. That is, they are “conceived and composed by their speakers even as they are spoken” [2] (p. 136). One consequence of this is that spontaneous speech contains disfluencies. These are generally defined as “phenomena that interrupt the flow of speech and do not add propositional content to an utterance” [3] (p. 709). They include pauses, interruptions (midphrase or midword), repeated words and phrases, restarted sentences, words with elongated pronunciations, such as the pronounced thee and a as ay, and fillers such as uh and um. Such disruptions are very frequent: Averaging across a number of studies, and excluding silent hesitations, it has been estimated that disfluency in spontaneous speech affects about 6 per 100 words [3], [4].

At first glance, it would seem that the many disfluencies in spontaneous speech present a formidable challenge to the speech perception system, because disfluencies result in strings of words that are not grammatically correct and strings of sounds that are not words. This view that disfluencies are ‘noise’ and present obstacles to perception [5] (p. 275) is probably the reason that radio broadcasters tend to edit out disfluencies from interviews and that the written media tend to omit disfluencies from their renditions of people's speech [6]. It may also be one of the reasons that most studies on spoken language comprehension have used idealized, fluent utterances. However, the small number of studies that have investigated the effect of disfluencies on word or sentence comprehension converge on the opposite conclusion, namely that under some circumstances, disfluencies can in fact help the listener [5]–[13]. There may be several reasons why disfluencies can facilitate comprehension (see below). This article investigates one property of disfluencies that might make them helpful, namely that they delay the onset of the following word.

Earlier studies have shown benefits of disfluencies on both participants' comprehensibility ratings of sentences and on on-line processing measures. For example, listeners rate utterances including self-repairs as more comprehensible when those repairs are preceded by pauses [13]. In on-line tasks, participants are quicker to identify a ‘correct’ repair word (orange) following either a between-word interruption (yellow–orange), or a mid-word interruption with or without a filler (yel–uh–orange, yel–orange), compared to fluent controls [5]. The quickest identifications are in cases where the interruption includes a filler. These findings strongly suggest that pauses and fillers help the identification of upcoming words. This conclusion is partially supported by a word-spotting study [6] which shows that both English and Dutch listeners are faster to identify a target word in a carrier sentence when it follows an uh in comparison to a control condition without the uh. In a comparable set of conditions with or without an um there is no effect.

Why would (some) fillers facilitate auditory word recognition? On one account [6], fillers ‘signal’ delays in speech, with specific fillers signaling specific lengths of delay. The fillers could therefore heighten attention for upcoming speech. In particular, this account rests on the assumption that um signals a longer delay than uh does. This assumption is supported by an analysis of transcriber-rated pause lengths in a speech corpus [14]. Given this difference, uh is predicted to signal a relatively short delay, and it is functional for the speech perception system to immediately heighten attention for upcoming speech. But in the case of um, heightening attention is less functional, because there is no reason to expect the next word anytime soon[6]. In short, on this account, the speech perception system is sensitive to the patterns of delays that tend to co-occur with fillers in natural speech, so that attention can be allocated in a way that is appropriate to the particular filler that occurs.

However, it is also possible that the benefits for perception emerge from the fact that disfluencies like uh and um, and any silent pauses preceding or following the filler, considerably delay target word onset themselves. For example, in [6] (Experiment 2) the average duration of um was 615 ms; the average durations of preceding and following pauses were 592 ms and 412 ms respectively. According to the delay hypothesis tested here, temporal delay facilitates word recognition. Anecdotal evidence suggests that speakers who are difficult to follow will be easier to understand when they speak more slowly (presumably, slowing speech affects pausing). Furthermore, there are theoretical reasons why a delay in word onset might facilitate word recognition. For example, delays might make the listener's segmentation problem easier, because speech sounds spanning a delay most likely do not belong to the same word. It is also possible that delays help top-down processes: The more time passes, the more time there is to make top-down predictions about the next word. Finally, it may be the case that attention builds up over the course of any delay.

The delay hypothesis predicts that um, like uh, should aid word recognition and therefore appears to conflict with previous findings [6]. But as noted above, in that study there were relatively lengthy silent pauses preceding and following the uhs and ums, and these pauses were left intact in the control stimuli. As a result, there were delays in the disfluent stimuli, but also in the ‘fluent’ stimuli. According to the delay hypothesis, these silent pauses in the fluent stimuli would themselves facilitate word recognition, so that the further delay from having a filler would do little to facilitate this process even more.

To test the delay hypothesis, we conducted three experiments that assessed the effects of three types of delays on auditory word recognition. In all experiments, listeners viewed displays of two images and listened to instructions to press a button corresponding to one of them. In the delay conditions, listeners heard an instruction with a delay immediately before the target word. In the control conditions, the instruction also contained a delay, but earlier in the sentence. This led to instructions like those in (1).

(1a) Now press the button for the <delay> <target>, please

(1b) Now press the <delay> button for the <target>, please

We opted for a control condition with an early delay, so that the total time to target was constant across conditions, and so that both experimental and control stimuli contained exactly the same auditory materials (albeit in a different order). Thus, any difference between experimental and control conditions can only be ascribed to the difference in the position of the delay. Note that instructions with an invariant syntactic structure and invariant lexical content (bar the target item) have also been used in eyetracking experiments [15]. Additionally, to ensure maximal comparability across experiments we used the same acoustic token of the carrier sentence, and delays of exactly the same length in every condition of all experiments.

Experiment 1 examined the effect of a naturalistic disfluency, namely um. Experiment 2 tested whether any facilitative effect on word recognition is a specific property of fillers like um, or whether silent pauses can also help word recognition (as predicted by the delay hypothesis). Experiment 3 tested whether there is also facilitation when the delay is clearly not a naturalistic disfluency. We therefore used an artificially generated sine wave tone.

In each experiment, upon hearing an auditory instruction naming one of two depicted objects, participants had to respond by pressing one of two buttons (corresponding to the left-hand or right-hand object). Half of the instructions included a late delay, just before the object was named, and the other half included an earlier delay. To establish that the paradigm had sufficient power to reveal reaction time effects on auditory word recognition, each experiment also included a task difficulty manipulation. In the difficult condition, the target picture had a low-frequency (LF) name and was visually blurred. In the easy condition, the target picture had a high-frequency (HF) name and was visually intact. Thus the two factors manipulated orthogonally in a within-participants design were delay and task difficulty. In each experiment, recordings were made of times (relative to the onset of the target item name) taken for accurate responses to the instructions.

Results

We analysed our experiments using Generalized Linear Mixed-Effects models. Effects, and their probabilities, were estimated using 10,000 Markov chain Monte Carlo samples.

Experiment 1: Early vs. late um

Table 1 shows the participant mean correct reaction times. There was a main effect of delay, with participants 32 ms faster to respond in the delay (late um) condition (). There was a separate effect of task difficulty, with participants taking 63 ms longer to respond to blurred LF items (). Reaction time also decreased over the experiment, by an estimated mean of 1.5 ms per trial (). In this experiment there was a marginal interaction of trial number with frequency (, ), such that responses to blurred images with LF names speeded up by an additional 0.8 ms per trial (). There were no other significant interactions (all , ).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Experiment 1.

https://doi.org/10.1371/journal.pone.0019792.t001

Experiment 1 showed a clear recognition advantage when the word followed a local delay which was filled with an um (a pilot study showed a similar delay advantage in a comparison of the same late um condition as reported here with a completely fluent condition). The results extend those of Fox Tree [6] by showing that an um before a target can help recognition of that target, just as uh can. The findings are consistent with the delay hypothesis, but it is possible of course that the facilitation resulted from the nature of the delay (i.e., a delay containing a filler) rather than from the delay itself. Experiment 2 therefore substituted um with silence.

Experiment 2: Early vs. late silences

In this experiment, each um in the materials of Experiment 1 was replaced with a silence of the same length, resulting in auditory stimuli with a silence before the word button in the control condition, or before the target in the delay condition.

Table 2 shows the mean correct reaction times by participants. There was a main effect of delay, with participants 27 ms faster to respond in the delay (late um) condition (). The effect of task difficulty was also significant, with blurred LF items resulting in responses which were 51 ms longer (). Reaction time decreased over the experiment by an estimated mean of 2.9 ms per trial (). None of these effects interacted with each other (all , ).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Experiment 2.

https://doi.org/10.1371/journal.pone.0019792.t002

Experiment 2 showed that a silent pause immediately before a target word affects listeners in the same way as um did in the previous experiment, providing further evidence for the delay hypothesis. One might argue, however, that the facilitatory effect did not result from the delay, but from the fact that silent pauses, just like uhs and ums, occur as naturalistic disfluencies. Silent pauses fulfill several functions in discourse [16] and it is conceivable that they, just like uh and um, sometimes co-occur with production difficulties and therefore increase listeners' attention to the following word. We therefore put the delay hypothesis to the test again, but now used sounds that cannot be reasonably interpreted as naturalistic disfluencies, namely artificial tones (consisting of sine waves). The delay hypothesis predicts that tones directly preceding the target will still facilitate recognition, but any account on which the facilitation in the previous experiments results from listeners interpreting the content of delays as a signal predicts no effect.

Experiment 3: Early vs. late tones

Experiment 3 was a replication of Experiments 1–2 using delays filled with non-speech sounds. Each um in the original materials used for Experiment 1 was replaced with a sine wave tone of the same duration, and with a frequency (400 Hz) chosen to approximate the baseline frequency of the recorded speaker.

Table 3 shows the mean correct reaction times by participants. There was a main effect of delay, with participants 56 ms faster to respond in the delay (late um) condition (). The effect of task difficulty was also significant, with blurred LF items resulting in responses which were 44 ms longer (). Reaction time decreased over the experiment by an estimated mean of 1.1 ms per trial (). None of these effects interacted with each other (all , ).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Experiment 3.

https://doi.org/10.1371/journal.pone.0019792.t003

Discussion

In each of three experiments we found clear effects of delays on word recognition. It did not matter whether such delays were filled with um (Experiment 1), were silent (Experiment 2), or were filled with a tone that was clearly not part of speech (Experiment 3). A conclusion that stands out from this work is that any delay in word onset can help word recognition. This has an important theoretical implication. The current data certainly do not rule out that listeners are sensitive to the distributional properties of speech following fillers like uh, um, and the like, so that they can predict when the next word will follow or even what word will follow. However, our data do show that it is not necessary to postulate such sensitivity. The benefits of fillers on word recognition can just as easily be explained in terms of the delay that such fillers create.

Our findings contrast with an earlier study by Fox Tree which showed an uh-advantage, but no um-advantage in a word-spotting task[6]. This was interpreted as a consequence of the differences in delay that uh and um would signal[6], [14]. However, the present study did find an effect of um. A possible reason for this difference in findings is that the ums in Fox Tree's study were preceded and followed by lengthy silent pauses, which were left in the “fluent” control stimuli. Our Experiment 2 shows that silent delays also facilitate word recognition. This may have masked any effect of um in the earlier study.

It is true of course that our study differed from that of Fox Tree in several further ways. The most striking difference is probably that Fox Tree used naturalistic utterances, with different carrier sentences and different uhs and ums on every disfluent trial, whereas we used one and the same carrier sentence, and always used the same delay (i.e., same um, silence, or tone). Fox Tree's choice of stimuli undoubtedly promoted ecological validity more than ours, but this of course traded-off with experimental control. Whereas our design allows us to directly compare the effects of different types of delay (because everything else was held constant across experiments, except for the random factor participants), it is rather difficult to directly compare Fox Tree's uh and um conditions, because the carrier sentences and target words were different, the uhs and ums varied in length, and the pauses before and after um varied too[6].

Two sets of studies have, however, used different types of delays in circumstances which allow for direct comparison of their effects. In one study, participants were asked to judge the grammaticality of recorded utterances which included spoken disfluencies (uh uh) or “environmental noises” (such as dog barks) in positions which were expected either to facilitate or interfere with understanding [17]. In each of two experiments, the effects of environmental noise were shown to be effectively the same as those of disfluency. A second set of studies examined ERP responses to words following silent [18] and disfluent uh [10] delays. In each case, the N400 component associated with contextually less predictable words was attenuated following a delay. Taken together, these studies suggest that different types of hesitation can be shown to have similar effects across a variety of paradigms and materials, providing converging evidence that the form of a delay to the spoken signal may be less important than the time taken.

One potential caveat with such studies is that they tend to repeat acoustic tokens in order to achieve experimental control. In the present study our use of a fixed token of the carrier sentence, and a fixed token of um, may have led the participants to process the um and following words in ways that differ from the normal listening situation. Specifically, in normal listening situations, listeners might interpret um and the like as a signal of upcoming delay (which heightens attention in ways appropriate for the ‘meanings’ of the particular disfluencies). However, because in our experiments the target word almost immediately followed the um, and because listener sensitivity to session-local distributional properties of um would gradually overwrite their global sensitivities, listeners would, in the course of a session, stop expecting a further delay following um. Including the effects of trial number in our analyses provides a direct test of whether this caveat threatens our conclusions. Specifically, any account on which listeners pick up on the properties of our materials would need to further assume that the effects of delays change over the course of the experimental session. As is often the case in reaction time experiments, the analyses showed that responses became somewhat faster over the course of the experiment, but in none of the experiments was there an interaction between item number and delay, showing that participants' responses to delays did not change over time.

Our main conclusion is that delays in word onset facilitate word recognition, and that such facilitation is independent of the type of delay. There are several reasons why a delay in itself might help. On one account, delays help low-level speech segmentation processes. Because running speech often contains no clear word boundaries, the segmentation process has to figure out where one word ends and the next one begins, as illustrated by the classic example I scream for ice cream. Obviously, this segmentation problem can be reduced when there are delays between words. However, we do not think a segmentation account can explain our results. In particular, in our experimental condition, the um always occurred between the and the target item. If the segmentation account is correct, then listeners should encounter difficulties segmenting the string of sounds consisting of the and the initial sound(s) of the target item. The cohort of words starting with the, however, is very small, and only contains words in which the schwa is followed by a /t/ (e.g., that), an /m/ (themselves), or a /w/ (the one). Only one target word (tree) started with one of those consonants and could therefore have led to segmentation problems.

Another reason why delay helps word recognition may be that delay allows time for any top-down processes to affect recognition processes. Visual-world eyetracking experiments suggest that listeners, when hearing speech in a visual context, make linguistic predictions about upcoming references [19]–[21]. Given the ubiquity of NPs consisting of a determiner and noun in the language, it is possible that determiners lead to the prediction of nouns (and help subsequent identification of nouns), and that these predictions become more effective as more time passes.

Finally, it is possible that delays do not affect the mechanisms of word recognition themselves, but affect an attentional modulation of recognition processes. On such an account, any delay in speech will lead to a transient increase in attention, so that the next word can be more readily identified. Consistent with this account, it has been shown that stimuli containing an uh, as compared to fluent controls, modulated the amplitude of the mismatch negativity and P300 components in the ERP signal[9]. It is well established that these components are sensitive to variations in attention. Additionally, in subsequent memory tests, words that had followed uh were recalled better than control words, which is again consistent with an attentional account[9], [10].

Our findings support the perhaps counterintuitive conclusion that fillers like um can sometimes help (rather than hinder) listeners to identify spoken words. But critically, the data show that the same is true for silent pauses and pauses filled with artificially generated tones. It thus seems unnecessary to postulate listener sensitivity to the distributional properties of pause durations after fillers to explain why fillers help.

Materials and Methods

Acknowledgments

The authors wish to thank Evelien Akker, who designed and ran an earlier version of this work (not reported here) when at Edinburgh. The authors would also like to thank Hannah Furness, Jo Goll, and Katharine Nierop for help with data collection, and Manon Jones for providing the ‘voice’ used in the experiments reported.