Conceived and designed the experiments: LT SRA MAG. Performed the experiments: LT SRA. Analyzed the data: LT SRA. Contributed reagents/materials/analysis tools: LT. Wrote the paper: LT SRA MAG.
The authors have declared that no competing interests exist.
A small number of blind people are adept at echolocating silent objects simply by producing mouth clicks and listening to the returning echoes. Yet the neural architecture underlying this type of aid-free human echolocation has not been investigated. To tackle this question, we recruited echolocation experts, one early- and one late-blind, and measured functional brain activity in each of them while they listened to their own echolocation sounds.
When we compared brain activity for sounds that contained both clicks and the returning echoes with brain activity for control sounds that did not contain the echoes, but were otherwise acoustically matched, we found activity in calcarine cortex in both individuals. Importantly, for the same comparison, we did not observe a difference in activity in auditory cortex. In the early-blind, but not the late-blind participant, we also found that the calcarine activity was greater for echoes reflected from surfaces located in contralateral space. Finally, in both individuals, we found activation in middle temporal and nearby cortical regions when they listened to echoes reflected from moving targets.
These findings suggest that processing of click-echoes recruits brain regions typically devoted to vision rather than audition in both early and late blind echolocation experts.
Research has shown that people, like many animals, are capable of using reflected sound waves (i.e. echoes) to perceive attributes of their silent physical environment (for reviews see
To this point, research into natural human echolocation has been exclusively behavioural in nature. As a consequence, the neural processes underlying this ability are completely unknown. Some expectations about these mechanisms can be gathered from a positron emission tomography (PET) study
Two blind skilled echolocators participated in the current study. Participant EB (43 years at time of testing) had partial vision up to 13 months of age. At 13 months, his eyes were removed due to retinoblastoma (early onset blindness). Participant LB (27 years at time of testing) lost vision at age 14 years due to optic nerve atrophy (late onset blindness). Both were right-handed, had normal hearing and normal auditory source localization abilities (
The data show that the presence of echoes within a train of complex sounds increases BOLD signal in calcarine cortex in both EB and LB. This increase in activity in calcarine cortex is absent in C1 and C2. Importantly, the presence of echoes within a train of complex sounds does not lead to an increase in BOLD signal in auditory cortex in any of the four participants. This finding suggests that brain structures that process visual information in sighted people process echo information in blind echolocation experts.
To overcome the difficulties posed by studying echolocation in an MRI environment (i.e., hearing protection must be worn, head and mouth movements must be minimized, etc.), a passive listening paradigm was adopted whereby the echolocation clicks and their echoes were pre-recorded in the listener's ears (
To obtain stimuli that would elicit strong echolocation percepts, we recorded echolocation clicks and echoes from EB and LB outside of the MRI under three scenarios: i) as they sat in an anechoic chamber in front of a concave or flat surface that was placed 40 cm in front of them and 20° to the left or right (for examples of sounds used during the experiment listen to
Functional MRI revealed reliable blood-oxygen-level dependent (BOLD) activity in auditory cortex as well as in the calcarine sulcus and surrounding regions of “visual” cortex in EB and LB when they listened to recordings of their echolocation clicks and echoes, as compared to silence (
Concavities and convexities are colored dark and light, respectively. CS-central sulcus, CaS-calcarine sulcus, LS- lateral sulcus, MFS – middle frontal sulcus. Top panel: BOLD activity while EB and LB listened to recordings of their own echolocation sounds that had been made in an anechoic chamber and judged the location (left vs. right), shape (concave vs. flat) or stability (moving vs. stationary) of the sound reflecting surface (see
Remarkably, however, when we compared BOLD activation to outdoor recordings that contained click echoes with activation to outdoor recordings without echoes, activity disappeared in EB and LB's auditory cortex, but remained in calcarine cortex (
Marking of cortical surfaces and abbreviations as in
The lack of any difference in activity in auditory cortex in all the participants for the contrast between outdoor recordings with and without echoes was not unexpected, because we had created echolocation and control stimuli so that the acoustic differences were minimal and the only difference was the presence or absence of very faint echoes (
Given the echo related activation of calcarine cortex in both EB and LB, the question arises as to whether the echo related activity in calcarine cortex shows a contralateral preference – as is the case for light related activity in calcarine cortex in the sighted brain. To test this, we performed a region of interest analysis that compared BOLD activity in left and right calcarine in response to echolocation stimuli that contained echoes from surfaces located on the left or right side of space. For comparison, we also applied this analysis to the left and right auditory cortex. Previous fMRI research has shown a contralateral bias in auditory cortex for monoaural stimulation
Regions of interest (ROI) were defined based on anatomical and functional criteria. For illustration purposes, we show projections of ROI on the partially inflated cortical surfaces. However, all statistical analyses were performed in volume space. Bar graphs indicate beta values for the various ROIs. Gray and white bars indicate beta weights for ‘echo from surface on left’ and ‘echo from surface on right’, respectively, averaged across voxels within each ROI. Colored bars denote the difference between beta weights within each brain side (red bars indicate higher beta values for ‘echo from surface on right’; blue bars the reverse). Error bars denote SEM. To determine if activity during echolocation exhibits a contralateral preference, we applied independent measures ANOVA to the beta weights with ‘echo side’ (i.e. ‘echo from surface on left’ vs. ‘echo from surface on right’) and ‘brain side’ (e.g. ‘left calcarine’ vs. ‘right calcarine’) as factors to each ROI. ANOVA results are summarized below each bar graph. Results show that activity in calcarine cortex exhibits contralateral preference for EB (significant interaction effect), but not LB. Activity in auditory cortex shows neither contra- nor ipsilateral preference in either subject. For both EB and LB, beta values in the right calcarine exceed those in the left calcarine (main effect of ‘brain side’).
Finally, we also examined BOLD activity related to echolocation stimuli that conveyed object movement with activity related to stimuli that did not convey such movement in both the blind and the sighted participants. Both EB and LB showed activity in areas of the temporal lobe commonly associated with motion processing (
Concavities and convexities are colored dark and light, respectively. STS-superior temporal sulcus, ITS -inferior temporal sulcus, LOS – lateral occipital sulcus.
The comparison between concave vs. flat conditions, as well as the comparison between tree vs. car vs. pole did not reveal significant differences. It is evident from the behavioural data, that EB and LB certainly perceived these conditions as different; so at some level, there must be a difference in neural activity. It is likely that the temporal and spatial resolution of our paradigm was not able to detect these differences.
It is well established that the cerebellum is involved in the control and coordination of movement, and there is also mounting evidence that the cerebellum may be involved in higher order cognitive function (for reviews see
When EB and LB listened to recordings of their echolocation clicks and echoes, as compared to silence, they both showed significant BOLD activity in lobules VI and VIII (
Data are shown in neurological convention, i.e. left is left. Activity in the cerebellum was analyzed in stereotaxic space
The result was different, however, when we compared BOLD activation to outdoor recordings that contained click echoes with activation to outdoor recordings that did not contain echoes. Specifically, this analysis did not reveal any differential activity anywhere in the cerebellum for the two sighted control subjects C1 and C2. In contrast, for both EB and LB, this analysis revealed differential activity in lobule X and lobule VIIAt/Crus II (
Of course, when we compared activation associated with both the outdoor echolocation and control recordings as compared to silence, the pattern of activity in the cerebellum was very similar to when we compared activation associated with echolocation sounds to activation associated with silence (
The comparison between concave vs. flat conditions, as well as the comparison between tree vs. car vs. pole did not reveal significant differences
Here we show that two blind individuals can use echolocation to determine the shape, motion and location of objects with great accuracy, even when only listening passively to echolocation sounds that were recorded earlier. When these recordings were presented during fMRI scanning, we found that ‘visual’ cortex was strongly activated in one early blind participant (EB) and to a lesser degree in one late blind participant (LB). Most remarkably, the comparison of brain activity during sounds that contained echoes with brain activity during control sounds that did not contain echoes revealed echo related activity in calcarine, but not auditory cortex.
The question arises if the activity that we observe in calcarine cortex is truly related to echolocation, or if it is simply due to the fact that EB and LB are blind. Blindness can result in re-organization of many brain areas, including but not limited to visual, auditory and somatosensory cortex and subcortical structures, even though the underlying mechanism and exact nature of the changes are still unclear
It would be useful in future neuroimaging studies of echolocation to include sighted people who have been trained to echolocate, or blind people who have a ‘regular’ sensitivity to echoes. With respect to the latter, there is evidence that blind people, even when they do not consciously echolocate, are more sensitive to echoes than sighted people
The patterns of activation observed in their brains might shed some light on the possible role that sensory deprivation plays in the recruitment of visual cortex during echolocation in the blind. On the behavioural level, of course, sighted people's echolocation abilities have been repeatedly shown to be inferior to those of blind people (for reviews see
It is important to emphasize that the use of echolocation in the blind goes well beyond localizing objects in the environment. The experts we studied were also able to use echolocation to perceive object shape and motion – and even object identity. In addition, they were able to use passive listening with 10-kHz cut-off to do these kinds of tasks – which made it possible for us to probe neural substrates of their abilities. Clearly more work is needed comparing performance with active and passive echolocation across a range of different tasks – where the available frequency ranges in both conditions are systematically varied.
It could be argued that the contralateral bias that we observed in EB's calcarine cortex reflects differences in spatial attention between the two conditions. Effects of attention on brain activity have been shown for visual
Both EB and LB show BOLD activity in temporal cortical regions typically devoted to motion processing, but this activity is absent in C1 and C2. In a similar fashion, both EB and LB reported to perceive motion, but this percept was absent in C1 and C2. Thus, we see good correspondence in terms of brain activity and perception. The question remains, however, as to what the ‘preferred modality’ of the neurons is that are active in EB and LB when they perceive motion using echolocation. Neurons adjacent and inferior to the ITS/LOS junction are sensitive to both visual and auditory motion as determined with functional localization techniques
An obvious question that arises from our findings is what function calcarine cortex might serve during echolocation. One possibility is that it is involved in the comparison between outgoing source sound (e.g. mouth click) and incoming echo. This explanation seems unlikely, however, because if the calcarine computed a comparison between outgoing source sound and incoming echo, it would also compute that comparison in the absence of echoes. If that were the case, however, we would expect the calcarine to be equally active in the presence and the absence of echoes – provided the corresponding clicks were present. The pattern of activity we found in EB and LB does not support this interpretation (
The cerebellar structures linked to visual sensory processing
In addition to lobule X, we also found activity in left lobule VIIAt/Crus II during echolocation. Since this part of the cerebellum is involved in a non-motor loop involving Brodmann area 46 in pre-frontal cortex
The current study is the first to investigate which brain areas potentially underlie natural echolocation in early- and late-blind people (EB and LB). In EB, we found robust echolocation-specific activity in calcarine cortex – but not in auditory cortex. A similar pattern was observed in LB, but the activity in the calcarine cortex was not as extensive. We also found that the calcarine activity was greater for echoes reflected from surfaces located in contralateral space in EB but not LB. Our findings also shed new light on how the cerebellum might be involved in sensory processing. In addition, our study introduced novel methodology that can be used in future experiments on echolocation.
From a more applied point of view, our data clearly show that EB and LB use echolocation in a way that seems uncannily similar to vision. In this way, our study shows that echolocation can provide blind people with a high degree of independence and self-reliance in their daily life. This has broad practical implications in that echolocation is a trainable skill that can potentially offer powerful and liberating opportunities for blind and vision-impaired people.
All testing procedures were approved by the ethics board at the University of Western Ontario, and participants gave written informed consent prior to testing. The consent form was read to participants, and the location to sign was indicated manually.
Software used to conduct testing was programmed using Psychophysics toolbox 2.54
All imaging was performed at the Robarts Research Institute (London, Ontario, Canada) on a 3-Tesla, whole-body MRI system (Magnetom Tim Trio; Siemens, Erlangen, Germany) using a 32-channel head coil.
(see
Standard routines were employed for fMRI data pre-processing, coregistration and cortical surface reconstruction (see
From this analysis we obtained a separate beta value for ‘left’ and ‘right’ predictors for each voxel. To determine if there was a right or left echo preference in the left or right portion of the calcarine sulcus or Heschl's gyrus, we subjected those beta values to an ANOVA with ‘brain side’ and ‘echo side’ as independent factors, separately for the calcarine sulcus and Heschl's gyrus. Technically, we could have used the number of beta values to determine error degrees of freedom (
In this way we could use data obtained from all voxels inside each ROI to determine interaction effects between ‘brain side’ and ‘echo side’ for each participant. In contrast, a traditional ROI analysis averages across voxels before applying the GLM, such that interaction effects can only be computed when data from multiple participants is available.
With the exception of the outdoor recordings, all auditory stimuli were recorded in the Beltone Anechoic Chamber at the National Centre for Audiology in London, Ontario, Canada, that was equipped with a 125 Hz cut-off wedge system on the walls and ceiling, and a vinyl covered concrete floor. Ambient noise recordings indicated a background noise (i.e., ‘noise floor’) of 18.6 dBA. The participant was seated in the center of the room. For each recording trial, the experimenters placed an object at a desired position, and then retreated to the back of the chamber (approximately 1.5 m behind the participant) before instructing the participant to start producing echolocation clicks. High-quality stereo recordings of the entire sessions' audio were acquired with the in-ear microphones and saved for off-line editing. EB and LB participated in separate recording sessions, i.e. during any recording session three people were in the room (two experimenters and one participant).
Stimulus recording for the Scenes experiments took place in a garden-style courtyard, approximately 40 m long by 20 m wide and surrounded by an elliptical driveway. Two thirds of the driveway was bordered by two-storey buildings (see
To determine angular position discrimination thresholds we employed a 2-Interval-2-AFC adaptive staircase method, with step-sizes in the first two trials computed based on
During passive listening we used the same procedure as during active echolocation with the exception that participants did not actively echolocate, but listened to recordings of their own clicks and echoes. See
Results of source localization experiment. Plotted on the ordinate is the probability that the participant judges the source to be located to the right of its straight ahead reference position. Plotted on the abscissa is the position of the test position with respect to the straight ahead in degrees. Negative numbers indicate a position shift in the counter clockwise direction. Psychometric functions were obtained by fitting a 3-parameter sigmoid to the data. 25% and 75% thresholds and bias (denoted in red) were estimated from fitted curves. The zero-bias line is drawn for comparison (dashed line). It is evident from the data that EB and LB can determine the angular position of a source with high accuracy, i.e., thresholds for EB and LB are 2° and 2.5°, respectively. The localization thresholds for both EB and LB are within the range of what has been reported for source localization thresholds of sighted participants with respect to a centrally located reference source (Blauert, 1998; page 39, table 2.1). For both EB and LB, performance is slightly better during source localization than during active or passive echolocation (compare
(TIF)
BOLD activity projected on participants reconstructed and partially inflated cortical surface. Shown is the contrast between activations for outdoor recordings containing echoes from objects, and outdoor recordings that did not contain such echoes, evaluated at a more liberal statistical threshold then in the main text, i.e. p<.01 instead of p<.001 (compare
(TIF)
BOLD activity projected on participants reconstructed and partially inflated cortical surface. Marking of cortical surfaces and abbreviations as in
(TIF)
BOLD activity in C1 and C2 brains that is related to recordings of echolocation sounds that convey movement to EB and LB, evaluated at a more liberal statistical threshold than reported in the main text, i.e. p<.01 instead of p<.001 (compare
(TIF)
BOLD activity in the cerebellum while participants listened to outdoor scene recordings (both echo and control sounds) and judged whether the recording contained echoes reflected from a car, tree or pole or no object echoes at all. Each EB and LB listened to recordings of his own clicks and echoes as well as to recordings of the other person. Similarly, each C1 and C2 listened to recordings he had trained with as well as to the recordings of the other person, e.g. C1 listened to both EB's and LB's recordings (see
(TIF)
Bird's eye view of the courtyard (highlighted in red) that was used to make outdoor scene recordings.
(TIF)
Illustrations of outdoor scenes used to make echolocation recordings (the participant stood in front of each object and made clicks) and background recordings used to make outdoor control sounds (the participant stood silently in front of each object).
(TIF)
Expanded Classification Results (incl. sample size) for location, shape, motion and outdoor scenes experiments for EB and LB. Asterisks indicate that performance is significantly different from chance (p<.05). Unless otherwise indicated, chance performance is 50%. Tests of significance were only computed for entries in black (also contained in the main text). Sample sizes (shown in parenthesis) fulfill minimum requirement for confidence intervals for a proportion based on the normal approximation
(DOC)
Expanded Classification Results (incl. sample size) for location, shape, motion and outdoor scenes experiments for C1 and C2. Asterisks indicate that performance is significantly different from chance (p<.05). Unless otherwise indicated, chance performance is 50%. Tests of significance were only computed for entries in black (also contained in the main text). Sample sizes (shown in parenthesis) fulfill minimum requirement for confidence intervals for a proportion based on the normal approximation
(DOC)
Statistical results of ROI analysis (contrast: EchoMoving−EchoStationary ) applied to area MT+ in C1 and C2. We applied regions of interest analysis to MT+ ROIs for both control participants to determine if the contrast EchoMoving−EchoStationary was significant (contrast values and SEM are shown in
(DOC)
Center-of-Gravity Talairach Coordinates for MT+ ROIs. For ROI selection methods see Methods S1.
(DOC)
Binaural recording of a click and click echoes made in EB's ears in the anechoic chamber, while he made a click in the presence of a position marker located 150 cm straight ahead. This sound accompanies
(WAV)
Binaural recording of a click and click echoes made in LB's ears in the anechoic chamber, while he made a click in the presence of a position marker located 150 cm straight ahead. This sound accompanies
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Illustrations of sounds used during
(WAV)
Summary of audiological test results for EB and LB (Air Conduction Thresholds, Tympanograms, Acoustic Reflex Thresholds, Distortion Product Otoacoustic Emissions).
(PDF)
Additional information about the experimental methods.
(DOC)
We thank Daniel Kish and Brian Bushway from World Access for the Blind, who acted as consultants throughout the experiments, providing invaluable technical and practical advice about echolocation and the nature of the testing materials we used. We thank D. Purcell for conducting the audiological testing. We thank J. Ween, G. Dutton, L. van Eimeren, and H. Yang for technical support and logistics.