Ethics Statement

The experiment was conducted in accordance with the Declaration of Helsinki. All participants gave their written informed consent prior to participation. The experimental protocol and consent forms were approved by the ethical commision of the medical faculty of the Eberhard-Karls University in Tübingen, Germany.

Apparatus

The experiment was performed using the Max-Planck Institute CyberMotion Simulator. This is an anthropomorphic robot arm fitted with an additional cabin (KUKA Roboter GmbH, Augsburg, Germany, model Robocoaster; [38]). The cabin has a double-curved projection screen with dual projectors, with a field of view of approximately 140° horizontal by 70° vertical. The dual projection system was used in conjunction with stereo glasses (Infitec®GmbH, Ulm, Germany, model INFITEC®Premium Glasses) to generate stereoscopic visual stimuli. The area of overlap between projectors used for stereo projection is approximately 90° by 70°. Participants were seated within the simulator cabin and secured by a five-point safety harness. Participants wore ear-enclosing headphones that dimmed simulator noise. An integrated microphone allowed permanent communication with the experimenter. Responses were provided via a custom-made pointer device. The pointer device consisted of a stainless steel rod of about 20cm that was connected to a potentiometer in its middle. One end of the rod was covered by 5cm of black heat shrink tubing. The device was placed on an aluminum frame right above the participant’s laps. Participants were instructed to hold the rod by the covered end with their right hand; the other end was to be interpreted as the arrow’s head. The rod could be rotated in the horizontal plane, did not have any discontinuities, and provided a < 0.1° resolution. It was visible to the participants during the response phase of all stimuli.

Task & Stimuli

Stimuli consisted of presentation of visual and/or inertial horizontal linear motions with different headings. For both modalities the motion profile was a raised cosine bell in velocity, specified as (1)

With ω = 2πf, f = 0.5Hz, and v_max = 0.3m/s (a_max ≈ 0.5m/s², x_max = 0.3m).

The visual motions consisted of movement through a three dimensional environment with a ground plane and limited-lifetime particles (Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Visual scene.

To provide disparity information, the visual stimuli were presented stereoscopically using a dual-projector setup. The figure only shows the image presented by one projector.

https://doi.org/10.1371/journal.pone.0127104.g001

The ground plane was blurred to make it virtually impossible for participants to determine the heading of visual motion by tracking a particular point. The particles appeared at random times and positions in a large spherical volume around the main camera. Each particle disappeared after a 0.3s lifetime. Visual stimuli were generated using the Unity game engine (Unity Technologies, San Francisco, United States, version Unity 4.2.2f1), and presented stereoscopically. The particularities of the visual environment were determined in pilot experiments, and chosen such that performance in terms of response variability in the visual- and inertial-only conditions was approximately equal.

Inertial motions were generated by moving the simulator cabin either forward or backward over a distance of 0.3m along an imaginary horizontal line. Different headings were achieved by varying the cabin’s yaw angle relative to this line (Fig 2). The direction in which the simulator was moved (forward or backward) was chosen separately for each stimulus, in such a way that the duration of reorientation between stimuli was minimal (See Supporting Information S1 Video and S2 Video for animations of forward and backward motions, respectively).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Operationalization of heading angle.

The participant was oriented at angle θ relative to the simulator motion path (represented by black arrow). For straight ahead motion, the participant’s naso-occipital axis (dashed gray line) would be aligned with the simulator motion path.

https://doi.org/10.1371/journal.pone.0127104.g002

There were three different experimental conditions: a visual-only condition, an inertial-only condition, and a ‘combined’ condition.

In both the visual- and inertial-only conditions, stimuli were presented with headings ranging between −180° (−π rad) and 180° (π rad), where 0 heading corresponds to straight-ahead motion, and positive values are orientated away from 0 in the clockwise direction. Per condition 180 stimuli were presented, with headings in the aforementioned range distributed in evenly spaced steps.

The stimuli presented in the combined condition were 180 discrepant combinations of visual and inertial motions, randomly generated for each participant. Visual motions with headings evenly spaced between −180° and 180° (every 2°) were coupled with inertial motions with headings that were the sum of the visual heading and a deviation drawn from a normal distribution with a mean of 0 and a variance of 50°, resulting in an effective maximum discrepancy of ${90}^{°}(\frac{1}{2}\pi \mathrm{\text{rad}})$. This range includes discrepancies that are below as well as above the threshold for detection of a discrepancy, given that 50% correct detection of a discrepancy between a visual motion stimulus with 0° heading and an inertial motion stimulus has been estimated to occur at ±14° (0.24, rad) [39].

The task participants had to perform was to use the pointer device to indicate the heading of self-motion after playback of each stimulus. Because the visual-only stimuli did not induce a sensation of actual self-motion, participants received the additional instruction to indicate the heading of self-motion that was suggested by the visual stimulus for visual-only trials.

Procedure

Prior to the experiment, participants were given a general explanation of the experimental goals and procedures, and given the opportunity to ask additional questions. They were instructed to keep their gaze focused on the central part of the projection screen, and to give their heading judgments as quickly and accurately as possible after a motion was completed. A fixation cross was not implemented. It has been shown that a fixation cross does not affect inertial heading estimation [40, 41]. Moreover, on a theoretical basis, a fixation cross could be used to track a particular element of the optic array relative to the fixation cross, changing the nature of the task. After giving their informed consent, participants completed a series of 10 random stimuli in order to get familiar with the setup and the task.

After familiarization, the actual experiment started. Stimuli of the visual-only, inertial-only and combined conditions were presented intermixed, in a random order that was defined at onset of the experiment. Playback of stimuli with visual motion was preceded by a high pitched beep (0.5s, 1760Hz); stimuli without visual motion were preceded by a low-pitched beep (0.5s, 440Hz). Participants were instructed to keep their eyes closed for stimuli without a visual component (i.e., after a low-pitched beep). Participants were monitored during the training phase of the experiment and reminded of these instructions if they did not follow them.

At the onset of each stimulus, the cabin was realigned to the stimulus’ heading angle, or, in case the stimulus was visual-only, the heading angle of the first next stimulus with an inertial motion component. The duration of this reorientation depended on the difference between the simulator’s current heading angle and the heading angle of the stimulus, and the direction of linear motion of the simulator cabin. Cabin motion could be forward or backward along the simulator motion path, which was fixed in space. Because the order of stimuli, their nature (i.e., visual-only, inertial-only, or combined), and the direction of the linear motion were randomized, the duration of reorientation could not be used as a cue to estimate the heading of the next stimulus.

The maximum angular acceleration and velocity for reorientation were 4.98°/s² (0.087 rad/s²) and 10.03°/s (0.175 rad/s), respectively. A 2s break was implemented after the reorientation phase to allow any after-effects to fade.

The total number of 540 stimuli were presented in four blocks of approximately 135 trials, where a trial constitutes a stimulus-response cycle. Each block took about 30 minutes to complete. Including instruction and mandatory breaks between blocks, the experiment took between 4–4.5 hours to complete.

Participants

Twelve participants (5F) were recruited to take part in the experiment. The participants all reported minimal susceptibility to motion sickness and claustrophobia, and no history of any intestinal, neurological, or vestibular illnesses. Due to safety regulations, participation was only allowed for people between 18–65 years old, weighing up to 90kg, and measuring at most 1.95m. Data of three participants had to be excluded from the analysis due to either an apparent misinterpretation of the instructions (2 cases: responses were limited to the cardinal axes), or due to an inability to discern stimulus headings (zero correlation between stimulus heading and response heading). Because of the nature of the responses provided by these participants, we could not characterize their unisensory heading estimation, and consequently could not assess the nature of multisensory processing.

Data Analysis

Analysis of the data was completed in two steps. In the first step, models of unisensory heading estimation were fitted to the data obtained in the unisensory conditions. In the second step of the analysis, the tenability of a number of models of multisensory heading was assessed, which relied on the parameters obtained in the first step. The analyses were performed separately for each participant.

Unisensory Models The model of unisensory heading estimation was: (2) with (2a) (2b)

Here R represents the response, which is assumed to be equal to the output of the sensory system, signal x. ℳ(μ, κ) represents a von Mises distributed random variable with mean μ and dispersion parameter κ. The von Mises distribution is a circular analogue of the normal distribution [42]. The mean μ is a function of the heading angle θ of the stimulus (in radians). The mean is composed of a parameter that reflects any constant offset in responses β₀ and the four-quadrant inverse tangent of y = β₁ sin θ and x = cos θ. Parameter β₀ was assumed to be zero, but was included to allow for any deviations; parameter β₁ affects the size of bias. To illustrate: a value of β₁ = 1 indicates that there is no heading dependent bias; values of β₁ < 1 indicate bias toward the fore-aft axis; and values of β₁ > 1 indicate bias away from the fore-aft axis. The dispersion parameter κ is an analogue of the reliability $\frac{1}{{\sigma }^2}$ of a Normal distribution. It was expressed as the sum of a baseline level γ₀ and a part that varies periodically with heading angle, with parameter γ₁. The periodical nature of the bias and the dispersion in the model mimics experimental observations for patterns of bias and dispersion of both visual and vestibular heading estimation [26, 41, 43]. We assume that the CNS does not have any knowledge of these biases, as known biases can be compensated for. It should be noted that this model serves to characterize unisensory heading estimates, but does not reflect any assumptions on underlying generative mechanisms. For both unisensory conditions, there were four free parameters: β₀, β₁, γ₀, and γ₁. We assume that the likelihood of the parameters has a single strong peak, and consequently determine the optimal value for each of the parameters by maximum-likelihood estimation, using the MATLAB fminsearch routine (The MathWorks Inc., Natick, United States, version 2013b). To assess the goodness of fit, we calculated the Generalized Coefficient of Determination, R². This measure compares the model likelihood to the likelihood of a null-model, which assumes zero bias μ = θ, and κ as a constant [44]. This measure ranges between zero and one, corresponding to a lack of improvement and strong improvement compared to the null model, respectively.

Multisensory Models In the second step of the analysis, the fit of several alternative models of multisensory heading estimation was compared in order to assess how to best explain the observed multisensory heading estimates. The analysis is largely analogous to the approach taken by [34], adapted for circular stimuli. The models come from a ‘spectrum’ of strategies that reflect how the CNS could estimate heading when visual and inertial information is simultaneously available. A schematic for the models is provided in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Schematic of the models.

Sensory neurons are excited by stimuli θ_V and θ_I, and pass representations of the stimuli on to the CNS. These internal representations carry average estimates x_V and x_I, as well as information about their dispersions, κ_V and κ_I. The probability that the internal representations share a common cause P(C | …) is assessed; a fused estimate ρ_FF is generated; and an individual representation ρ_SS is chosen. A final estimate R is generated by processing intermediate estimates ρ_FF and ρ_SS according to a particular strategy. These strategies take the estimated probability of a common cause into account.

https://doi.org/10.1371/journal.pone.0127104.g003

On one end of the spectrum of models, the CNS fuses the signals provided by the visual and inertial system in a statistically optimal fashion, regardless of any discrepancy between them. By ‘statistically optimal’ we mean that the expected value of a certain cost function c of heading estimation error R−S is minimized given the a-posteriori belief about the heading S: (3)

It can be shown that if a) c is an even function and b) P(S | x_V, x_I, κ_V, κ_I) is unimodal and symmetric around its point of maximum μ, then the expression above is minimized if R = μ. The posterior is given by (4)

As explained in the previous section for the unisensory case, we assume that the CNS treats visual and inertial representations of heading as veridical: it does not have any knowledge of sensory bias, and it is also unaware of the relation between heading angle and perceptual noise. Therefore, we assume that the belief about S does not depend on κ_V, κ_I: P(S | κ_V, κ_I) = P(S). Stated differently, this implies that κ_V, κ_I do not provide additional information about S.

The dispersions κ_V and κ_I are treated as knowns, as it has been shown that the dispersion is present in the activity of populations of neurons (e.g., [45]). We calculated their respective sizes using the unisensory models, with values for parameters β₀, β₁, γ₀, and γ₁ estimated from the unisensory data. Given that x_V and x_I are independent, the posterior becomes a product of two von Mises distributions, which is itself von Mises distributed with parameters μ_VI, κ_VI: (5) where f_VM(⋅ | μ, κ) represents the von Mises probability density with mean parameter μ and dispersion parameter κ, and x_V and x_I are determined as in the unisensory models. The resulting distribution satisfies the conditions of being unimodal and symmetric around its mean μ_VI. Therefore we take μ_VI as an estimator of S: (6)

This strategy will be henceforth referred to as Forced Fusion (FF). It is equivalent to Bayesian Integration schemes using a uniform prior, known as Maximum-Likelihood Integration, but uses circular distributions (e.g., [21–24]). A notable difference of the present implementation compared to models using Gaussian distributions is that the combined estimate can be less certain than the constituent estimates. For a detailed account on cue combination for circular data see [36].

On the other end of the spectrum of models, the CNS does not fuse visual and inertial signals at all, but relies on either source of information to construct response R. In the present experiment, participants were instructed to report the perceived heading of self-motion, thus possibly ignoring the visual stimulus. However, since previous work on cue combination in self-motion perception has shown that people may alternate between sources of information regardless of their instructions [32, 46], we allowed the source of the heading estimate to vary on a trial-to-trial basis. This strategy will be referred to as the Switching Strategy (SS; [46]), and can be thought of as the CNS flipping a biased coin to decide between the visual signal x_V and inertial signal x_I: (7)

The probability of choosing a particular source is likely to depend on the task or instructions given to a participant. Consequently, we assumed that this parameter was constant throughout the experiment. Because its value was unknown and may differ between participants, probability ξ was treated as a free parameter.

In between the FF and SS models as extreme cases come Causal Inference (CI) models [33, 34]. In CI models, an assessment of the likelihood of alternative causal structures (i.e., a common cause or independent causes for x_V and x_I) affects how the final estimate is constructed. There are myriad ways for the CNS to incorporate such an assessment in the construction of a final estimate. Plausible alternatives are Model Averaging (MA), Model Selection (MS), and Probability Matching (PM) [34, 35]. MA states that the final estimate is a weighted average of intermediate estimates ρ_FF and ρ_SS according to the FF and SS strategies, respectively. The weights are the respective probabilities of a common cause and independent causes. Because of the circular nature of heading estimates, we cannot simply take a weighted sum of the intermediate estimates. A weighted average of the intermediate estimates causes erroneous results at the extremes of possible headings (i.e. −π and π): consider the case where FF would yield an estimate of $-\frac{3}{4}\pi$ and SS an estimate of $\frac{3}{4}\pi$. The arithmetic mean would be zero, while intuitively, the mean should be π (or −π). Therefore, we represent the intermediate heading estimates as complex numbers, take a weighted sum, and revert the resulting complex representation z back to a heading angle: (8) where (8a) and (8b) (8c)

Here P(C | x_V, x_I, κ_V, κ_I) represents the probability of a common cause given signals x_V and x_I and their respective dispersions κ_V and κ_I, and $\mathrm{\text{P}}(\overline{\mathrm{\text{C}}}|x_V,x_I,{\kappa }_V,{\kappa }_I)$ represents the probability of independent causes. A discussion on the properties and calculation of the probability of a common cause given a-priori beliefs and sensory information P(C | x_V, x_I, κ_V, κ_I) is provided in the section Probability of causal structures.

For MS, the final estimate is either the result of FF or SS, depending on which causal structure is more likely: a common cause or independent causes, respectively. (9)

The PM decision strategy is very similar to MS, but the decision to base the final estimate on either SS or FF is made at random, where the probability of deciding in favor of SS is equal to the probability of a common cause. (10)

Probability of causal structures There are two possible causal structures for two sensory signals: either the signals share a common cause, C, or they do not, $\overline{\mathrm{\text{C}}}$. The probability of a common cause given a set of sensory signals P(C | x_V, x_I, κ_V, κ_I) can be found using Bayes’ rule: (11)

Here P(C) is the prior probability of a common cause, which we assume to be independent of κ_V, κ_I. It should be noted that parameter P(C) only represents an observer’s a-priori beliefs about whether or not cues will have a common cause (i.e., in the absence of any evidence). The a-posteriori probability of visual and inertial signals having a common cause is proportional to the a-priori beliefs on the one hand and the particular values and dispersions of the internal signals on the other. A value of parameter P(C) close to 1 does not imply that the model behaves almost identically to the Forced Fusion model: when this parameter is close, but not equal to one, this allows the model to attribute high common cause probabilities to multisensory stimuli with small discrepancies, and near zero common cause probabilities to multisensory stimuli with larger discrepancies, and allows for a different decision strategy when the sensory signals provide strong evidence that the stimuli are incongruent (i.e., P(x_V, x_I | C, κ_V, κ_I) is close to 0).

The probability of a set of sensory signals x_V and x_I given a common cause can be found by calculating the integral of the probability of the signals over all possible values for S: (12)

Assuming a uniform prior (i.e., all heading angles are equally likely to occur) and independence of the visual and inertial channels, this becomes (13a) (13b)

Here I₀(…) is the modified Bessel function of the first kind and order zero. κ_V and κ_I are treated as if they are independent of S, which reflects the assumption that any relation between the size of the dispersion and heading angle is unknown to the CNS (see: Unisensory Models). Put in words, the probability of a common cause given a set of sensory signals is equal to the total probability that any common cause S gave rise to both sensory signals. The less noisy the individual signals are, the more narrow the range of discrepancies that lead to a ‘common cause’ judgment will be.

Similarly, provided that the sensory channels are independent, the probability of sensory signals x_V and x_I given independent causes, $\mathrm{\text{P}}(x_V,x_I|\overline{\mathrm{\text{C}}},{\kappa }_V,{\kappa }_I)$, is (14) With uniform priors, this becomes (15)

The last expression means that assuming no prior information and incongruent stimuli, all combinations of sensory outputs x_V and x_I are equiprobable (i.e., a 2-dimensional uniform distribution for circular data).

An illustration of how P(C | x_V, x_I, κ_V, κ_I) varies for different combinations of θ_V, θ_I, the visual and inertial heading angles, is provided in Fig 4. Note that for values for P(C) of 0 or 1, the surface becomes flat.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Probability of a common cause.

The value of P(C | x_V, x_I, κ_V, κ_I) as a function of θ_V, θ_I; the visual and inertial heading angles, respectively. Values calculated using data of an example participant (7).

https://doi.org/10.1371/journal.pone.0127104.g004

Model Fitting & Selection In the models of multisensory heading estimation, the prior probability of a common cause P(C) and reliance on the visual signal ξ for the SS-component were free parameters. Because the likelihoods of the CI models are intractable, we could not obtain estimates for these parameters analytically: the model likelihood is $P(R|{\theta }_V,{\theta }_I)={\int }_{-\pi }^{\pi }{\int }_{-\pi }^{\pi }P(R|x_V,x_I)P(x_V,x_I|{\theta }_V,{\theta }_I)d{x}_{V}d{x}_I$, where P(x_V, x_I | θ_V, θ_I) = f_VM(x_V;μ_V(x_V), κ_V(x_V))f_VM(x_I; μ_I(x_I), κ_I(x_I)). μ_V, κ_V, μ_I, κ_I are calculated from Eqs (2a) and (2b), and P(R | x_V, x_I) is calculated based on Eqs (7)–(15), depending on decision strategy. We could not compute the resulting integrals analytically. We therefore used the models specified above to generate large sets of simulated responses for given sets of parameters P(C) and ξ, and assessed the similarity of the simulations with the actual observations. Specifically, 1000 responses were simulated for each of the approximately 180 completed stimulus-pairs. For each stimulus pair, the likelihood of the actual response was derived from kernel density estimation using the simulated responses [47, 48]. The ‘circ_ksdensity’ kernel density estimation procedure for periodic data made for MATLAB was used to obtain density estimates [49]. The sum of the logarithm of the estimated probability densities for each of the responses forms the model’s log-likelihood. The optimal values for P(C) and ξ are those for which the likelihood attains its maximum value. Optimization was done using the MATLAB ‘patternsearch’ algorithm.

Because the fit of each model depends to some extent on the particular values of the simulated responses, the complete procedure was repeated 100 times for each model, using different random seeds. For 1000 simulated responses for each stimulus-pair, the observed variability in obtained log-likelihoods amounted to about 1.6%. For each model, the best fit obtained was chosen as the final estimate. The model with the smallest Bayesian Information Criterion score (BIC; [50]) was considered the best explanation of the observations. The BIC score is a measure of relative model quality based on the likelihood, and includes a penalty for the number of free parameters in the model. Simulations were done on a desktop computer with 3.60 GHz quad-core Intel Xeon processor and 16GB of RAM, and took about one week to complete.

We could not assess the goodness of fit of the multisensory models by means of the Generalized Coefficient of Determination R², as was done to assess the fit of the unisensory models. Calculation of this statistic requires the specification of a null-model similar to the zero bias model used as comparison in the unisensory case. Since there is no single true heading angle for a discrepant combination of heading stimuli, such a null-model could not be defined for the multisensory case.

Unisensory Conditions

An overview of the unisensory data is presented in Figs 5 and 6, for the visual and inertial conditions respectively. The panels of each figure show the data and fitted unisensory models for each participant. The obtained R² statistics ranged between 0.042 and 0.780, indicating small to large improvement over a model assuming zero bias and a constant kappa (Supporting Information S1 Table).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Individual data points and the fitted models for the visual-only condition.

The panels show the data for each of the participants separately. The abscissa represents the stimulus heading angle; the ordinate represents the difference between the reported heading and the stimulus heading. The blue line is the model mean response μ minus stimulus heading for the range of stimuli; the shaded area represents the 95% CI. Each dot is a single data point.

https://doi.org/10.1371/journal.pone.0127104.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Individual data points and the fitted models for the inertial-only condition.

The panels show the data for each of the participants separately. The abscissa represents the stimulus heading angle; the ordinate represents the difference between the reported heading and the stimulus heading. The orange line is the model mean response μ minus stimulus heading for the range of stimuli; the shaded area represents the 95% CI. Each dot is a single data point.

https://doi.org/10.1371/journal.pone.0127104.g006

Data for the visual-only condition show bias towards the fore-aft axis for seven out of nine participants (as indicated by a β₁ < 1, Supporting Information S2 Table). The maximum bias according to the fitted models ranges between 2.39° (0.04rad, participant 6) and 22.28° (0.39rad, participant 2), with a median value across participants of 6.59°. The remaining two participants show a slight bias away from the fore-aft axis. In the inertial-only condition, we found bias away from the fore-aft axis for six out of nine participants (as indicated by a β₁ > 1, Supporting Information S2 Table). According to the fitted models, the maximum bias ranges between 1.12° (0.02rad, participant 1) and 9.80° (0.17rad, participant 9), with a median value across participants of 2.97°. The other three participants show a slight bias towards the fore-aft axis. The plateaus observed in the inertial-only condition for participants six and nine are thought to be a spurious result caused by a tendency to align the rod with the cardinal axes.

Multisensory Condition

We assessed which of the alternative models provided the best explanation of the responses in the multisensory conditions by comparing the models’ BIC-scores. These scores are a measure of relative model quality (see: Model Fitting & Selection). BIC-scores for the alternative models of multisensory heading perception are presented in Table 1. The best fitting model was FF for five participants; for the remaining participants the data were best explained by the MA model. The maximum-likelihood estimates of the parameters P(C) and ξ for the participants for whom the causal inference model provided the best fit are presented in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Model BIC scores.

https://doi.org/10.1371/journal.pone.0127104.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Causal Inference model parameter estimates.

https://doi.org/10.1371/journal.pone.0127104.t002

For the five participants for whom the BIC scores were in favor of the FF model, the strength of the evidence for preference of this model over the best fitting CI model ranged between a △BIC of 1.58 and 6.54. For the four participants for whom the BIC scores favored the MA model, the △BIC ranged between 0.83 and 5.00. A △BIC ranging between 0–2 is considered weak evidence; 2–6 positive; 6–10 strong; and > 10 is considered decisive evidence [50]. The evidence to discern between the FF and MA models was weak for participants two, five, seven and nine, and positive for participant four. A visual representation of the data of the multisensory conditions is provided in Fig 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Individual data points for the multisensory condition.

The panels show the data for each of the participants separately. The abscissa represents the visual heading angle of multisensory stimuli; the ordinate represents the inertial heading angle. The color of each dot corresponds to the response heading angle. Note that the responses lie on the circle, hence dots colored dark green and bright yellow have a minor angular difference.

https://doi.org/10.1371/journal.pone.0127104.g007

Unisensory heading estimation

In general, visual heading estimates were found to be biased towards the fore-aft axis, while inertial heading estimates were biased away from the fore-aft axis. The median size of the bias across participants at its maximum was 6.59° in the visual condition, and 2.97° in the inertial condition. These values are in agreement with recent literature [41, 43]. The observed nature of visual bias, i.e., towards the fore-aft axis, is in line with a number of earlier studies [51–54]. In contrast, more recent behavioral studies have found both visual and inertial heading judgments to be biased away from the fore-aft axis [41, 43]. Moreover, the latter findings confirm predictions of population vector decoding (PVD) models. The PVD model makes predictions on perceived heading based on the distribution of preferred directions in populations of neurons. The distribution of preferred directions for otolith afferents has shown a considerable over-representation of lateral headings [2]. Similar findings were reported for preferred direction of populations of neurons in cortical area MSTd [26]. Consequently, the over-representation of the lateral aspect of motion stimuli at the sensory epithelium and in the cortex leads to heading bias away from the fore-aft axis [43].

It has been suggested that the visual bias towards the fore-aft axis that was observed in earlier studies—and the present—may have been caused by presentation of only a limited range of heading stimuli [41]. Although the range of heading stimuli presented has indeed been shown to affect the magnitude of the observed bias [41], it does not reverse the effect; and because a full range of heading stimuli has been presented in the present study while an opposite bias was observed, this explanation is unlikely.

An alternative explanation for the differences in the visual bias patterns observed in the present study compared to others may be found in the nature of the visual stimuli. The studies in which MSTd tuning curves were characterized as well as the recent studies reporting visual bias away from the fore-aft axis all presented visual stimuli with a smaller field of view, a fixation cross, and employed visual stimuli without a ground plane.

Interestingly, fusion of a visual heading estimate with an inertial heading estimate that show opposite bias patterns could lead to a veridical final estimate.

Multisensory heading estimation

For five out of nine participants in the present study, the best fitting model was the Forced Fusion (FF) model. For three of the four participants for whom the Model Averaging Causal Inference (CI) model provided the best fit in an absolute sense, the evidence was not strong enough to clearly distinguish this model from the FF model. Hence, the overall findings are in favor of the FF model. The finding of forced fusion is surprising considering the large discrepancies that were introduced, but it is in line with recent work showing integration in heading estimation for visual and inertial cues with conflicting motion profiles [55], and work showing a similar effect for discrepant visual-inertial information on yaw-rotations [30]. Integration has even been reported for combinations of inertial yaw-rotations and visual roll/pitch motions [31]. For one of our participants, the evidence was in favor of the Model Averaging Causal Inference (CI) model. Evidence in favor of a CI model implies that the integration process was affected by discrepancies: the final heading estimate was constructed of an integrated heading estimate on the one hand and an unisensory estimate on the other. It is interesting to note that different people rely on different channels in case a discrepancy was detected by the CNS (according to the model), despite instructions to rely on the inertial motion (Table 2).

In general, our findings imply either that judgments on causality of visual-inertial motion stimuli do not depend on discrepancies in heading angle, or that the detection of discrepancies was impeded by extraneous factors.

In the former case, a possible reason that discrepancies between visual and inertial heading are ignored by the CNS is that these are only encountered in the artificial environments of motion simulators, making an ability to merge causality judgments based on heading discrepancies with actual heading estimates superfluous from an evolutionary standpoint [30]. In this regard, it is interesting to note that in pilot studies conducted for this experiment, one participant remarked feeling that ‘something was off’, without knowing exactly what, when there was an 180° difference in visual and inertial heading angle.

In the latter case, our findings may be explained by other factors: in an earlier study on statistical optimal integration of visual and inertial heading cues, it was found that participants’ responses for multisensory stimuli agreed with the least precise internal signal, instead of with statistically optimal integration of the visual and inertial signal [32]. The results of this study were later explained by the nature of the visual stimuli that were used, which were presented binocularly, but without disparity information. The absence of disparity information may have led to an interpretation of visual stimuli as object-motion, whereas the inertial stimuli were interpreted as self-motion, as it was shown that the incidence of integration is higher for visual stimuli with than without disparity information [28]. Although this explanation fits the observations of a lack of integration, it raises the question why integration was observed in other studies, including the present, as these studies use visual stimuli that are unlikely to elicit vection (i.e. a visually induced sensation of self-motion; [56]), because of the short duration of the motions. Because the motion stimuli used in the study where integration was not observed had a much longer duration (9.3s) than is generally the case, an alternative explanation may be that causal inference is a slow top-down inhibitory process, where discrepancies of a different nature need different times to be detected. Given that people differ in their ability to detect discrepancies, such processing could also explain why causal inference was observed for at least one, and possibly four out of nine participants in the present study.