Subjects and Ethics Statement

Two male adult macaque monkeys were used in this study. Before training, the monkeys were prepared with head restraints and recording chambers implanted stereotaxically on the dorsal surface of their skull. Implantation was performed under aseptic conditions while monkeys were anesthetized with sodium pentobarbital. All experimental procedures were in accordance with the National Institutes of Health guide for the care and use of laboratory animals. They were also approved by the animal care and use committee of Institute for Research in Fundamental Sciences (04-11-64122008). Some ethical standards incorporated into our routine laboratory procedures include housing the primates in a large space with sunshine, providing them with a psychological enriched environment (TV and toys), frequent contact with other animals (visual, auditory, touch and grooming) and pharmacological amelioration of pain associated with surgeries. The monkeys received food twice a day including fresh fruits, nuts, vegetables and special biscuits. None of the animals used in this study were sacrificed.

Stimuli

The stimuli were 7° x7° in size grayscale photographs of body (including human, monkey and quadruped subcategories) and object categories (including aircraft, car and chair subcategories). There were 90 images in each category (30 images per subcategory, Figure S1). Body images had no facial features. Each stimulus was presented in four different noise levels. Each noise level was generated by assigning a uniformly distributed grayscale value to a random selection of X% of image pixels, where X was the absolute noise level and had one of the values of 10, 30, 45 or 60. These 720 noisy stimuli [(2 categories) x (90 stimuli in each category) x (4 noise levels)] and 90 full noise images (100% noise) were randomly presented to the monkeys without repetition. Figure 1A shows two exemplar images in different noise levels. The noise pattern was fixed for one stimulus but different among stimuli. The stimuli were presented on a 19 inch CRT computer monitor placed 57 cm in front of the monkey seated in a primate chair.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Stimuli and task.

A. Exemplar body (monkey body) and object (car) stimuli in different levels of noise. Numbers below the images show percent of the noise for each column of images.

B. Sequence of task events. Two macaque monkeys were trained to perform a passive viewing task. The presentation of the stimulus sequence started after the monkey maintained fixation for 400 ms on a small white fixation point at the center of the screen. Images from two different categories (body and object) were presented randomly for 70 ms with a variable (650 to 950 ms) inter-stimulus interval (ISI). The monkeys’ task was to keep their gaze fixed on the center of the screen. They were rewarded every 1.5 to 2 seconds for maintaining the fixation.

https://doi.org/10.1371/journal.pone.0076856.g001

Among different methods of degrading the visual information (such as phase-scrambling, morphing, etc.), we chose the high-frequency "salt & pepper noise” for our study. Large receptive fields of the IT cells and the low-pass filtering process make these cells more robust to such a high-frequency noise, compared to the smaller receptive fields of cells in lower visual areas such as V1. Taking advantage of this property of the IT cells and by using different noise levels, we studied the IT neural responses in different levels of ambiguity.

Task

Monkeys were trained to perform a passive viewing task (Figure 1B). Following 400 ms of fixation on a white fixation point at the center of the screen, a randomly selected sequence of images were presented to the monkey. Each image was presented for 70 ms with a variable inter-stimulus interval (650 to 950 ms). The monkey was rewarded with a drop of apple juice every 1.5–2 seconds as long as its gaze was fixated within a 2.4° x2.4° invisible fixation window at the center of the screen. The eye position was measured by an infra-red eye-tracking system. The sequence of images stopped when the monkey broke the gaze fixation and the fixation point reappeared after 1500 ms of blank interval. Stimuli in the broken trials were presented again later. Recording sessions in which monkeys successfully completed at least half of the trials are included in the analysis. Median number of trials per category was 360 (mean±sem: body category=352±13, object category=353±13).

Recording

Craniotomy was performed to record from the inferior temporal cortex of the monkeys. The recording positions were defined by the stereotactic measurements and magnetic resonance images (MRIs) acquired prior to surgery. Subdivisions of the IT cortex were defined using the location of cortical sulci as described by Tanaka and colleagues [15-17]. Recordings were made on an evenly spaced grid, with 1-mm intervals between penetrations over a wide region of the lower bank of superior temporal sulcus (STS) and TE cortical areas (12 to 18 mm and 13 to 20 mm anterior to interauricular line in monkey1 and monkey2, respectively). During each recording session, a single tungsten electrode (FHC, 0.5 to 1 MΩ) was inserted into the IT cortex. The electrode was advanced with an Evarts-type manipulator (Narishige, Japan) from the dorsal surface of the brain through a stainless steel guide tube inserted into the brain down to 10-15 mm above the recording sites. Neural activity of multi-units (MU) in the inferior temporal cortex was recorded extracellularly, while monkeys were performing the task. To separate the spiking activity from noise we set a threshold in each recording session depending on the signal quality and the amplitude of the spikes relative to the baseline noise. Each recorded MU was the superimposed activity of several neurons around the electrode tip. A total of 66 visually responsive MUs were recorded from two monkeys (41 from monkey1 and 25 from monkey2). Visual responsiveness was defined as significantly larger evoked responses compared to the baseline activity occurring in at least one of the noise levels in any of the sliding 50-ms windows from 100 to 300 ms after the stimulus onset (t-test, alpha= 0.01). The baseline activity was measured during -50 to 0 ms relative to the stimulus onset. We explored the MU activity (MUA) because of several reasons:

First, several studies have shown that SUs and MUs in different cortical areas such as V1 and V4 behave in a similar fashion [18-20]. In V1, the response onset latency and the timing of the modulations in the response caused by the context or attention have been shown to be similar in MUAs and SUAs during a figure-ground task [19]. . We also know that MUs are even more informative than SUs for movement prediction in the motor cortex [21]. This large amount of information retained in the superimposed activity of multiple neurons suggests that the response properties of adjacent neurons are consistent and neighboring cells process similar information [19,21].

Second, Previous studies have shown that cells with similar selectivity are clustered in columns in the IT cortex [22-24]. It has also been shown that the object selectivity of a given cell in an active optical imaging spot is similar to that of the averaged cellular activity within the spot [25]. Therefore, recorded MUs in the IT cortex consist of a group of homogenous SUs with similar response properties, rather than reflecting the activity of heterogeneous single cells with larger response amplitudes. Considering the similarity of the response properties of nearby neurons it could be advantageous to collect the activity of a pool of neurons to increase the signal/noise ratio.

Third, MU recordings have some technical advantages over SU recordings. They do not require spike isolation and are more stable over time [19,26]. A concern about SU recordings is that the neurons with large action potentials are more easily and reliably isolated as a SU, creating a bias towards large neurons [19]. Furthermore, reliable isolation of SUs during a recording session is not always possible.

Forth, the goal of our study was to examine the modulations of different response properties of the IT cells by stimulus ambiguity. We added different levels of visual noise to our images to explore a full range of visual ambiguity. We predicted a significant decrease in the evoked response of IT cells especially in high noise levels. So in our study in order to obtain a more reliable signal to noise ratio we focused on MUA. We believe that especially for exploring the pattern of modulation of different response properties in time (such as response onset latency, offset latency and duration, and also SI modulation in time), the higher reliability of MUA is a clear advantage in this study.

However, the advantages of analyzing MUA do not imply that they are necessarily used by the brain. The MUA could be considered as a tool to better understand the brain physiology [21], just like LFPs [27] or functional MRI [28].

Data Analysis

Based on the similarity of the results in monkey1 and monkey2, data from two monkeys were combined in all of the analyses.

Analysis of the Amplitude of the Evoked Response

The window used for the analysis of the evoked response was 100 ms to 300 ms after stimulus onset, unless otherwise mentioned. Changing this time to other windows (e.g. 70 to 300 ms after stimulus onset) did not change any of the main results.

Analysis of the Selectivity Index (SI)

The degree of category selectivity of each unit for body versus object images was measured by:

μ(B) and μ(O) were the mean evoked response of each unit to body and object images in 10% noise level, respectively. This index could vary from -100 (absolute object category selectivity) to 100 (absolute body category selectivity). Units with SI values larger than zero were considered as ‘body-selective’. With this definition there were 48 body-selective units in our data set (Figure 2D). The SI values of our units could not be directly compared to the selective responses of face cells reported in other studies which used different stimulus sets. This is mainly due to the presence of 10% noise in the most visible image and the complexity of the images used as object stimuli in our study as well as the different response properties of body selective cells compared to face selective cells.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Response amplitude in different noise levels.

A. The response of an exemplar unit (U10) to body images in different noise levels (black: 10%, red: 30%, green: 45%, blue: 60% and magenta: 100%). Here and in all other plots of temporal pattern of the events, responses of units were measured in 1-ms bins and smoothed by convolving with a 30-ms Gaussian kernel. The gray box represents the period of evoked activity used for the further analysis.

B. Mean response of the exemplar unit in A (U10) to body images with different levels of ambiguity, during 100 to 300 ms after stimulus onset. Error bars denote s.e.m. across different trials. Stars show the p-values of the t-test between pairs of noise levels (*: P<0.05; **: P<0.01; ***: P<0.001; ****: P<0.0001; *****: P<0.00001). Inset r and P show the correlation coefficient and its p-value for the Pearson correlation analysis between responses and noise levels.

C. The response of the exemplar unit (U10) to object images. Conventions as in A.

D. Distribution of the body selectivity index (see Methods) for all of the recorded units. Red data point shows the exemplar unit (U10).

E. Averaged response of all units to body images with different levels of noise. For normalization, peak response of each unit was measured before smoothing. Then smoothed responses of each unit in different noise levels were normalized to the peak response of that unit. Finally, normalized responses of different units in each noise level were averaged. Shaded area shows s.e.m. across different units. Conventions as in A.

F. Mean normalized response of all units to body images in different levels of noise, during 100 to 300 ms after stimulus onset. Error bars here and in other figures denote s.e.m. across different units. Conventions as in B.

https://doi.org/10.1371/journal.pone.0076856.g002

We also defined category selectivity as responses to body images with %10 noise being significantly larger than object images with 10% noise (t-test, alpha= 0.05, window: 100 to 300 ms after the stimulus onset). All of the reported results were similar if we did the analysis on the 21 category selective units selected by this definition.

Analysis of Onset Latency, Offset Latency and Duration of the Evoked Response

In order to measure the onset, offset and duration of the evoked response, the activity of each unit was smoothed by convolving it with a 30-ms Gaussian kernel in every noise level. Smoothing was done to prevent measuring random jitters in the response as onset or offset. For each unit and noise level, the firing rate during baseline activity was compared to different windows of evoked activity (paired t-test, one-tailed, alpha= 0.01). The firing rate of single trials during -50 to 0 ms relative to the stimulus onset was defined as the baseline activity and the firing rates of single trials in sliding 50-ms windows, during 50 to 400 ms after the stimulus onset, was defined as the evoked response. The onset of the evoked response was defined as the first 50-ms window of the evoked response with values significantly larger than the baseline. The time interval between the start of this window and the stimulus onset was considered as the evoked response onset latency. The end of the evoked response was defined as the first window after the response onset with values not significantly larger than the baseline. The time interval between the start of this window and the stimulus onset was considered as the evoked response offset latency. Duration of the response was measured as offset of the response minus its onset, in each unit and noise level.

Reliable measurement of the onset and offset was not possible in several units at noisier conditions due to the smaller amplitude of the evoked response. Onset and offset latency of the response in all of the noise levels could be measured in 25 and 21 of the units, respectively. Thereby the duration of the response was calculated in 21 units. The fewer number of measurable offsets compared to the onsets is related to four MUs not returning to the baseline activity until 400 ms after the stimulus onset. Furthermore, the observed difference could be related to the low response variability at the beginning of the evoked response [29]. Therefore, there is a higher chance of obtaining statistical significance when measuring response onset compared to offset.

Analysis of Classification

A linear ‘support vector machine’ was used to assess the neural performance. In each noise level, the evoked response of each unit to body and object images was used as an input to the classifier. In each round of classification, we randomly selected the 75% of trials of every unit in each noise level for training the classifier. The classification performance of the cell population was tested on the remaining 25% of trials. This procedure was repeated for 1000 rounds to evaluate the statistical difference in performance between conditions.

Amplitude of the Evoked Response

To explore the effect of different levels of stimulus ambiguity on the neural response of the IT cortex, we first looked at the responses of each unit to body and object images at different levels of noise. Figure 2A shows the mean response of one exemplar unit (U10) to body images at different noise levels. As more noisy images were presented a gradual decline in the amplitude of the responses of this unit was observed. To better quantify this modulation, we measured the evoked firing rate of this unit during 100 to 300 ms after stimulus onset in each single trial (Figure 2B). Consistent with the peristimulus time histogram (PSTH) in Figure 2A, the response amplitude declined as the stimulus ambiguity increased (Pearson correlation, r= -0.55, P< 0.00001). Also its response to object images at different noise levels was smaller compared to body images (Figure 2C). We defined a selectivity index (SI) to measure this difference more directly, (see Methods). For this exemplar unit the value of SI in the lowest level of noise (10%) was 18.2 which confirmed its body selectivity. The IT cortex, as the last unimodal stage in the ventral visual pathway, has units selective to complex objects like faces and bodies [1-4,30-38]. Body selectivity in the IT cortex is reported at the level of single cell [2] and cortical patches [30,31,35,37]. We measured SI in the 66 units to document the presence of body selectivity in the activity of several neighboring cells recorded as a unit in our study. Figure 2D shows the distribution of body selectivity index in the recorded units. We identified 48 body selective units (32 from monkey1 and 16 from monkey 2) and further analyzed their spiking activities.

The averaged responses of all 48 body selective units to their preferred category (body) at different levels of noise are shown in Figure 2E. There was a decline in the response amplitude as the stimulus ambiguity increased. We noted a clear distinction of the responses to body images in different degrees of noise at the population level which indicated a similarity in the sensitivity of these units to the degradation of their preferred category. To quantify the gradual decrease in the response of units to more noisy images we measured the average of normalized responses of the units during 100 to 300 ms after stimulus onset at each noise level (Figure 2F). Consistent with the data from the exemplar unit (Figure 2B) and also the PSTH in Figure 2E, the response amplitude declined linearly as stimulus ambiguity increased (Pearson correlation; both monkeys: r= -0.57, P< 0.00001; monkey1: r= -0.57, P< 0.00001; monkey2: r= -0.58, P< 0.00001).

Temporal Dynamic of the Evoked Response

So far, our results showed a gradual decline in the response amplitude of the population of the body selective units as a function of noise. The amplitude of the evoked response is a common coding mechanism in different cortical areas [39,40]. Onset of the evoked response is another potential mechanism for visual stimuli coding in the IT cortex [41]. We previously demonstrated that the onset latency of the evoked response is shorter for specific categories (faces) compared to the others [41]. We predicted that the onset latency as a neural coding tool could also be affected by the level of stimulus ambiguity. Observation of the temporal pattern of responses to images in different noise levels in Figure 2 is consistent with this idea. In Figure 2A the onset and offset of the response of the body selective unit to its preferred category was found to be different among noise levels. To better quantify this effect, we measured the onset, offset and duration of the response of this unit (U10) to body images in different levels of stimulus ambiguity (see Methods). The evoked response of this unit started later and decayed earlier as stimulus ambiguity increased (Figure 3A). As a result of these modulations, the duration of the response was shorter for more ambiguous images (Figure 3A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Response onset latency, offset latency and duration in different noise levels.

A. Onset latency, offset latency and duration of the response of the exemplar unit (U10) to body images in different levels of ambiguity. When full noise images were presented there was no increase in the response of this unit relative to the baseline activity (Figure 2A). Therefore, onset, offset and duration were not measurable in this noise level and are not shown in this figure.

B. Onset latency of the response of the units (n=25) to body images with different levels of noise.

C. P-values of the comparison of onset latency values in different pairs of noise levels (t-test, paired).

D. Offset latency of the response of the units (n=21) to body images with different levels of noise.

E. P-values of the comparison of offset latency values in different pairs of noise levels (t-test, paired). Conventions as in C.

F. Duration of the response of the units (n=21) to body images with different levels of noise.

G. P-values of the comparison of response duration values in different pairs of noise levels (t-test, paired). Conventions as in C.

https://doi.org/10.1371/journal.pone.0076856.g003

Based on these findings and also the temporal pattern of the population response in Figure 2E, we expected to see similar results in other body selective units. Figure 3B shows the onset of the response of body units to body images with different levels of ambiguity. Response onset was earlier for less noisy images (Figure 3B; Pearson correlation; both monkeys: r= 0.31, P= 0.0004; monkey1: r= 0.31, P= 0.0046; monkey2: r= 0.34, P= 0.03; see also Figure 3C). We found a larger variability of the onset latency among body units at 100% noise level indicating possible increased within unit variability of the onset latency in this condition.

We then tested the correlation between the response amplitude and response onset latency across all noise levels (r= -0.38, P<0.00001). We also measured the onset latency of the response to object images in different noise levels (Pearson correlation, r= 0.22, P= 0.02). Similar to the body images, we found a significant negative correlation between the response amplitude and the onset latency (r= -0.46, P<0.00001). One concern with respect to this finding might be that the shorter onset latency in less noisy conditions was simply the result of a larger response amplitude in these conditions [17]. We addressed this issue using two-way ANOVA in which image categories and noise levels were the two factors to compare the onset latency of the responses to body and object images at different noise levels. The results showed that the onset latency was different across noise levels (P= 0.006), while it remained unchanged between two categories (P= 0.24). We also compared the amplitude of the response to body and object images at different noise levels using two-way ANOVA. We found that the response amplitude was significantly different both across noise levels (P< 0.00001) and between categories (P< 0.00001, larger response amplitude for bodies). Collectively these results suggest that while the amplitude of the response to bodies was larger than the response to objects at different levels of noise, the onset of the responses were not different at any given level of noise. Therefore, if the observed difference in the response onset latency across noises (Figure 3B) was simply the result of the difference in the response amplitude, it should have been also different between body and object images at each noise level. This analysis confirms that the modulations in the onset latency could not be explained by the differences in the response amplitude.

We also measured the offset latency and duration of the response which both decreased as more ambiguous stimuli were presented (offset: Figure 3D; Pearson correlation; both monkeys: r= -0.43, P= 0.00004; monkey1: r= -0.45, P= 0.0001; monkey2: r= -0.47, P= 0.0095; see also Figure 3E) (duration: Figure 3F; Pearson correlation; both monkeys: r= -0.55, P< 0.00001; monkey1: r= -0.57, P< 0.00001; monkey2: r= -0.57, P= 0.0011; see also Figure 3G). These findings suggest that the increased duration of the evoked response to more visible images is the result of an earlier rise and a later fall in the response.

Selectivity of the Evoked Response

Exploring the noise-related modulation of the responses to the preferred category helps to understand the cortical sensory processing. However, for a better understanding of the effect of noise on object recognition it is essential to compare noise-related modulations of the responses to the preferred versus non-preferred category (category selectivity). The responses of the body selective units to the non-preferred category (object images) are shown in Figure 4A. These responses were smaller than the responses to body images (Figure 2E) and declined as the noise level increased (Figure 4B; Pearson correlation; both monkeys: r= -0.42, P< 0.00001; monkey1: r= -0.43, P< 0.00001; monkey2: r= -0.42, P= 0.0001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Response to body vs. object images in different noise levels.

A. Averaged response of all units to object images with different levels of noise. Normalization was done as described in Figure 2E. The gray box represents the period of evoked activity used for the further analysis.

B. Mean response of all units to object images in different levels of noise, during 100 to 300 ms after stimulus onset. Conventions as in Figure 2B.

C. Response of all body selective units (n=48) to body and object images in different noise levels during 100 to 300 ms after stimulus onset. Each data point shows the mean response of one unit. The red data point shows the exemplar unit (U10). Full noise (100%) is not shown in this figure because there is no category information in full noise images. The inset p-values show the results of paired t-ttest between responses to body and object images.

https://doi.org/10.1371/journal.pone.0076856.g004

To explore how the difference of the response to body and object images changes as a function of noise, we plotted the mean response to these images in different noise levels during 100 to 300 ms after stimulus onset (Figure 4C). As the noise level increased the difference of the response amplitude of the units to body and object images diminished. At 10% noise level the response amplitude to body image was significantly greater in comparison with the response to the object image. There was no statistically significant difference in response amplitude to body compared to object images at 60% noise level (t-tests, paired, one-tailed, 10%: P< 0.00001, 30%: P< 0.00001, 45%: P= 0.002, 60%: P= 0.46). Measuring SI at different noise levels in the exemplar unit (U10) showed a gradual decrease in SI as stimulus ambiguity increased (90%: 18.15, 70%: 12.87, 55%: 3.95, 40%: 0.08). Measuring SI in all units confirmed the same observation: SI of the body selective units decreased as stimulus ambiguity increased (Figure 5A, Pearson correlation; both monkeys: r= -0.45, P< 0.00001; monkey1: r= -0.46, P< 0.00001; monkey2: r= -0.42, P= 0.0006). The difference of the response amplitude to preferred and non-preferred categories decreased to the point of no difference at 60% noise level. Therefore, at this noise level SI was not significantly different from zero (t-tests, paired, P= 0.5) meaning that the threshold of noise tolerance for category selectivity of IT units was 60%. Note that the evoked response to both body and object images at 60% noise was significantly larger than both the baseline activity (baseline window: -50 to 0 ms relative to the stimulus onset; t-tests, paired, bodies: P<0.00001, objects: P<0.00001) and the response to full noise images (t-tests, paired, bodies: P<0.00001, objects: P<0.00001). This suggests a neural signal that indicates the presence of a stimulus but not its category; as there was not any significant difference between responses to body and object categories.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Category discriminability in different noise levels.

A. SI of the all units in different noise levels, measured during 100 to 300 ms after stimulus onset. Stars show p-values of t-tests between pairs of noise levels (**: P<0.005).

B. The performance of a classification trained to categorize body versus object stimuli. Stars show p-values of t-tests between pairs of noise levels (*****: P<0.00001).

C. Temporal dynamic of SI of the exemplar unit (U10). SI was measured in different noise levels in sliding 50-ms windows. Data points are plotted at the middle of each bin. The gray box represents the window used for the analysis in A.

D. Temporal dynamic of SI of all body selective units in different noise levels. The gray box represents the window used for the analysis in B. Conventions as in C.

https://doi.org/10.1371/journal.pone.0076856.g005

To see how responses of the population of MUs in IT could represent stimulus category in noisy conditions, we trained a classifier to categorize body versus object stimuli (Figure 5B). Classification performance decreased as the noise increased (Pearson correlation, r= -0.9, P< 0.00001). The observed chance level performance of the classifier for the stimuli with 60% noise level is a further indication that, in passive viewing condition and at high noise levels, IT units convey information about stimulus presence without signaling the stimulus category.

To explore the SI temporal dynamics we measured SI in sliding 50-ms windows at various noise levels. Figure 5C shows the results for the exemplar unit. SI in less ambiguous stimuli increased earlier and decayed later. Temporal dynamic of SI in all units, at different noise levels showed a similar pattern (Figure 5D). At 60% signal SI fluctuated around zero during the evoked response which is consistent with the lack of selectivity in Figure 5A for this noise level.

We measured body cells’ cumulative SI which is the differential response to body versus object images developing in time. It was calculated separately for low (10% and 30%, blue line) and high (45% and 60%, red line) noise images during 100 to 300 ms after the stimulus onset (Figure 6A). The enhancement of SI started later and showed a shallower slope in the noisier condition.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Schematic model.

A. Body cells’ cumulative SI in more and less noisy conditions. Cumulative SI was measured separately for less (10% and 30%, blue line) and more (45% and 60%, red line) noisy images in non-overlapping 50-ms windows during 100 to 300 ms after the stimulus onset. Dashed lines (a and b) represent hypothetical lines representing possible decision boundaries.

B. Drift diffusion model and evidence accumulation for visible and ambiguous stimuli. Decision variable (DV) is the cumulative sum of the evidence. The bounds represent the decision boundaries for different choices. Slower drift rate in ambiguous condition is the result of lower response amplitude, shorter response duration and smaller response selectivity in this condition.

https://doi.org/10.1371/journal.pone.0076856.g006