The “Dots” method

The “Dots” method was designed to enable the experimenter to precisely control the amount and type of object-related information provided to the subject. The method exploits several Gestalt principles such as grouping by proximity and good continuation. Visual stimuli are generated by the controlled deformation of a lattice of dots that is driven by a single feature of the original image: local contour density. The method creates a map of points of interest (POI) to compute local information content (local contour density) around each pixel in the source image. This computation is similar to assigning a “saliency” value to each pixel in the source image [20]. To generate the stimulus, a lattice of dots with square structure [21] is then progressively deformed based on the POI map such that dots converge towards corresponding salient regions in the source image. Local contour density information from the source image is therefore progressively represented by the lattice. By adjusting the amount of deformation, the stimulus from the source image can be more or less visible in the deformed lattice.

The POI map computes the local information around pixels in the original image (first converted to grayscale) by applying a local Gabor Wavelet (Gabor Jet) decomposition (Figure 1A) around each pixel [22]. We used Gabor Jets composed of a set of zero-mean Gabor filters spanning a range of spatial frequencies (0.1, 0.25, and 0.5 cycles/pixel implemented as convolution kernels with Gabor sigma of 1.16, 2.16, and 4.83 pixels, respectively, and spanning 7, 13, and 29 pixels, respectively) and different orientations (0°, 30°, 60°, 90°, 120°, and 150°). Each Gabor filter in the jet is convolved with the local image around the pixel and yields a local response. The sum of responses across the jet reveals the importance of the respective pixel in the source image. The POI map can be interpreted as a map describing the contour information content of the original image estimated locally around each pixel. Points located close to contours with higher local contrast or at the intersection of multiple contours will have a stronger response in the summed responses of local filters than points located close to surfaces with smooth luminance changes. Thus, the POI map uncovers the most important points required to identify an object by its representative local contours.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Stimulus creation.

(A) The source image is filtered with Gabor wavelets to yield a map of points of interest (POI) estimating the local information in the source image (dark points are most informative). (B) Dots of an elastic lattice are attracted towards points in the POI plane by the projection F_g of a gravitational force G; an elastic force, F_k, limits their displacement. Dot movement is confined to the “Lattice” plane. (C) Progressive deformation of a lattice of dots as g is increased. (D) Set of objects used to test the method on human subjects.

https://doi.org/10.1371/journal.pone.0022831.g001

The method employs two parallel plane surfaces (xy coordinates) located at a distance h (Figure 1B). One plane consists of the POI map, normalized to contain values between 0 and 255. The other plane is a regular lattice of dots. Each pixel in the POI map exerts a gravitational attractive force upon lattice elements proportional to its information content; lattice elements resist movement with an elastic force. Consider one pixel from the POI map and a lattice element situated apart at distance, r, (Eq. 1). A gravitational force, G, (Eq. 2) attracts the lattice element towards the position of the pixel in 3D space (Figure 1B). Each lattice element has a generic mass, m_L, and each pixel in the POI map has a mass, m_P, directly proportional to its local information content (sum of filter responses across the Gabor Jet). The movement of lattice elements is restricted to the lattice plane and thus only the projection of G on the lattice plane, F_g, acts on lattice elements (Eq. 3). Assume that the lattice element has already been pulled by gravitational forces and lies at distance, d, (Eq. 6) from its original position. At this position, an elastic force, F_k, proportional to the displacement (Eq. 7) acts like a spring that pulls the lattice element back towards its initial position.(1)(2)(3)(4)(5)where, h is the distance between the POI plane and the lattice plane, r is the distance between the POI and the lattice element; r_x and r_y are projections of the vector distance r onto the x and y axes of the lattice plane; G is the gravitational force that pulls one lattice element towards a POI; g is the gravitational constant; m_P is the mass of the pixel in the POI map (proportional to the energy of the corresponding Gabor jet); m_L is a constant value for the mass of lattice elements (we fixed it to a value of 1); F_g is the projection of the gravitational force G on the lattice plane, F_gx and F_gx are projections of F_g on the x and y axes.(6)(7)(8)(9)where, d is the distance of the lattice element from its original position, d_x and d_y are its projections on the x and y axes; F_k is the elastic force, F_kx and F_ky are its components on x and y axes; K is an elastic constant.

The deformation of the lattice is solved by an iterative algorithm. At each step, for each lattice element the resultant force is computed as the superposition of gravitational forces from all POI pixels and the corresponding elastic pull. The movement on both xy directions is then directly proportional to the resultant forces (Eqs. 10, 11). The iterative process stops either when lattice elements have stabilized, or when a certain number of iterations have been performed.(10)(11)where, i indexes all POI pixels; Δx and Δy are the movements of one lattice element on the x and y directions respectively; S is a scaling constant (here a value of 5) that converts the forces into displacement.

The entire process is controlled by three parameters. First, the elastic constant, K, (Eq. 7) restricts the movement of lattice elements. Second, the gravitational constant, g, (Eq. 2) controls the strength of the gravitational forces and determines how much the grid is deformed to represent the information in the original image (Figure 1C). Third, the distance, h, between the two planes controls how diffuse are the gravitational forces (Eqs 1, 2) and ensures a minimum distance between POIs and lattice elements thereby limiting gravitational forces and ensuring convergence of the iterations (Eqs 1, 2). In practice, K and h are fixed to some satisfactory values and only g is then varied to manipulate visibility. Thus, the “Dots” method enables relevant visual information about objects in the source image to be transferred into the stimulus in a controlled fashion.

Behavioral data

To study object detection and explicit object recognition during free visual exploration, we used stimuli generated with the “Dots” method in psychophysical experiments. Images of 50 objects without background (Figure 1D) were used to create 7 stimuli for each object, with visibility level ranging from no visibility (g = 0; no information from the source image was transferred into the dot lattice) to easily visible (g = 0.3; where subjects could easily identify the object from the source image) (see Materials and Methods, Stimuli). Depending on whether reaction times were stressed as being important or not, two different experiments were carried out (see Materials and Methods, Experiments). In the first experiment, instructions given to subjects emphasized accuracy but mentioned that speed was important as well, while in the second experiment subjects were given no instructions regarding response speed. Twenty six subjects participated, 14 in the first experiment and 12 in the second experiment (see Materials and Methods, Subjects). Subjects were allowed to visually explore each stimulus for as long as they wanted. They were instructed to decide whether the dot pattern represented something meaningful by indicating with a separate keypress whether they perceived “Nothing”, they saw something but were “Uncertain” what it was, or they have “Seen” a clear object (see Materials and Methods, Procedure). After the button press, participants were required to verbalize their response and also the name of the object, when this was the case.

The psychophysical experiments relied on the ability of the “Dots” method to precisely control recognition at threshold, such that the middle ground between detection (signaling the presence of an object without the ability to identify it) and explicit recognition could be thoroughly investigated. Importantly, detection and explicit recognition were not disentangled by limiting exposure duration [19] but by limiting the amount of available object-related information while allowing the subjects to freely explore stimuli, for unlimited duration. This strategy enables the brain to explore visual solutions and to incrementally integrate visual information to reach a decision. In addition, previous research has shown that recognition can be affected not only by the visibility of a stimulus, but also by previous exposure to it (i.e., perceptual priming [5]; see also [23], [24] for reviews), and that such effects have measurable fMRI/EEG correlates [17], [25], [26]. Therefore, in addition to manipulating stimulus visibility, we also attempted to determine how recognition-related aspects (performance, visual exploration of the stimulus) are affected by the stimulus presentation strategy (order of presentation of stimuli with different visibility levels) and thus the ensuing effect of top-down processes. Using a between-subjects design, two versions of the task (conditions) were run for each of the two experiments (see Materials and Methods, Experimental design). Stimuli obtained from the 50 source images and with the same visibility level (same g value) were grouped in blocks, each block corresponding to a different visibility level. In one condition, blocks were presented in an ascending order of visibility (from g = 0 to g = 0.3), corresponding to a naive subject that had no prior information about the identity of the objects before reaching recognition threshold. In the second condition, blocks were presented in reverse order of g value (from g = 0.3 to g = 0), such that when approaching recognition threshold from above, subjects were already primed by previous exposure to the fully visible objects.

Dependent variables of interest were (1) percentage of subjects' button-press responses for each response type (“Nothing”, “Uncertain”, “Seen”), (2) verbal response accuracy (percentage incorrect, “Uncertain”, correct), and (3) reaction time (RT). A verbal response was considered correct if the subject correctly identified the object at a basic or subordinate level, or if the subject responded “Nothing” to a stimulus with g = 0. Since dot stimuli do not allow for fine details to be discriminated, in one case we also considered a response correct if the subject-given label was of a structurally similar object (e.g., “hand mirror” instead of “tennis racket”). A verbal response was considered incorrect if the subject assigned the wrong label to a stimulus, or if he/she responded “Nothing” at g>0. “Uncertain” responses were not considered as either correct or incorrect, but were treated as a separate category. Because all psychometric curves, except for reaction times, were very similar in the two experiments we collapsed the former across experiments.

We first computed the percentage of responses as a function of response type for both the ascending and the descending experimental conditions (Figure 2A). In order to test the effects of experimental condition and stimulus visibility, for each response type we conducted a separate 2 (experimental condition: ascending vs. descending)×7 (g value) mixed ANOVA, with response percentage as the dependent variable. Detailed results for these analyses are presented in Table 1. The percentage of “Nothing” responses decreased gradually as visibility (g) increased (see Table 1 and Figure 2A, left) and this pattern was similar for the ascending vs. descending condition, as indicated by the absence of effects for experimental condition or the interaction of the two factors. Post-hoc contrasts revealed that there were significant drops in the percentage of “Nothing” responses from g = 0.05 to g = 0.20 (see Table 1 and Figure 2A, left).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Psychometric curves, subjective and objective thresholds.

Psychometric curves grouped by response type (A) and by verbal report accuracy (B) as a function of visibility (g value) and experimental condition (ascending and descending). (C) Thresholds for sigmoidal response curves corresponding to “Seen” (subjective) responses (left) and correct (objective) verbal responses (right). Error bars represent s.e.m.

https://doi.org/10.1371/journal.pone.0022831.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Analysis of variance results.

https://doi.org/10.1371/journal.pone.0022831.t001

The value of g had a significant effect on the percentage of “Uncertain” responses, but this effect manifested differently in the two experimental conditions (as indicated by a significant g× experimental condition interaction; see Table 1). More precisely, in the ascending condition there was a clear peak at g = 0.10, reaching 46.46% (post-hoc contrasts indicated significant increases from g = 0.05 to g = 0.10, and decreases from g = 0.10 to g = 0.30; see Table 1 for details). In the descending condition, subjects reported uncertainty in less than 22% of cases, and this was, on average, relatively constant across different visibility levels (the trend was flat for gs between 0 and 0.25 and had a significant increase only from g = 0.25 to g = 0.30) (Figure 2A, middle).

“Seen” responses increased with increasing g value from g = 0 to g = 0.25, according to a sigmoidal curve (Figure 2A, right). The effect of g interacted with experimental condition: there was a steady increase in response percentage from g = 0.05 to g = 0.30 in the ascending condition, and from g = 0 to g = 0.20 in the descending condition, where we also found a drop in response percentage from g = 0.25 to g = 0.30. There was no effect of experimental condition alone on “Seen” response percentage.

We next carried out the same type of analysis−2 (experimental condition)×7 (g value) mixed ANOVA (see Table 1 for results details)−grouping the data according to verbal response accuracy (percent of correct and incorrect responses) (Figure 2B). Results were, with very few exceptions, similar to those found when grouping data according to response type. In the first block (g = 0) subjects reported correctly that there was “Nothing”, but their failure to detect objects for larger visibility levels (e.g., g = 0.05) led to a peak of incorrect responses (Figure 2B left) that was then progressively reduced with increasing g. The percentage of incorrect responses (Figure 2B, left) was affected by g (after the initial peak at g = 0.05, there was a steady drop until g = 0.20), but not by experimental condition or the interaction of the two variables.

Correct responses were a majority in the first block (response “Nothing” to lattices containing no object-related information when g = 0), but then dropped for g = 0.05, and subsequently increased across a smooth sigmoidal threshold (Figure 2B, right). As in the case of “Seen” responses, there were significant effects of g (a steady increase from g = 0.05 to g = 0.25) and experimental condition×g value (ascending: a performance improvement from g = 0.05 to g = 0.30; descending: improvement from g = 0.05 to g = 0.20, with a performance drop between g = 0.25 and g = 0.30). However, overall correct recognition performance did not differ between the two groups of subjects (i.e., no effect of experimental condition alone).

Grouping of data according to response type (“Nothing”, “Uncertain”, “Seen”) reflects the dependence of subjective recognition on g. When responses are grouped according to correctness of verbal response (incorrect, “Uncertain”, correct), this represents a more objective dependence of recognition performance as a function of g. To quantify recognition thresholds, we fitted subjective (Figure 2A, right) and objective (Figure 2B, right) recognition curves (portion with g>0) with a sigmoid function, for each subject (see Materials and Methods, Sigmoid fitting for threshold identification). In the case of subjective curves (Figure 2A, right), recognition thresholds were significantly lower for the descending (φ_descending = 0.09) than for the ascending (φ_ascending = 0.12) condition [t(24) = 4.41, p<0.001, d = 1.80] (Figure 2C, left). The same was true for the objective recognition curves (Figure 2B, right), with thresholds significantly lower for the descending (φ_descending = 0.08) than for the ascending (φ_ascending = 0.13) condition [t(24) = 4.95, p<0.001, d = 2.02] (Figure 2C, right). Within each experimental condition, the subjective and objective recognition thresholds were not significantly different (paired-samples t-test: t(12) = 2.11, p = 0.06 for the ascending and t(12) = 0.69, p = 0.50 for the descending condition) and were highly correlated (r = 0.94, p<0.001 for the ascending and r = 0.87, p<0.001 for the descending condition).

The above recognition thresholds were computed for each subject by taking performance curves across the entire set of objects. We next turned to investigate how individual objects were detected and recognized. To this end, for each object we computed the separation border between “Nothing”/“Uncertain” (detection) and between “Uncertain”/“Seen” (recognition), by finding the lowest g value where “Uncertain” and “Seen” responses occur, respectively. Results revealed that separation borders were specific to individual objects (Figure 3A). The two borders were also correlated and this correlation was higher for the ascending (r = 0.79, p<0.001) than for the descending condition (r = 0.58, p<0.001). This indicates that detection and recognition borders were more coherently modulated by object identity for the ascending than for the descending condition. In other words, if the detection border was lower/higher, then the recognition border was also lower/higher for the same object, but this relation was more prominent for the ascending condition. On average across objects, borders in the descending condition were both lower than their corresponding borders in the ascending condition (Figure 3B; t(49) = 3.74, p<0.001 and t(49) = 12.19, p<0.001 for detection and recognition, respectively; paired-samples t-test). This was the case for individual objects as well. The detection border was, for most objects, lower in the descending than ascending condition, and this effect was even clearer for the recognition border, where it held for all objects (Figure 3C). In addition, the magnitude of the border-lowering effect by experimental condition was object-specific because the effect was stronger for some objects (e.g., “headphones”) than for others (e.g., “piano”) (see Figure 3C).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Detection and recognition of individual objects.

(A) Borders between “Nothing”/“Uncertain” (detection) and “Uncertain”/“Seen” (recognition) computed on individual objects, for the ascending (left) and descending (right) conditions. Error bars are s.e.m. (B) Average and SD of detection and recognition borders from (A) across all objects. (C) Individual object detection and recognition borders in the descending versus ascending condition.

https://doi.org/10.1371/journal.pone.0022831.g003

Lastly, we investigated RT differences between response types and as a function of experiment (Figure 4). For each subject, we extracted median RTs for each g value and then averaged them for each response type to obtain a global RT. RTs differed significantly between subjects included in Experiment 1 versus Experiment 2. We therefore included experiment as a separate independent variable in this analysis. We conducted a 2 (experiment: 1 vs. 2)×3 (response type: “Nothing”, “Uncertain”, “Seen”) mixed ANOVA with RT as the dependent variable. Overall, RTs were faster for subjects in Experiment 1 compared to those in Experiment 2: F(1, 24) = 13.54, p<0.01, η_p² = 0.36. We also found a significant effect of response type: F(2, 48) = 47.09, p<0.001, η_p² = 0.66; post-hoc pairwise comparisons with Bonferroni correction indicated that subjects were fastest when responding “Seen” and slowest when responding “Uncertain” (all comparisons were significant at p<0.01). Finally, we found a significant experiment×response type interaction [F(2, 48) = 6.63, p<0.01, η_p² = 0.22], explained by the fact that the effect of response type was smaller for Experiment 1 (η_p² = 0.53) compared to Experiment 2 (η_p² = 0.73) (see Figure 4 for details). These results show that instructions to subjects related to RT did have effects on visual exploration. However, the relation between trial durations across different response types remained the same, i.e. lack of instruction regarding response speed merely scaled up RTs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Exploration time estimated by measuring reaction time as a function of response type in the two experiments.

Error bars represent s.e.m.

https://doi.org/10.1371/journal.pone.0022831.g004

Eye tracking data

To shed new light on the integration of visual information during free visual exploration and on how visual exploration is affected by previous exposure, we complemented the psychophysical measurements with eye-tracking recordings. We investigated how eye movement patterns differed depending on the availability of sensory evidence and previous exposure (top-down processes). To this end, we monitored eye position for subjects in Experiment 2 and computed the patterns of saccades/fixations (see Materials and Methods, Identification of fixations). The analyses conducted on this data were similar to the ones conducted for response percentages – i.e., 2 (experimental condition: ascending vs. descending)×7 (g value) mixed ANOVAs, with fixation-related measures (see below) as the dependent variable in each case (see Table 1 for details on statistical results).

When the stimulus was visible, fixations were localized in stimulus areas where dot displacement (Figure 5A, left) represented the underlying local contour density from the POI map (Figure 5A, right). The spatial distribution of fixations was markedly different in the ascending versus the descending condition. Fixations for all objects are shown for two representative subjects (one performing the ascending the other the descending task) across different g values in Figure 5B. In the ascending condition subject a010 explored almost the entire stimulus surface at g = 0. As visibility was increased, fixations became more concentrated towards the center, where objects were located (Figure 5B, left). This was not the case for a subject (d014) that viewed stimuli in descending order of visibility. In this case, the spatial extent of fixations at g = 0.3 was compressed for intermediate visibility levels and then expanded again as g approached 0 (Figure 5B, right). The reported effects were also consistent when we investigated population data and computed the distances of fixations from the center of the stimulus (fixation spread) (Figure 5C). Overall, fixation spread was significantly larger in the ascending versus descending condition. We also found a main effect of g (see Table 1 for details), but the g-related curves differed, as indicated by a significant experimental condition×g interaction. Fixation spread was largest in the ascending condition for g = 0 and then decreased until g = 0.10. For the descending condition, fixation spread had a “U” shape [quadratic trend: F(1, 5) = 7.66, p<0.05, η_p² = 0.60].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Eye-tracking analyses.

(A) Pattern of fixations/saccades revealed in relation to the “cannon” stimulus (left) and its underlying POI map (right). (B) Pooled fixations on all stimuli as a function of visibility for a subject performing the ascending (left) and one performing the descending protocol (right). (C) Average fixation spread (distance from image center). (D) Fixation count, normalized per subject. (E) Fixation duration. (F) Average local contour density (computed from POI map) in areas of 0.5° in diameter around explored locations corresponding to fixations on the dot stimulus. (G) Integrated dot displacement, computed as a sum of displacements of dots (in areas of 0.5° in diameter around each fixation) relative to the undeformed lattice. The sum runs over all fixation locations in the trial. Error bars represent s.e.m.

https://doi.org/10.1371/journal.pone.0022831.g005

To further describe the image exploration process we next investigated fixation count per trial and fixation duration. For each subject, fixation count was first normalized to the individual average because its absolute value tended to be highly subject-specific (fixation count per trial is tightly related to reaction time, and thus to individual subjects' exploration strategy). Normalized fixation count was not affected by experimental condition, but it changed as a function of g (see Table 1) and the interaction of g with experimental condition: it decreased monotonically and linearly in the ascending condition [linear trend: F(1, 5) = 19.77, p<0.01, η_p² = 0.80] and was “U”-shaped in the descending condition [quadratic trend: F(1, 5) = 129.56, p<0.001, η_p² = 0.96] (Figure 5D). In addition, at maximum visibility (g = 0.3) subjects made significantly more fixations in the descending than ascending condition [t(10) = 3.75, p<0.01; independent-samples t-test], indicating that stimulus novelty was a factor influencing fixation count. Fixation duration was modulated differently by visibility as it varied only as a function of g, while experimental condition or its interaction with g had no statistically significant effect. In both conditions, the duration of fixations was longer for low and intermediate visibility levels (significant decrease from g = 0.15 to g = 0.20; see Figure 5E) indicating that subjects tended to maximize the integration of available visual features under difficult viewing conditions. In addition, stimulus novelty also played a role because at maximum visibility (g = 0.3, which is the first block in the descending protocol) the descending (versus ascending) condition was associated with significantly longer fixations: t(10) = 2.33, p<0.05.

Local contour from the original image is a hidden variable because subjects have only indirect and limited access to it by means of dot displacement. It is however possible to estimate how subjects accessed this information. To this end, we considered locations of fixations on the stimulus (dot image) and then computed the corresponding underlying average local contour density (from the POI map) in a region spanning 0.5° visual angle around each fixation. We found that the average local contour density in explored locations increased with increasing g value, indicating that information about local contours was progressively made available to subjects and they used this information to reach a decision (Figure 5F): we found no difference between the two experimental conditions, but there was an effect of g (increase from g = 0 to g = 0.15) and of the experimental condition×g interaction (increase from g = 0.05 to g = 0.15 in the ascending condition, and from g = 0 to g = 0.10 in the descending condition). These analyses reveal that explored contour density reached a plateau around g = 0.10–0.15, showing that already around the observed recognition thresholds subjects were able to seek out most informative locations. In addition, at very low visibility levels subjects in the descending condition were able to more efficiently explore informative locations than in the ascending condition [at g = 0.05: t(10) = 3.25, p<0.01; independent-samples t-test], suggesting that top-down effects can guide visual exploration (Figure 5F).

Finally, we investigated how subjects used the information available in the stimulus itself, that is, the deformation of the lattice by displacing the location of dots. We considered each fixation and computed the displacement of dots from the grid (contribution of jitter not included), within an area of 0.5° around the fixation. We then computed the total integrated dot displacement per trial by summing displacements over trial fixations. Intuitively, this measure reflects the amount of dot displacement integrated by the subject to reach a decision. Results indicate that integrated dot displacement (Figure 5G) varied as a function of g (increases from g = 0 to g = 0.10, from g = 0.15 to g = 0.20 and from g = 0.25 to g = 0.30) and experimental condition×g (an increase from g = 0 to g = 0.10 in the ascending condition, and an increase from g = 0 to g = 0.10 and g = 0.15 to g = 0.30 in the descending condition), but not experimental condition alone. Thus, in the ascending condition integrated dot displacement saturated already around the perceptual threshold. Even though displacement increased with increasing g, subjects explored roughly the same amount of dot displacement to reach a decision by making progressively fewer fixations (see Figure 5D). In the descending condition, subjects first explored a large amount of displacement at maximum visibility (g = 0.3), because of stimulus novelty, but at intermediate visibility levels they integrated less dot displacement than did subjects in the ascending condition [independent samples t-test: t(10) = 4.25, p<0.01 for g = 0.15; t(10) = 2.50, p<0.05 for g = 0.20]. The latter effect suggests that strong priming can have a dramatic influence on the integration of visual information required to support a decision.

Ethics Statement

Psychophysical and eye-tracking measurements were performed on human subjects who gave their prior written informed consent to participate in the experiments. The experimental protocols have been approved by the local ethics committee of the University of Medicine and Pharmacy “Iuliu Haţieganu” of Cluj-Napoca (approval No. 150/10.12.2009).

Stimuli

The original images used for generating the stimuli were selected from the Caltech 101 [68], Caltech 256 [69] and ETH-80 [70] databases, or from various internet sources. We used 50 images (plus 2 additional images for the practice trials) of different plants, fruits, animals and human-made objects (Figure 1D). Images were rescaled such that the image frame occupied the same size (600×400 pixels) and, additionally, all background information was removed and the object of interest was centered in the frame. The lattice deformation procedure was applied to each image, using a lattice of black dots on a white background (600×400 pixels) with a dot diameter of 5 pixels and a distance between dots of 10 pixels. The elastic constant K was set to 10, and the distance h between POI and lattice planes was set to 5. Seven stimuli were generated from each image, corresponding to seven different gravitational constant (g) levels, from 0 to 0.3, in steps of 0.05. To prevent subjects from instantaneously detecting that no object was present when g = 0 (perfect, undeformed lattice), we added a uniform random jitter of zero-mean and 3 pixels maximum amplitude to the position of dots after deformation. This small jitter was applied for all levels of visibility. Examples of dot stimuli generated from the same source image are presented in Figure 1C. Thus, the final stimulus set consisted of 350 stimuli and an additional 14 for the practice trials (the full set of stimuli is freely available under http://www.raulmuresan.ro/sources/lattdef).

Object recognition difficulty from dot stimuli depended to a large extent on the physical spacing between dots and on the distance of the subject from the monitor. Therefore, we had to calibrate monitor distance such that subjects had a very hard time identifying the objects for g = 0.05 and could effortlessly identify them for g = 0.3. After the calibration, stimulus images of 600×400 pixels spanned 8.7°×5.6° of visual angle corresponding to an inter-dot spacing of 0.1015° in the original undeformed lattice. Stimuli were displayed on a 22 inch Samsung SyncMaster 226BW LCD monitor with fast response time (2 ms), placed at a distance of 1.12 meters from the subject. Viewing distance was maintained using a chinrest.

Experiments

Because the present study focused on free visual exploration, the task was designed for measurement of accuracy rather than reaction time. Nevertheless, because we wanted to quantify the duration of exploration for different response types we additionally measured reaction times. All results regarding reaction times should therefore be interpreted not in the classical sense but as a reflection of exploration duration. Two different experiments were carried out. In Experiment 1, instructions given to subjects emphasized accuracy, but mentioned that speed was important as well (see Procedure for details). In Experiment 2, subjects were given no instructions regarding response speed. In addition, in the latter experiment we carried out concurrent eye-tracking to identify the pattern of saccades/fixations during each trial.

Subjects

Twenty six subjects (15 females), aged 20–34 years (M age = 25.88, SD = 4.20), took part in the study. They were either volunteers or undergraduate psychology students who received course credit for participation. All had normal or corrected-to-normal vision, and no known neurological or visual impairments. Fourteen subjects (10 female, M age = 23.93, SD = 2.64) participated in the first experiment and twelve (5 female, M age = 28.17, SD = 4.61) in the second experiment. In each experiment, subjects were assigned to one of two experimental conditions (see Experimental Design), resulting in N = 7 and N = 6 subjects in each condition for Experiment 1 and 2, respectively.

Procedure

Subjects were instructed that upon viewing each target stimulus their task was to decide whether the dot pattern represented something meaningful. They had to respond by pressing buttons “A” (if they decided that nothing meaningful was there—response “Nothing”), “S” (if they thought they saw something but were uncertain what it was—response “Uncertain”) or “L” (if they perceived something meaningful in the pattern and knew what it was—response “Seen”). Each trial started with a fixation mark, presented centrally for a random variable duration (500–1000 ms in Experiment 1 and 1500–2000 ms in Experiment 2). The fixation mark was followed by the target stimulus, presented continuously until the subject responded by pressing one of the three buttons. After the button-press response, a message was displayed on the screen asking subjects to verbalize their response. An experimenter was present in the room throughout the experiment and manually recorded the subject's verbal responses. When subjects were able to identify an object they had to name it explicitly. When they were uncertain about the object, they were instructed to guess, if they could. The task started with a practice block of 14 trials, followed by 7 experimental blocks, with a break period after each block. A block consisted of 50 trials corresponding to stimuli obtained for the 50 objects at a given g level. Within each block, presentation order of stimuli was randomized.

In Experiment 1, it was stressed that subjects should press one of the three response buttons as soon as they had reached a decision, but to take as much time as was needed to reach that decision. It was also suggested that accuracy was slightly more important than speed. In Experiment 2 subjects were only told to take as much time as was needed to respond, with no mention that reaction time was a relevant variable.

In Experiment 2, an ASL EyeStart 6000 system was used to record eye movements (see Identification of fixations). Calibration was conducted before each experimental block using a nine-point display, according to the indications included in the manufacturer's manual. To correct for potential shifts of eye position estimates in between calibrations, the fixation mark was presented for an extended duration compared to Experiment 1. Subjects were instructed to maintain precise fixation on the fixation mark at the beginning of the trial and this information was later used to correct for potential shifts in each trial.

Experimental design

In both experiments we used a between-subjects design, with two versions of the task: (1) ascending (each block contained stimuli with the same g level – corresponding to the 50 objects in Figure 1D – and blocks were ordered from g = 0 to g = 0.3); (2) descending (this version was similar to the previous one, except that blocks were presented in reverse order of g value – i.e., from g = 0.3 to g = 0). We used these two conditions in order to investigate whether previous exposure affects stimulus visibility. Whereas in the ascending condition visibility relies mainly on stimulus evidence, in the descending procedure visibility relies on an interaction between top-down sensory expectations and stimulus evidence. Each subject was assigned to one of these two conditions.

Sigmoid fitting for threshold identification

To identify thresholds of object recognition we fitted the sigmoidal-shaped response and accuracy curves corresponding to each subject with a sigmoidal function f_sig dependent on the gravitational constant, g:(12)where, a is the slope of the sigmoid at the threshold, b is the vertical offset, and φ is the horizontal shift on the g axis, i.e. the threshold.

The fit was implemented using a gradient descent method to minimize approximation error. Instead of using percentages, values of the accuracy curves were normalized to the interval [0..1] before fitting, to match the sigmoid function described in Eq. 12. The relevant parameter for our purposes was φ, that is, the threshold. It represents the point on the g axis where the sigmoid function crosses half of its maximum amplitude (offset not considered).

Identification of fixations

Fixations were identified by a simplified version of the velocity based algorithm introduced by Nyström and Holmqvist [71] that uses two adaptive velocity thresholds. A saccade is detected when the velocity of eye movements rises above the saccade identification threshold, v_I. A second, lower, threshold, v_S, is used to identify the onset and ending of the detected saccade. The original algorithm [71] starts with a high v_I and in each iteration updates its value according to the formula:(13)where, μ_v and σ_v are the mean and standard deviation respectively of all velocities smaller than v_I, and k_vI = 6.

The iterative process stops when the difference between two successive values of v_I is below 1°/s. Next, the saccade onset threshold, v_S, is computed in similar fashion:(14)where, k_vS = 3.

In our case, setting k_vI = 3 and k_vS = 1.5 yielded better saccade identification. As a difference from the Nyström and Holmqvist algorithm [71], we could not identify glissades because our eye-tracker sampling period (20 ms) was in the range of glissade duration. Therefore, we used the same threshold to identify both saccade onset and ending. Once saccades have been identified, fixations were defined as samples between successive saccades. Fixations in which more than half of the eye position samples could not be correctly recorded by the eye-tracker (due to pupil loss, corneal loss, or blinks) were discarded (this was extremely rarely the case). To cope with variable level of noise in eye tracking recordings the following measures were taken. First, both thresholds were computed on a trial basis. Second, for each trial, potential shifts in eye position estimates were corrected by using the position of the fixation mark onto which subjects were explicitly asked to fixate before stimulus onset. Third, eye-tracker calibration was performed before each experimental block (fifty trials).