The authors have declared that no competing interests exist.
Conceived and designed the experiments: VP WB AK. Performed the experiments: VP. Analyzed the data: VP WB AK. Contributed reagents/materials/analysis tools: VP. Wrote the paper: VP WB AK.
In this study the saliency of hardness and softness were investigated in an active haptic search task. Two experiments were performed to explore these properties in different contexts. In
In daily life, we encounter many compliant objects. Common examples of soft objects are the sponge one uses for washing up and the stuffed animals children play with. It is important to be able to distinguish efficiently between soft and hard objects, for instance when judging the ripeness of fruit. Also in medical palpation procedures sensitivity for compliance is necessary, since an increased softness or hardness of a body part (e.g. the skin) can indicate a disease.
Several studies have been performed to determine the discrimination threshold (just noticeable difference, JND) in softness or hardness perception
The discrimination threshold indicates how well one can distinguish between two stimuli that differ in compliance. Other interesting questions concern the efficiency with which hardness or softness is perceived and whether these features are salient. Salient features are easily accessible object properties that are almost instantly perceived. Therefore, these properties are likely to be important for the recognition of objects and used in the early phases of object recognition
The saliency of an object feature can be investigated in a search task, where one has to determine whether a target is present or not among a variable number of distractors. If a target object property is easy to find, it stands out among the distractor objects' properties. This is called the pop-out effect, which has originally been described in visual search
If the target is easily distinguished from the distractors, a parallel search strategy can be used, in which all items are examined at once. As a result, the time to search for the presence of a target can be very short and is independent of the number of items. If the target is more difficult to distinguish from the distractors, the items have to be explored one by one. This is called a serial strategy and is less efficient. In a serial strategy, the time to search for the presence of a target increases if more items need to be searched. Following this reasoning, the efficiency of search can be measured by plotting the reaction time, i.e. the time to decide whether or not a target is present, against the number of items that are explored. The slope of the regression line fitted through the reaction time data is called the search slope. The search slope indicates the difficulty of the search and the search strategy: a flat slope indicates a parallel strategy, whereas a positive slope implies a serial strategy. Therefore, the search slope can be used as a tool to measure search efficiency and the saliency of the target feature. However, caution must always be taken with the interpretation of slope values, because the exact distinction between the two search strategies is not very strict. In the visual literature, a range of search slopes can be found
Haptically searching for objects can be very efficient. Research into haptic search reveals a number of haptic salient features (e.g. roughness
To sum up, in this study we wanted to further explore the saliency of hardness and softness in a haptic search task that involves active grasping of multiple objects. In this way, exploratory movements are not restricted and perception is not limited to a small part of the hand; the whole hand can be used to determine compliance, which might be more efficient when multiple objects are explored. The main question is whether a pop-out effect can be found for hardness or softness or both in an active search task.
Two experiments were performed in this study, in which participants had to determine whether a target was present among distractors. In
Ten participants (7 males) with a mean age of 21±3 years were recruited for the experiment. All were strongly right-handed as confirmed by Coren's test
The stimuli consisted of spheres with a radius of 9.3 mm, which was the same size as used in the study of Plaisier et al.
A: The two halves of the mould, with a sphere in the right half. B: The experimental set-up of
A piece of string was attached to each sphere and the spheres were grouped in bundles of 3, 4, 5, 6 or 7 spheres. Seven spheres was the maximum number of items that could fit comfortably in the hand. A bundle could be hung onto a hook, which was attached to a tripod (see
To measure the reaction time, the tripod was placed on a weighing scale (Mettler Toledo SPI A6). When the participants touched the spheres, the resulting weight change started the clock. The end of the reaction time was determined by a vocal response, recorded with the microphone of a headset placed on the participants' heads. The reaction time was sampled with a frequency of 100 Hz. The weighing scale had a delay of 90±20 ms, which was added to the raw reaction time data.
The compliance of the spheres was measured with an Instron 5542 Universal Testing Machine. A sphere was pressed between two flat metal plates with 1 mm/s, and the force and compression were measured (in steps of 0.1 N per sample). The compression was stopped at 10 N for the hard and middle-soft spheres, and at 2 N for the soft spheres. The lower endpoint for the soft spheres was chosen to make sure they remained intact. For each compliance type, 5 spheres were measured 3 times, totalling 45 measurements.
The 15 measurements for each compliance type were averaged and the resulting values are displayed in
Grey areas represent confidence intervals (2 standard deviations).
In each trial, blindfolded participants had to grasp a bundle of spheres and determine as quickly as possible whether a target was present or not among distractors. The experiment was divided into two sub-experiments. In
The order of
Participants were seated in front of the weighing scale. They were told the nature of the task and instructed to try to determine the presence of a target as quickly as possible, but also to make as few mistakes as possible. Before each trial, they put their flat hand, with the palm up, upon a resting cushion underneath the bundle of spheres. They were instructed to lift their hand and initially grasp the whole bundle, but if necessary explore the spheres individually or drop spheres out of their hand. As soon as they knew whether a target was present or not they responded by calling out the Dutch equivalents of ‘yes’ or ‘no’. They received feedback whether their answer was correct. Before the start of a condition, participants performed at least 20 practice trials continuing until they answered correctly 10 times in a row. This was done to get familiar with the task and to find a fast strategy to perform the task. The maximum number of practice trials needed was 27. Trials answered incorrectly were repeated at the end of the session.
One trial (0.03%) was removed from the analysis because of a measurement error. In addition, outliers in the reaction time data were removed from further analysis. A trial was considered an outlier if it differed more than 4 standard deviations from the mean, when the trial itself was not included in the calculations of the mean and standard deviation.
The mean reaction times were plotted against the number of items for each participant, separately for each condition and for target-present and absent trials. A straight line was fitted through the data points, giving the search slopes and intercepts. Furthermore, for each trial was scored whether participants dropped spheres out of their hand. The percentage of trials in which this behaviour was seen was calculated for each number of items. For
A 2 (experiment)×2 (target type)×2 (target presence) repeated measures Analysis of Variance (ANOVA) was conducted on the slopes and intercepts. Post-hoc tests were performed by using paired-sample
The percentage of incorrect answers is displayed in
experiment | condition | 3 | 4 | 5 | 6 | 7 | |
1a | Hard | present | 0 | 1 | 3 | 10 | 6 |
absent | 1 | 0 | 1 | 1 | 0 | ||
Soft | present | 2 | 3 | 5 | 10 | 14 | |
absent | 0 | 0 | 0 | 0 | 0 | ||
1b | Hard | present | 2 | 0 | 2 | 3 | 0 |
absent | 0 | 0 | 0 | 0 | 0 | ||
Soft | present | 0 | 1 | 2 | 0 | 6 | |
absent | 0 | 0 | 1 | 0 | 0 |
Search slopes of experiments 1a and 1b are shown in
A: Search slopes for
Slope (s/item) | Intercept (s) | ||||
experiment | condition | present | Absent | present | absent |
1a | hard | 0.47 |
0.92 |
0.54 | 0.20 |
middle-soft | 0.52 |
0.97 |
0.16 | −0.50 | |
1b | hard | 0.080 |
0.19 |
0.43 |
0.24 |
soft | 0.048 |
0.27 |
0.53 |
0.16 | |
2 | hard | 0.010 | −0.013 | 0.54 |
0.91 |
soft | 0.12 |
0.13 | 0.43 |
1.47 |
The number of times a sphere was released from the hand is plotted in
A: The hard-target conditions in
Another natural way to determine the compliance of an object is pressing it
A 3×3 grid was used to present the stimuli (see
A. Stimulus display with 4 hard distractors and one soft target. B. A hard and a soft sphere on poles.
To measure the reaction time, the grid was placed on the weighing scale. As the participants touched the grid, the weight change started the clock. The end of the reaction time was determined by a vocal response, recorded with a headset, similar to
Two conditions were performed in
Participants were instructed to initially put down their whole hand flat upon the grid, but if necessary they could then lift their hand and press again or use their fingers to individually touch the items. They were told to only press down on the spheres or press from the sides. Between trials they held their hand on a resting platform. Before a condition, participants performed a practice session with the same requirements as in
One trial (0.06%) was removed from the analysis due to a measurement error. Outliers were removed from further analysis using the same criterion as in
The position of the distractors could influence the ability to detect the target, especially in the case of a soft target. To investigate this, the relation between the number of adjacent distractors and the reaction time was calculated. A regression line was fitted through the averaged reaction time against the number of distractors that directly neighboured the target (horizontally or vertically; direct distractors). A similar fit was made for all adjacent distractors, that is, including the ones that diagonally enclosed the target (+diagonal distractors). A weighted fit was used to adjust for the difference in number of trials for each number of adjacent distractors. In addition, the relation between the reaction time and the location of the target was determined by averaging the reaction time for each of the 9 target locations.
A 2 (target type)×2 (target presence) repeated measures ANOVA was performed on the search slopes and intercepts. Furthermore, a 2 (target type)×2 (direct/+diagonal distractors) repeated measures ANOVA was conducted on the slopes and intercepts of the reaction time against the number of adjacent distractors. Finally, a 2 (target type)×3 (horizontal position)×3 (vertical position) repeated measures ANOVA was performed on the reaction times averaged over target location. The significance value was set at 0.05. If the sphericity assumption was violated according to Mauchly's test, a Greenhouse-Geisser correction was used. Post-hoc tests were performed using paired-sample
As in
3 | 5 | 7 | 9 | ||
hard | Present | 2 | 0 | 0 | 3 |
Absent | 0 | 0 | 1 | 0 | |
soft | Present | 1 | 1 | 10 | 8 |
Absent | 0 | 0 | 0 | 1 |
Search slopes of
In the analysis of the intercept, effects of target type (
Based on the results of
Lines are plotted separately for direct distractors (black) and direct+diagonal distractors (grey). Point size indicates the weight of the point (number of trials) according to a logarithmic scale.
Slope (s) | Intercept (s) | |||
direct | +diagonal | direct | +diagonal | |
hard | 0.019 | 0.012 | 0.58 |
0.58 |
soft | 0.32 |
0.20 |
0.63 |
0.61 |
p<0.05,
p<0.01.
Besides the position of the distractors, also the location of the target might have been of influence on the reaction time, as illustrated in
Each plot is located at the corresponding x- and y- position of the display. Error bars represent standard errors.
There existed a very subtle difference in texture between the hard and the (middle-) soft spheres, where the hard spheres were somewhat smoother. To investigate whether the texture difference could have explained the results, a control experiment was performed in which participants had to quickly classify the stimuli.
Ten new right-handed participants (5 males) with a mean age of 24±3 years took part in the experiment. They had no previous experience with the stimuli. All signed an informed consent form and were paid for their contribution.
The same experimental set-up as in
One item was placed on the display and participants had to classify the item in two separate tasks. In the compliance task, they had to say whether the item was hard or soft and in the texture task, participants had to choose between smooth or rough. The order of the tasks was counterbalanced between participants. Participants were instructed, similar to
A 2 (task)×2 (type) repeated measures ANOVA was performed on the reaction times. An effect of task (
The aim of this study was to investigate the saliency of hardness and softness in an active haptic search task. In
In
When a hard target was searched for among soft distractors, reaction times were short. In addition, the slope of the reaction time against the number of items (search slope) that were searched was shallow. This means that the time it takes to determine the presence of a target does not depend on the number of items in the hand. A completely different behaviour was seen when the distractors were middle-soft. The reaction times were much higher and the search slope was almost half a second per item in target-present trials, or a second per item in target-absent trials. This indicates that the items had to be explored one by one, thus in a serial way, which would explain the increase in reaction time with more distractors. The behaviour of participants in
The same results were found in the search for a soft target. If a soft target was searched for among hard distractors, the search slope was flat and the reaction times were low. This implies that the reaction time is independent of the number of items. This was not the case in the search for a middle-soft target among hard distractors. The reaction time increased markedly with more items, especially when no target was present. In addition, participants more often dropped one or more spheres out of their hand when searching for a middle-soft target. This indicates that the items were searched for with a serial strategy.
Taken together, the results show that hardness and softness can be efficiently searched for when the difference between target and distractors is large. However, the slope values of
In theory, one would expect a 2∶1 relationship in the search slopes for target-absent and target-present trials if the search strategy were serial. If the items are explored one by one and the search is stopped when a target is found, then on average half of the items are explored in target-present trials, whereas all items need to be searched in target-absent trials. So, search slopes in target-absent trials would be twice as high as in target-present trials. In
Another point of discussion is that participants might not have based their judgement on compliance alone. Although we tried to avoid other perceptual differences between the item types besides compliance, possible other cues, like weight or texture, should be considered. First, there were some small differences in texture between the item types, where the hard spheres were somewhat smoother. Therefore, a control experiment was performed, which showed that participants classified the stimuli much faster and with fewer errors based on compliance than on texture. The difference in texture was much harder to perceive, making it unlikely that participants based their judgement on texture. Furthermore, the soft and middle-soft spheres were made of the same rubber material, giving the same texture difference with the hard spheres. If participants had performed the task using texture as a useful cue, one would also expect low search slopes in
Secondly, the soft spheres were hollow, whereas the hard spheres were solid. It cannot be ruled out that ‘hollowness’ was perceived instead of softness. With our current stimulus materials it was not possible to make soft items that were also solid, but it could have influenced the results. Nevertheless, it might be questioned what hollowness is, whether it can be perceived and how it differs from the perception of softness.
A third possible factor that could have influenced the perception of hardness in
In
However, in the search for a hard target, the presence of the target causes a large increase in total weight, which could have been used as a cue. On the other hand, if weight had popped out instead of compliance, one would expect a lower search slope in the search for a heavy item among light distractors than the other way round. Yet, the slope in the search for the hard, heavy target was higher than in the soft, thus light, target search. To exclude the possible influence of this weight difference, the spheres were placed on a display in
The difference in search efficiency between the search for a hard target in
The search slope for the hard-target condition in
Unexpectedly, we found a search asymmetry in
Furthermore, if the target was located close to the fingers, the target was found quite fast, whereas it was more difficult when the target was located near the palm of the hand. When the target is located near the palm of the hand and/or many hard distractors surround it, the hand is blocked by the distractor items. The target cannot be compressed enough to perceive it as a soft target. Since there were also empty spots on the display, participants might have mistaken the target for an empty space. They had to press again with their fingers to make sure no target was there and thereby increased their reaction time. So, the higher reaction times in the search for a soft target were not caused by the inability to locate the target, but by the inability to compress it.
An interesting finding is that the slopes of target-absent and target-present trials did not differ in
To summarize, a pop-out effect for both hardness and softness was found. When the items are arranged in a 2D-display the search is even easier for a hard target. Because weight cues are not possible in this set-up, the pop-out effect was not caused by the weight differences in the items. If items were placed on a 2D-display, the search is more difficult for a soft target, but this is caused by the inability to compress the target, since the hand is blocked by the hard distractors. In conclusion, both hardness and softness are salient haptic features. This holds for passive and active search and for different exploratory procedures that are used for compliance perception. This knowledge could be useful for the development of simulation systems. These tools can, for example, be used for the training of palpation in medical students.
The authors would like to thank Betty Verduyn for her assistance with the Instron machine.