Introduction

With a prevalence of 40 to 65%[1,2], cognitive impairment is one of the most common symptoms of multiple sclerosis (MS) that may already arise at an early stage of the disease[3]. Typically affected domains are memory, attention, information processing and executive functions, whereas language and simple attention span are scarcely impaired (for review: 4-6).

Notably, cognitive deficits negatively affect patients’ quality of life[7], ability to work[8] and rehabilitation outcome[9]. Despite those debilitating consequences of cognitive impairment, it is not routinely assessed during clinical examination or trials. One reason for this shortcoming is that most available cognitive tests have been designed for other patient groups (i.e. stroke) and thus are not sensitive enough to assess cognitive functions in MS, where, particularly during an early stage, only mild deficits predominate. Moreover, to cover the variety of potentially affected functions in MS the conduction of comprehensive test batteries is required, making assessment time-consuming and impracticable. Short screening tests (i.e. Paced Auditory Serial Addition Test; PASAT) are susceptible to measurement errors, assess only few facets of cognitive functioning and are not suitable to assess changes in cognitive functions due to considerable practice effects[10]. Self-reports are just as little suited for the assessment of cognitive deficits, since they strongly correlate with depressive symptoms[4] and diverge with objective measures of cognitive impairment[6].

The importance of the assessment of cognitive impairment in MS patients on the one hand and the lack of suitable instruments for this purpose on the other hand, emphasize the need to identify a suitable measure of cognitive deficits in MS patients circumventing the problems of currently available tests. Such a test could be the visual search task. This paradigm was introduced by Treisman and Gelade[11] and is one of the best understood tasks in the field of cognitive neurosciences. In this task, a target stimulus, presented on a computer screen, has to be detected amongst similar distracting stimuli via button press (see Figure 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Illustration of the visual search tasks.

A: single feature search – colour. The red X has to be found amongst several green Xs varying in number. B: feature conjunction search. The red X has to be found amongst green Xs and red Ys varying in number. C: difficult feature search – orientation. A white backslash (\) has to be found amongst white slashes (/) varying in number.

https://doi.org/10.1371/journal.pone.0081531.g001

Depending on the displayed array of stimuli visual attention or purely perceptual processes (figure ground segregation) are required[11]. This allows the separate assessment of sensory and attention-related deficits. Additionally, position priming can be induced by repeating the target location leading to faster responses, which is an indicator of implicit perceptual memory[12]. Apart from priming, practice effects are typically not observed[13], suggesting that the visual search task can be reliably used to assess changes in cognitive functions via repeated testing. To broaden the diagnostic information of the visual search task, the response key can be replaced by a touch screen, requiring the subject to directly touch the target on the screen[14]. In this way, indicators of visuomotor functions can be assessed such as movement time or accuracy of pointing. Moreover, using a pointing response instead of the more conventionally employed button-press response offers one further advantage. Conventionally the participant has to decide whether a target item is present or not and press the corresponding button to indicate their decision. However, this means that a substantial number of trials without a target item (catch trials) have to be presented. These trials are frequently not used for the final analysis. When a pointing response is used, the accuracy of the pointing response is proof that the target item has been detected. Catch trials are thus superfluous and thus more data can be collected in less time.

To sum up, the visual search task seems to be an ideal tool for the assessment of cognitive impairment in MS: It assesses different facets of cognitive functions and motor behaviour, shows no practice effects and thus in principle could be able to assess disease- or medication-related changes in cognitive functions of MS patients.

In the present study we examined MS patients and healthy individuals both with our modified visual search task and with a set of standard neuropsychological tests. Our aim was to determine which cognitive assessment procedure would provide the most reliable discrimination between individuals with and without MS.

Materials and Methods

Visual search task

In this task, participants had to search a target amongst a varying number of distractors. Three different versions of visual search tasks were conducted (see Figure 1). All tasks were programmed in E-prime 2 (Psychology Software Tools, Inc., Pittsburgh, PA). All stimuli were presented against a black background on a 17” computer monitor covered by an add-on touch screen (KTMT1700 USB; Magic touch, Keytec Inc., Garland, Texas) and driven by a Pentium M 750 notebook. Table 2 shows the different parameters of the search tasks, and Figure 2 illustrates the sequence of each trial.

Block no.	Task	No. of practice trials	Set sizes: green X	Set sizes: red Y	Set sizes: white /	No. of trials per set size/set size combinations*	No. of standard/priming trials	Total no. of experimental trials
1	single feature search – colour	10	7/15/31	–	–	12	18/18	36
2	feature conjunction search	10	3/4/7/8/15/16*	3/4/7/8/15/16*	–	12	18/18	36
3	difficult feature search – orientation	10	–	–	7/15/31	12	18/18	36
4	single feature search – colour	–	7/15/31	–	–	12	18/18	36
5	feature conjunction search	–	3/4/7/8/15/16*	3/4/7/8/15/16*	–	12	18/18	36
6	difficult feature search – orientation	–	–	–	7/15/31	12	18/18	36

Table 2. Parameters of the different visual search tasks.

Notes: In priming trials, the location of the target was identical to the one in the preceding trial. Priming trials were presented in sequences of three. This means that the same location was repeated three times. In standard trials, the location of the target item was different from the target location in the previous trials.* The two kinds of distractors were paired such that there was the same total number of distractors (set sizes: 7/15/31) as in the other two tasks.

CSV

Download CSV

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. The sequence of a trial using the example of the single feature search.

At the beginning, a central fixation cross was displayed along with the instruction to hold down the start button. Participants had to press a button on a Serial Response Box (Psychology Software Tools, Inc., Pittsburgh, PA). At the moment of button press, the stimuli appeared on the screen. The participants were instructed to visually search the target, and release the button only after they had detected the target. Next they should touch the target on the screen. The trial was completed (and the next trial initiated) when the touch screen detected a touch or when more than 3000 milliseconds (ms) had elapsed since button release. Trials, that were aborted because no touch was registered within the 3000 ms time window, were excluded from the analysis.

https://doi.org/10.1371/journal.pone.0081531.g002

Reaction (search) time (until button release), movement time (time between button release and target touch) and pointing error (distance between touch and target location in mm) were computed (see Table 3). Furthermore, the percentage difference between the reaction time for the first and third presentation of priming trials was computed as an indicator of implicit location learning. The assumption being that when participants remember the location of the previous trials, they should take less time to detect the target, thus leading to shorter reaction time for the third repetition and thus a bigger difference between reaction times for the first and third presentation of the same target location.

Test	MS patients	Healthy individuals	p
SPART/SPARTDR	18.57 (±5.39)/6.51 (±2.50)	20.05(±6.04)/7.08 (±2.57)	.144/.335
PASAT-3’’	44.69 (±10.09)	49.73 (±9.44)	.028
Digit span forward/backward	7.76 (±1.88)/6.71 (±1.93)	8.32 (±1.9)/8.18 (±1.91)	.194/.001
Spatial span forward/backward	8.34 (±2.27)/7.76 (±1.82)	8.57 (±1.99)/8.4 (±1.71)	.630/.115
Logical Memory I	25.63 (±8.98)	29.68 (±6.28)	.023
Go/Nogo; reaction time/errors	572.3 (±85.13)/1.1 (±2.36)	541.75 (±73.68)/0.48 (±1.24)	.094/.158
Divided attention; omissions	3.5 (±4.80)	1.08 (±1.4)	.005

Table 3. Group means (±standard deviations) and group differences in the standard tests.

Notes: SD: standard deviation; SPART: 10/36 Spatial Recall Test; SPARTDR: 10/36 Spatial Recall Test delayed recall; PASAT-3’’: Paced Auditory Serial Addition Test-3 seconds.

CSV

Download CSV

Results

ROC analysis

Figure 3 illustrates the ROC-curves and Table 5 shows the respective Areas Under the Curve (AUC). Mean reaction time and movement time in the different visual search tasks, as well as for the combined data of all three tasks, turned out to discriminate best between healthy individuals and MS patients, as indicated by the highest AUC values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Receiver-Operator-Characteristic-Curves of the tests with significant group differences.

Abbreviations: PASAT-3’’: Paced Auditory Serial Addition Test-3 seconds; RT: Reaction time; MT: Movement time. For reasons of clarity, only the curves of the combined mean RT and mean MT of all three visual search tasks are displayed.

https://doi.org/10.1371/journal.pone.0081531.g003

Test	AUC (±SEM)	p
PASAT-3’’	.66 (±.06)	<.05
Digit span backward	.69 (±.06)	<.05
Logical Memory I	.62 (±.06)	<.05
Divided attention	.69 (±.06)	<.05
Singe feature search colour – RT	.75 (±.06)	<.001
Singe feature search colour – MT	.78 (±.05)	<.001
Feature conjunction search – RT	.74 (±.06)	<.001
Feature conjunction search – MT	.78 (±.05)	<.001
Difficult feature search – RT	.75 (±.06)	<.001
Difficult feature search – MT	.73 (±.06)	<.001
Visual search – RT (all 3 tasks)	.76 (±.06)	<.001
Visual search – MT (all 3 tasks)	.77 (±.06)	<.001

Table 5. Areas Under the Curve (±Standard Error of the Mean) of the neuropsychological tests.

Notes: AUC: Area Under the Curve; SEM: Standard Error of the Mean; PASAT-3’’: Paced Auditory Serial Addition Test-3 seconds; RT: Reaction time; MT: Move time.

CSV

Download CSV

Discussion

We compared a selection of commonly used cognitive tests with the paradigm of visual search regarding their ability to discriminate between healthy individuals and MS patients and regarding their potential practice effects. Reaction time and movement time in the visual search task turned out to discriminate best between the two groups, without showing any practice effects, confirming the results of Ball et al.[13]. This suggests that the visual search task is a suitable method to assess cognitive impairment in MS patients and furthermore potentially disease- or medication- induced changes in MS patients due to robustness against practice effects.

Inferior mean performance in visual-attentional functions were observed in the visual search task for MS patients as indicated by longer reaction times for this group compared to the healthy individuals. However, no group differences regarding priming effects were found, meaning that implicit learning was unimpaired in our MS patients, as previously reported[1]. This is in line with findings in Alzheimer’s patients who also demonstrate intact perceptual priming[17] suggesting that implicit memory is a surprisingly robust skill that remains intact even when many other cognitive functions decline.

The lack of group differences concerning reaction time between patients with and without movement deficits of the right arm, as well as the lack of correlation between reaction time and visual deficits, suggests that this outcome measure is relatively robust against mild motor and visual defects.

Findings from our battery of standard neuropsychological tests mirrored those from previous studies: MS patients showed significantly lower performance in tests assessing verbal long-term memory[18], divided attention[3], information processing speed[2], as well as verbal short-term memory/working memory (digit span backward[19,20]), but not in the simpler version of this task (forward[2]). However, for all of those standard tests the power to discriminate between healthy and MS participants was inferior to that of the visual search task. Interestingly, even the most commonly used test, the PASAT, showed considerable lower discriminatory ability and was moreover prone to practice effects, providing further ammunition for those who are critical of the PASAT’s usefulness as a reliable measure of cognitive function[10].

Given the increasing recognition of cognitive impairment in MS, different strategies have been proposed for its assessment, such as test batteries like the BRB-N[21] or the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS)[22], patient- or family member-reported scales[23,24] and different screenings such as single tests[25], a collection of tests[26] or internet-based cognitive testing[27]. However, whereas test batteries are too time-consuming, self- or family member-reported scales are not reliable[4,6], and many screening tests assess only one or few, mostly speech-dependent aspects of cognition and are often susceptible to measurement errors and practice effects, thus being inappropriate for the assessment of changes in cognitive functions[10,28].

The administration duration of the visual search task was between 15 and 20 minutes. However, as there were no differences regarding the performance during the first three and the second three blocks, the task duration could be shortened to seven to 10 minutes, thus being shorter than most of the available screening procedures. In fact given that all three search tasks discriminated equally well between participants with and without MS, it would be feasible to restrict the examination to just one of the three search tasks, thereby reducing the examination time to approximately 5 minutes.

Despite its brief administration duration, the visual search task assesses many different facets of cognitive functions, especially those frequently neglected by conventional declarative tests, such as visual exploration and visual memory.

Correspondingly, Feinstein et al.[29] recently found that the phenomenon of inattentional blindness, whereby healthy individuals fail to notice a perfectly visible person/object when they are concentrating on another task, is observed less in MS patients. This is taken as a sign of increased distractibility in MS patients. Interestingly, this pervasive deficit is not picked up by the MACFIMS[22] battery: MS patients with abnormal performance in the inattentional blindness task showed normal performance in the MACFIMS tests. The authors conclude that this could explain why some MS patients have difficulties in daily life situations requiring the filtering of distractors or multitasking without showing deficits in test batteries[29]. The visual search task assesses distractibility quite explicitly since successful performance requires that participants focus on a given target feature while filtering out irrelevant but often salient distracting features. This aspect of the visual search paradigm may in part account for its superior power to discriminate between participants with and without MS.

The visual search task offers other advantages which so far have not been mentioned. It is computer-based, therefore can be carried out by the patients without supervision and can also be automatically analyzed, thus yielding objective test results with minimal effort.

Since the mean disease duration of our sample of MS patients was over seven years, we do not know whether the same results can be expected for a sample of early-stage MS patients. We should also note another limitation of our study. Our procedure to identify sensitive cognitive tests was based on a comparison between healthy participants and MS patients. Tests which discriminate reliably between MS patients and healthy adults will help in the diagnosis of MS patients but may not provide the best measures to examine the impact of clinical drugs on disease progression. To find out whether our visual search tasks provide the sensitivity to detect spontaneous or therapy-induced changes over time a longitudinal study will be needed.

In conclusion, visual search seems to be a good tool for the assessment of cognitive impairments in MS due to its good discriminatory power, easy and objective computer-based administration, high accuracy of measurement due to the great number of trials, and assessment of a wide range of cognitive skills, which are often neglected by conventional procedures. Furthermore the visual search task is not susceptible to practice effects (at least in healthy individuals), predestinating this tool for the assessment of cognitive changes, i.e. due to disease progression or disease-modifying treatment.

Acknowledgments

We thank Tanja Stirnweiss and Julia Kratzer for assistance in the recruitment and examination of patients.