Introduction

Under a random incentive mechanism (RIM) subjects usually respond to numerous tasks (e.g. different binary choice questions, bidding for an object in several rounds, etc.) and at the end of the experiment one of the tasks is randomly selected and played out for real. RIM provides incentives for truthfully responding to all tasks while only paying one of them. This reduces expenditures for experimental studies and excludes wealth effects from paying all choices sequentially during the experiment as well as portfolio effects from paying all choices at the end of the experiment. Due to theses appealing features, RIM has been used in many experimental studies in psychology, e.g [1], [2] and economics, e.g. [3]–[5].

However, it was pointed out by Holt [6] for binary choice between lotteries that RIM is not necessarily incentive compatible. If the reduction of compound lottery axiom holds, RIM only provides incentives for truthfully reporting preferences which satisfy the independence axiom. Since there exists abundant evidence that independence is often violated, the argument of Holt challenges RIM seriously. This motivated several experimental studies aiming to test whether RIM does elicit true preferences [7]–[16]. All these studies did not observe serious distortions induced by the use of RIM. A convincing reason for this result is the isolation hypothesis from prospect theory [17] which implies that subjects evaluate each task in a RIM independently of the other tasks.

This note presents a simple experiment which tests isolation and incentive compatibility of RIM in the presence of asymmetrically dominated choice problems. The literature of context-dependent choice has shown that adding asymmetrically dominated alternatives in the set of options can systematically influence choice behavior [18]–[21]. In contrast to these studies, in the present experiment asymmetrically dominated alternatives are not included in the set of options in a given task but asymmetrically dominated choice problems are included in a RIM as additional, independent tasks. Given that isolation holds, choice behavior under RIM in one task should not be influenced by the presence of a different task even if preferences are menu-dependent. Asymmetrically dominated choice problems are understood as follows: Suppose there is a choice between a safe lottery S and a risky lottery R. Then a second choice problem, also consisting of a safe alternative S′ and a risky one R′, risky-dominates the first problem if R′ dominates R and S′ is dominated by S. Analogously, a third problem, consisting of a safe alternative S″ and a risky one R″, safely-dominates the first one if S″ dominates S and R″ is dominated by R. Our hypothesis is that in the presence of a risky-dominating choice problem alternative R (S) looks less (more) attractive, leading to a higher fraction of S choices. The opposite should hold in the presence of a safely-dominating choice problem.

Methods

Two experiments were run at the University of Kiel with altogether 581 subjects. Subjects gave written consent to participate in the study. As there was no possibility to lose money in the experiments, approval of an ethics committee was neither required nor obtained. In both experiments subjects were randomly assigned to one of five groups, referred to as Groups 1, 2.1, 2.2, 3.1, and 3.2 in the sequel. For Experiment 1 the stimuli received by the groups (in each case printed on a single sheet of paper) are presented in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Design of Experiment 1.

https://doi.org/10.1371/journal.pone.0090742.t001

In Group 1 subjects had just to choose between Options A and B. Subjects were told that everybody would receive the payoff of the chosen option in cash directly after the experiment and that the payoff of Option B would be determined by a coin flip. In Groups 2.1 and 3.1 there were two choice problems (presented in the order of Table 1) and a RIM was employed, i.e. there was a first coin flip which determined whether the first or the second choice problem was played out for real and a second coin flip which determined the payoff if one of the risky options (B, D, or F) was chosen. Group 2.2 (3.2) differed from Group 2.1 (3.1) only by the order in which the choices were presented, i.e. the choice between Options A and B was presented first in Groups 2.2 and 3.2. In all groups the left-right positioning of options was randomized. Design of Experiment 2 was identical to that of Experiment 1 with the only exception that the payoff of all safe options (i.e. Options A, C, and E) was increased by one Euro.

The aim of Group 1 is to elicit true preferences of subjects between Options A and B as a design with one choice problem played out for real offers perfect incentives to state true preferences [22]. Also in Groups 2 (3) we elicit preferences between Options A and B which could however be biased as the design here involves additionally a risky-dominating (safely-dominating) choice problem. If the isolation hypothesis holds, the fraction of subjects choosing A should be identical in Groups 1, 2, and 3. If isolation is violated, the additional choice problem in Groups 2 and 3 may influence the choice between A and B. In Group 2 Option A dominates Option C whereas B is dominated by D. Analogous to the evidence of asymmetrically dominated alternatives in the context-dependent choice experiments this could make Option A look more and Option B less attractive, leading to a higher fraction of A choices compared to Group 1. The opposite could be expected for Group 3 as here A is dominated by E whereas B dominates F. Comparing the fraction of B choices in Groups 2 and 3 with those in Group 1 provides a simple and direct test of isolation. In Groups 2 and 3 we observe choices between A and B which are embedded in a different RIM. Therefore, comparing the fraction of B choices in Groups 2 and 3 provides a test on the incentive compatibility of RIM.

Results

The results of both experiments are presented in Table 2 which states for all groups and both choices the fraction of subjects choosing the risky lottery. First, we can see that in Groups 2.1 and 2.2 indeed by far most subjects choose the risky option D. These subjects may be reluctant to choose B leading to a higher fraction of observed A choices as compared to Group 1. Also in Groups 3.1 and 3.2, most subjects chose as expected the safe option E and for those A could look less attractive.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results.

https://doi.org/10.1371/journal.pone.0090742.t002

Let us first look at Experiment 1. The differences between choices of B in the single groups are presented in Table 3 along with tests according to the test-statistics of Conlisk [23]. All tests are two-sided and *** (**, *) refers to a significance-level of 1% (5%, 10%). While 82.8% of subjects chose B in Group 1, this fraction reduces to 51.9% and 59.3% in Groups 2.1 and 2.2 respectively. In both cases, the difference is significant. As expected, A turns out to be more attractive in Groups 2 leading to a significant violation of isolation and, therefore, to a failure of isolation. In Group 3 we have expected the opposite effect as in Group 2 but the fraction of B choices is not significantly higher than in Group 1. This may be due to a large fraction of subjects preferring B anyhow and hence due to a ceiling effect. There are also in each case significant differences between the choice of R in Groups 2 and 3. This shows that the choice behavior in a RIM depends strongly on the other tasks involved. All four tests of the RIM (i.e. 2.1 vs. 3.1, 2.1 vs. 3.2, 2.2 vs. 3.1, and 2.2 vs. 3.2) lead to significantly different choice behavior. Therefore, RIM is not incentive-compatible in our experiment.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Differences in the Choice of B in Experiment 1.

https://doi.org/10.1371/journal.pone.0090742.t003

Ordering effects between Groups 2.1 and 2.2 as well as between Groups 3.1 and 3.2 can be observed which are all in the expected direction, i.e. the choice behavior between A and B should be less affected, if this choice is presented first. However, these effects are insignificant. The relatively small ordering effects can be explained by the fact that in the instructions to Groups 2 and 3 all alternatives were presented prior to the response of subjects.

Table 4 reports the differences of fractions of B choices in the single groups for Experiment 2. Here, in contrast to Experiment 1 we do not observe significant differences between Group 1 and Groups 2 but now the differences between Group 1 and Groups 3 turn out to be significant such that isolation is again violated. Also between Groups 2 and 3 in three out of four cases responses are significantly different which shows that RIM is not incentive compatible.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Differences in the Choice of B in Experiment 2.

https://doi.org/10.1371/journal.pone.0090742.t004

Discussion

This note has shown with a very simple experimental design that integrating asymmetrically dominated alternatives in a random incentive mechanism can manipulate choice behavior systematically. In our study isolation is violated significantly and RIM does not elicit true preferences. We ran eight tests of isolation (fraction of B choices in Group 1 versus the other four groups in both experiments) and observed a significant violation in four cases. This is rather clear-cut and not mixed evidence because isolation cannot be used to justify use of RIM unless it holds generally; holding 50 percent of the time clearly will not do. Additionally, we ran eight tests of the incentive-compatibility of RIM (fraction of B choices in Groups 2 versus Groups 3 in both experiments) and in seven out of these eight tests responses were significantly different. We can conclude that choice behavior in RIM depends substantially on the other tasks involved and asymmetrically dominated alternatives have an impact in the hypothesized direction.

Altogether, the presented results demonstrate that a common methodology in experimental studies may induce distortions. Further research is needed in order to investigate how serious these distortions are in practice.