Reproductive value

To quantify the effects of a decision, I begin with Fisher's concept of reproductive value [3]. The reproductive value of a worm is proportional to the expected number of its descendants at some distant future time, based on the information available to the worm. The descendants of animals with high reproductive value will, by definition, be a larger part of the future population than those of animals with low reproductive value. (Note that C elegans are usually found as hermaphrodites and that self-fertilization is the dominant method of reproduction in the wild [4], [5]. Thus there is no overlap between the descendants of two worms. In this paper I neglect the effect of the rare males that occur.) Animals that make decisions that maximize their reproductive value will therefore be favored by evolution.

The value of a worm depends on its age and condition. A gravid adult hermaphrodite with 20 eggs in her uterus about to be laid has a value at least 20 times the value of a single egg in the same environment. Yet that adult was herself a single egg a few days ago. Her value must therefore have increased in that time. It increased because the value of the egg derives from its potential to become a gravid adult. The older the egg and the worm that hatches from it become, the closer it gets to adulthood, the greater its chances of escaping dangers and finding resources so as to reach adulthood. Similarly, value depends on condition. A young larva near death from starvation is less likely to reach adulthood than a larva of the same age with abundant stored nutrient reserves, so the first is less valuable than the second.

Value also depends on the environment. A young larva near death of starvation in an environment devoid of food is unlikely to have any descendants, and therefore has low value. Its value is not zero because there is a small chance that it may find food before it dies. The same starved larva is more valuable in the presence of food because it is more likely to reach adulthood, and, if the food supply is very large, because many of its children and grandchildren will reach adulthood.

Less obviously, value depends on information available to the worm through its senses or internal state. The value of the starving larva is very low, but if the larva's chemical senses inform it of food in the vicinity, its chance of survival and therefore its value rise abruptly. It might be argued that the true value (whatever that might mean) of the worm is not changed by changing information—only the worm's estimate of its value is different. Readers more comfortable with this view may want to mentally replace “value” with “expected value” or “estimated value”. In any case, it is this information-based value that optimal decisions must maximize. Because value depends on information, it can change quickly.

Dependence of value on age

Historically, most C elegans eggs have not reached adulthood. The argument is essentially that of Malthus. It is based on two assertions: first, that the average C elegans adult produces many more than one egg, and second, that just one of the children of the average C elegans adult reaches adulthood. The first assertion is based on C elegans reproductive physiology. Under ideal laboratory conditions a C elegans hermaphrodite produces about 300 progeny [6], [7]. The mean brood size in the wild is likely to be less, but several arguments suggest that it is considerably more than 1. The capacity to produce such a large brood is achieved at a high cost: I estimate that the gonad and uterus of an adult hermaphrodite are about ¼ of her volume. It is unlikely that such a large gonad would increase fitness unless the worm actually used the reproductive capacity it affords. Furthermore, even when completely deprived of food an adult hermaphrodite can produce at least 8 progeny by “facultative vivipary”, i.e., consuming her own biomass to produce but not lay eggs, which then hatch internally and eat the mother [8]. It seems safe to assume that the average number of progeny produced by an adult hermaphrodite is at least 8, and probably larger.

That, on the average, only one of these progeny reaches adulthood is clear. C elegans has been in existence for at least 1 million years. The mass of a C elegans adult is roughly 3 µg. If a population started with a single adult doubled 111 times, its mass would exceed the mass of the Earth. Thus, the mean rate of growth of the Earth's C elegans population has been less than 1 doubling per 9,000 years. By similar reasoning, the population cannot have dwindled faster than one halving per 9,000 years. From this it can be deduced that the mean number of descendants of an adult C elegans hermaphrodite that reached adulthood in six months (the maximum plausible generation time) has been between 0.999962 and 1.000038. Even if one assumes a very recent huge expansion of the C elegans population these constraints are only slightly relaxed. For instance, a 100-fold expansion of the C elegans population in the last 100 years would require at most that the average adult give rise to 1.023 adults. In this paper I assume that populations are precisely at steady-state. This assumption is not necessary—the models described below also work for non-steady-state populations and lead to similar conclusions—but the exposition is simplified. The steady-state assumption together with the evidence that an adult produces more than eight eggs implies that at least seven in eight C elegans eggs fail to become fertile adults in the wild.

In a steady-state population, the value of an egg or a larva is proportional to the probability that it will become an adult. In one simple model, the probability of failing to advance in age, i.e., of dying or permanently arresting development, is a constant per unit time, which I call λ, the discount rate. (In reality, of course, λ may vary—the assumption that it is constant is a modeling simplification.) I have estimated λ by several different methods (see λ: discount rate in Methods), which give values from 0.027 h⁻¹ to 0.068 h⁻¹. The method I consider most reliable yields . This means that a worm of developmental age a hours has probability 0.042 of failing to reach age and probability 0.958 of advancing one hour. More generally, the probability that a worm of age a reaches age A is . The value of an age a worm is proportional to . This simple model makes the most sense for a worm committed to a single developmental pathway. An L2 larva, for instance, has no path to adulthood except by developing through every hour of L2.

These arguments suggest a solution to the puzzle of the L2d, and a method of quantifying the cost. Golden and Riddle [2] found that it takes 3–7 hours longer for a worm to develop from L1 to L3 via the L2d than through the L2 (Figure 1). One plausible reason for this is that the dauer stores fat in order to survive for months without food [9], and the L2d must therefore take the time to eat more. Thus an L2d must run the gauntlet of a dangerous world for up to 7 hours longer than one that takes the L2 pathway. A mutation that eliminated the L2 pathway, forcing the worm to always follow the L2d pathway, would suffer a reduction in value by a factor of between and in a good environment (0.88–0.74, using the estimate above). Such a worm would be at a serious disadvantage to wild-type in good environments and would have no advantage in poor environments.

Binary model

The value of a worm with options, e.g. an L2d which can choose to follow either the reproductive pathway or the dauer pathway, cannot be so simply characterized by a single probability of developmental advance. Its value is derived from the values of the choices it is free to make. A simple example illustrates how this works. Imagine a world in which there are only two possible environments, good or bad (Table 1). In this world, an L2 about to molt into an L3 has no value in a bad environment (it always dies without progeny). The value of a dauer depends less on the environment than that of other stages. For simplicity, the model assumes it is entirely independent of environment. The world can vary in two ways: the value of the new L3 relative to the dauer and the probability of the good environment may change.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Binomial model for L2, L2d, and dauer value

https://doi.org/10.1371/journal.pone.0100580.t001

Table 1A considers the most predictable variation, in which the future is certain: a good environment with probability 1. Since the value of the dauer is independent of environment, it is convenient to express the value of the L2 about to molt in terms of dauers. In this example I suppose it is worth 1 dauer in the good environment. An L2d about to molt will become a dauer when things are bad and an L3 when things are good, so its value is 1 dauer in both the bad and good environments. The worm must choose between L2 and L2d at the time of the L1 molt. If it chooses the L2 pathway, its chance of making it to the L2 molt, which takes 9 h, is . Its value is 1 dauer if it reaches the L2 molt, 0 if it doesn't, for a discounted mean of 0.68. If the L1 chooses the L2d pathway its chance of making it to the next molt is (assuming the maximum possible delay of 7 hours, for a total of 16). There it will be worth 1 dauer whichever choice it makes, for a discounted mean of 0.51. Thus, because of the L2d delay, the L2 is the better choice.

However, value depends on information. Suppose the worm doesn't yet know whether the environment will be good or bad, but only that they occur with equal probability 0.5 (Table 1B). Suppose also that in this world, the good environment is better than in example A, so that the value of the new L3 is 2 dauers. In this world, the question “What is the value of the L2 just before the molt?” has three different answers, depending on what information the worm has. A worm that knows that the environment is good has value 2, and a worm that knows the environment is bad has value 0. The value of a hypothetical worm that hadn't yet found out which environment it was in, however, would be the mean of these weighted by their probabilities: 0.5×0+0.5×2 = 1. Similarly, an L2d will choose dauer (value 1) in the bad environment and L3 (value 2) in the good. Its value, if it hadn't yet found out what kind of environment it was in, would be 0.5×1+0.5×2 = 1.5 dauers. In this toy example, when the environment is uncertain, the option to choose between dauer and L3 is worth half a dauer.

In fact, by the time of the molt an L2 or an L2d will know whether the environment is good, but it can't decide between L2 and L2d then. The real relevance of the informationless values is when the decision is made, at the time of the L1 molt. As before, these values must be discounted by the probability of successfully reaching the next molt. Thus, the L2d choice is worth 0.51×1.5 = 0.76 dauers at the L1 molt, the L2 0.68×1 = 0.68, and L2d is preferred.

Table 1C shows a third variation of this example, in which the good environment occurs with 0.667 probability and the L2 is worth 1.5 dauers at the L2 molt. At the L1 molt both the L2d and the L2 are worth 0.68 dauers, so the worm is indifferent.

These examples were contrived to make a point. In all three cases the value of the L2 choice is the same: 0.68. Also, in all three cases, the value of the dauer is the same: 1 at the L2d molt, or 0.51 at the L1 molt. Yet the optimum choice differs among the three. This shows that to make the best decision, it is not enough to know whether, on the average, the animal will have more future descendants on the reproductive pathway than the dauer pathway. Another piece of information is needed. This piece of information is the uncertainty of the predicted future. High uncertainty favors the worm that has options over a committed worm. Thus in case B, with high uncertainty, the L2d is favored over the L2, while in case A, with no uncertainty, the L2 is favored over the L2d. If the worm knows only the average quality of the future environment (as measured by the relative values of the L3 and dauer), it can't always make the optimum choice. But if it also knows the uncertainty (as measured, for instance, by the standard deviation of the value of the L3), it can always choose optimally.

An example may help to clarify the meanings of value and uncertainty. In nature, C elegans is thought to alternate between brief periods of rapid population growth, which occur when an animal finds a food source such as a dead snail or a rotting apple, and long droughts, which the worm survives as a dauer [10]. An individual that is lucky enough to find a rotting apple early can hope to found an exponentially expanding population—this is facilitated by the ability of a self-fertilizing hermaphrodite to reproduce alone and the explosive growth rate of an unrestrained C elegans population. We don't know how big such a local clonal population can grow, but considering the relative sizes of a worm and an apple, it is not implausible that a rare lucky worm gives rise to thousands or even a million dauers before the food is exhausted. Since, as argued above, the world-wide C elegans population is close to steady-state, it is inevitable that most of these dauers die without reproducing. In fact, if the mean number of dauers produced by a dauer that finds a food patch is N, the probability that one of these dauers similarly finds a food patch is .

Thus, consider two worms. One is an L1 in a rotting apple that has been nearly exhausted. It is teeming with worms and other invertebrates, and the food will be gone in 2–3 days. This worm's best strategy is to grow to adulthood and produce progeny, and it almost certainly can do that before the food runs out. The second worm is an L1 that hatched after the food had run out and proceeded to the L1 starvation diapause. This worm has only a probability of finding an apple before it dies, but if it finds one, it can expect to produce 1,000 fertile adults. Both of these worms have the same value: that of 1 adult. But their uncertainty is very different. Consequently, the L2d option is more valuable to the second worm than to the first.

The binary model is unrealistically simple, but some of its properties are quite general: an option is more valuable in an uncertain environment than a predictable one, and therefore, to make the optimal choice between the L2 and the L2d, the L1 should take into account not only how good or bad the future is likely to be, but its uncertainty about that future.

Continuous models

The binomial model allows for only two decision times, L1 molt and L2/L2d molt, and two environments, good and bad. For more realistic models whose quantitative predictions might be a useful approximation to reality, I looked for inspiration to finance. A dauer is like cash. Its value is relatively stable and independent of the environment. A reproductive larva is analogous to a share of stock. Its value represents the best estimate of future growth prospects. Like a worm, a share of stock can grow exponentially in value in a favorable economic environment, coming eventually to be worth hundreds or thousands of times its original value, or the company can die out and the stock become worthless. The value of stock depends on information and can thus change quickly. Some stocks are relatively stable in value, while others are more volatile. An L2d is like a call option on a stock: it allows but does not obligate the future exchange of a fixed cash price (one dauer) for a share of stock (an L3).

Black and Scholes [11] modeled stock value as a geometric Brownian motion: a random variable whose logarithm is normally distributed, with a variance that increases linearly with time. This model oversimplifies reality and fails in some important cases, but it is a useful approximation much of the time, and is widely used in pricing financial options. In the Black-Scholes model, like the simpler binary model, the value of an option depends on two things, the value of the assets whose exchange it enables (the dauer and the L3, in the case of an L2d), and volatility, which determines how uncertain the current estimate of those future values is.

The Black-Scholes model can be applied to the L2d (see Fixed-time European model in Methods), but it fails to describe the biological problem in two important ways. First, financial options have specific expiration dates. In contrast, the time at which an L2d can choose between reproductive and dauer pathways depends on developmental age, and the rate at which a worm develops is variable, depending on such things as temperature and food supply. The effect of this difference is to increase unpredictability, since the L1 not only doesn't know how the environment will change—it doesn't even know how long it has to make its decision. Second, the Black-Scholes model assumes that an option cannot be used before its expiration. However, an L2d need not wait until the molt to make its decision—it can commit to the reproductive pathway before the molt [2], [12]. The ability to decide early is valuable because it decreases the developmental delay associated with the L2d decision to as little as 2–3 hours, and therefore its cost.

Figure 2 shows how value depends on the quality of the environment in a model that takes these factors into account. The L2d value curve is plotted for four different values of uncertainty: 0 (a completely predictable environment), 0.5, 2, and ∞ (a maximally unpredictable environment). The line representing the value of an L2 is also shown, using a measure of environment quality such that L2 value does not depend on uncertainty. The L2 line crosses the zero uncertainty curve at an environment quality of 0.3 dauers, i.e., where the expected value of an L2 is equivalent to 0.3 dauers. Thus, in a completely predictable environment, the L1 should choose the L2 pathway if the environment is good enough that the L2 will produce progeny worth, on the average, 0.3 dauers. If the environment is worse than that, the L1 should instead choose the L2d pathway. In more uncertain environments the L2d option has more value, and the crossing occurs at higher quality values. Thus, at a modest uncertainty of 0.5, meaning that environment quality will typically change by 0.67–1.5-fold between the L1 molt and the L2d molt, the L1 should require an L2 value of 0.45 dauers to persuade it to take the L2 pathway, and at an uncertainty of 2, the threshold is 0.97 dauers. In a maximally uncertain environment, the L2 must have a value of 2.4 dauers (beyond the right-hand-edge of the graph) for the L2 to be optimal. The threshold environment quality thus varies as much as eight-fold, depending on the level of uncertainty. The phase diagram Figure 3 summarizes the optimal strategy based on both environment quality and uncertainty.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Volatility and option value.

Plotted in black is the value immediately after the L1 molt of an L2d that can choose between L3 and dauer, calculated with the Hybrid model described in Methods. Green and gray curves show the values of committed L2 and dauers at the same time, respectively. (The committed dauer is hypothetical—normal worms do not commit to dauer at the L1 molt.) Environment quality is measured by the value of an L3 larva. Black lines plot value for environments of different volatility, quantified as described in Methods by uncertainty, a number related to the factor by which environment quality will typically change during L2d development. The two heavy black lines show value when the future environment is completely predictable (uncertainty  = 0) or completely unpredictable (uncertainty  = ∞). Thinner black lines plot value against quality for intermediate levels of uncertainty 0.5 and 2.

https://doi.org/10.1371/journal.pone.0100580.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. L2/L2d decision curve.

This phase diagram shows the optimal decision strategy for the L1 choosing between L2 and L2d pathways on the basis of uncertainty and predicted environment quality, as measured by the value of the L2. L2d is favored in high uncertainty and poor environments (the blue region), L2 in low uncertainty or good environments (red).

https://doi.org/10.1371/journal.pone.0100580.g003

Low and high uncertainty extremes

The L2d value curve has a particularly simple form at low uncertainty and at high uncertainty. In a low uncertainty environment the future is completely predictable. The worm already knows at the L1 molt what decision it will make at the L2 molt. Indeed, it knows whether or not it will exercise the option to switch to the L2 pathway during L2d development. There are only two possible future courses. In one, the worm follows the L2d pathway all the way to the molt, then becomes a dauer. Its value, if it follows this pathway, is simply , the value of a new dauer discounted by the cost of 16 h development. In the other, the worm follows the L2d pathway only until the first possible time at which it can switch to the L2 pathway. Following this pathway, it takes 3 h longer than the 9 that would have been required to reach L3 than if it had chosen the L2 pathway from the start, so its value is . ( is the value of an L3 at environment quality q.) A switch from L2d to L2 later than the earliest possible time would cause more delay, and therefore lower value, so will never be optimal. Since the L2d will choose the most valuable of these two futures, its value is just the maximum of the two. That is, it is flat at up to the point at which it intersects , and then it follows the latter's linear increase.

The high uncertainty curve is more surprising and requires more explanation. In an extremely uncertain environment, the value of an L2d is the sum of the values of its two options, the L3 and the dauer, discounted by the developmental delay. How can one worm have the value of two? The answer is shown by the high-uncertainty example given above: an L1 with a chance of value 1,000. If the chance pays off, this worm follows the reproductive pathway, thus capturing all the value of the L2 option. If the environment goes bad, it chooses to become a dauer. Since this occurs 99.9% of the time, the L2d also captures 99.9% of the value of the dauer option. Thus, the L2d's value is the value of the L2 option plus almost all the value of the dauer option.

L2d value at intermediate uncertainty in Figure 2 and the strategy curve in Figure 3 depend on detailed assumptions of the model, which specify the nature of environmental variation and the timing of development in the wild. Strategies based on the binary model or the Black-Scholes model have similar features, but the exact shape is different. Surprisingly, however, the curves for high and low uncertainty are independent of these model assumptions. It is easy to understand why detailed assumptions would not matter in the low uncertainty environment: in this world, the future is completely predictable. But the high uncertainty limit is also independent of detailed assumptions. The reason, it turns out, is that there is basically only one way of having very high uncertainty. The value of a worm can never be negative, so it can never decrease by more than 100%. Since the value can never decrease by more than 100%, all the high volatility has to occur on the upside. But since by definition the average value of the future possibilities must equal the current value of the worm, high uncertainty can only mean that high future values occur with very low probability, and low future values occur with probability close to 1. In other words, in a high-uncertainty environment, almost all the value of a population lies in very rare worms that achieve extraordinary reproductive success. The L2d has nearly the value of both its L2 and its dauer options in any such environment.

Costs of ignoring uncertainty

The previous sections showed that to make the optimal L2/L2d decision, the L1 must take into account both environment quality and uncertainty. But how much difference does it make? How much less fit is a worm that ignores uncertainty? It is impossible to answer this question without knowing what kind of variability worms experience in the wild. However, the high and low uncertainty limits allow calculation of an upper bound.

Using uncertainty in decisions is valuable only if uncertainty varies. The world in which uncertainty matters the most is one in which very high uncertainty and very low uncertainty both occur, and each occurs with high probability. Assume, therefore, that a worm finds itself either in a low uncertainty or a high uncertainty environment with equal probability. In this world, compare the value of two types of worms: a Smart worm that bases its decisions on both the quality and the uncertainty of its environment, and a Dumb worm that bases its decisions solely on quality. The optimal strategies for such worms are shown in Table 2. The optimal strategy when ignoring uncertainty is to use an environment threshold intermediate between the low uncertainty and the high uncertainty thresholds.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Smart and Dumb worm strategies.

https://doi.org/10.1371/journal.pone.0100580.t002

Figure 4A compares the value curves for the Smart and Dumb worms. Even though differences in uncertainty can result in an eight-fold difference in the L2/L2d threshold, the cost of ignoring uncertainty is comparatively small. Figure 4B plots the cost in value, and Figure 4C as a percent of the value. In very poor environments and very good environments, where the optimal decision is independent of uncertainty, there is no cost to ignoring it. The cost is largest in mediocre environments where the L2/L2d decision is more difficult, reaching a maximum of over 5% of value.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Cost of ignoring uncertainty.

A. The value of a worm following either the Smart or the Dumb worm strategy of Table 2. B. The difference in value between Smart and Dumb worm strategies. C. Difference in value as a percent of the Smart strategy.

https://doi.org/10.1371/journal.pone.0100580.g004

In conclusion, taking uncertainty into account when making the L2/L2d decision may increase value by up to 5%.

Evaluation of the model

The main conclusion of this paper is that a worm that takes uncertainty into account will make better decisions than a worm that ignores it. These better decisions may increase the worm's contributions to future generations by as much as 5%. This conclusion is based on a model that, like all biological models, grossly oversimplifies reality. Some defects of the model are real, and some only apparent.

Among the latter is the apparent assumption that the value of a reproductively developing worm depends only on environment quality, and not on uncertainty. Surely a more unpredictable environment is worse, all else equal? In fact, the model is consistent with this intuitive insight. The apparent independence of L2 value and uncertainty is a consequence of choosing a measure of quality that, by definition, includes uncertainty. This can be seen most clearly in the Binomial model example (Table 1). The “predicted environment quality” q is identical in all three cases, as shown by the fact that the mean value for the L2 is the same. But it is the same only because greater uncertainty (in C, for instance) is compensated for by a higher upside value.

An example might help to clarify this. Consider a single person buying a house. She cares about both the price, and the size. However, in the end she can only purchase one house, and her best strategy is to select that house in which she expects to be happiest. If one defines her expected level of future happiness, based on the price and size of the house, as the “predicted house quality”, then she can make her decision solely on the basis of this variable. There is nothing interesting or insightful about this claim—it is a tautology, based on the way “predicted house quality” is defined.

It becomes nontrivial if we also consider another shopper, a man with a family. He also cares about price and size. Shopper 1's “predicted house quality” might be valuable to him in making his decision. But his choice cannot be based solely on the “predicted house quality” defined for her. This family man will care more about size. If he knew both the quality and the size of every house, he could make a better decision. For instance, he might choose the highest quality house that has at least three bedrooms.

The claim that an L1 needs to know both environment quality and uncertainty is something like this, but stronger. The L2 and the L2d are like the two shoppers. Even if we define a number, “predicted environment quality” that combines everything that determines the L2's future, this is not enough to figure out how good the future will be on the L2d pathway. Another piece of information is needed, uncertainty. But there is a closer relationship between the L2 and the L2d than between the two shoppers, because the L2d has the option of becoming the future L2. In the example one might argue that Shopper 1's evaluation is really of little relevance to Shopper 2. But there is no question that the value of an L2 is relevant to the value of an L2d.

Another apparent oversimplification is the attribution of an explicit option to the L2d, but not to the L2, the L3, or the dauer. Obviously all animals have options. And these options improve their chance of survival and therefore increase their value in uncertain environments. The dauer, for instance, can at any time recover, or remain a dauer. Even the L2 can slow down growth and development if food is scarce, and has the possibility of becoming an adult that may lay eggs, entering adult reproductive diapause, or undergoing matricide [13]. The L2d, however, clearly has more options than the L2, because it can become an L2, but it can also do something else. It is for this reason that the L2d is predicted to become more valuable relative to the L2 in a more uncertain environment. In the simplified models used here, the ability of the dauer to recover means that the dauer value curve is not a flat line as shown in Figure 2, but curves up at high values. This has no effect on the L2/L2d decision, since the dauer will only be chosen in a poor environment. The options available along the L2 pathway are not explicitly modeled, but are assumed to contribute to the environment quality, which is defined on the basis of the value of a reproductively developing worm. It would of course be interesting to develop more complicated and realistic models in which some or all these options in the C elegans life cycle are explicitly modeled.

This modeling choice has a consequence for experimental tests. It is not enough to determine whether uncertainty affects the L2/L2d decision. In fact, it is likely that uncertainty affects all developmental and diapause decisions, since all explicitly or implicitly affect future options. The strong prediction of the model is more subtle: The L2/L2d decision should be more sensitive to uncertainty than the L3/dauer decision or the roughly inverse dauer recovery decision. That is, the strategy curve for the L3/dauer decision should lie below and to the right of the curve for the L2/L2d decision (Figure 3).

Hedging

A true defect of all the models considered here is that they assume an individual worm has a reproductive value independent of other worms in the population. (Equivalently, the value of the population is assumed to be the sum of the values of the worms in it.) In such a model, bet hedging never makes sense: every worm should do whatever maximizes its mean contribution to future generations based on the information available to it, and all worms with the same information should behave identically. This assumption is justified if the random variations experienced by the worms in a population are uncorrelated with each other (“demographic stochasticity” [14]). If, however, every worm in the population experiences identical random variation (“environmental stochasticity” [14]), it may be optimal for the genotype if some worms make individually non-optimal decisions. The clearest case is decisions that risk death. A decision that risks the death of an individual may be good for that individual, if it also has substantial upside possibilities. However, in a population subject to pure environmental stochasticity, a decision that risks the extinction of the population is never optimal, no matter how high the upside may be [14]. In reality, of course, stochasticity is neither purely demographic nor purely environmental. The randomness experienced by different members of a population is correlated, but not identical. Probably, uncertainty is more environmental for short times, before a population has time to disperse, and more demographic for longer times.

It is likely that the worm does in fact hedge dauer decisions. The most striking evidence for this is the effect of ascaroside pheromones on dauer formation. In standard assays it has proven impossible, even at very high pheromone concentrations, to force 100% of worms to go dauer [2], [15], [16]. Even at concentrations 1000 times those required to induce >10% dauer formation, some worms do not become dauers [15]. (Interestingly, since it is possible to get 100% reproductive development, this suggests counterintuitively that in nature dauer development is the dangerous choice.)

The models used here assume pure demographic stochasticity, and therefore cannot account for hedging. It is intuitively clear, however, that the main conclusion of this paper, that uncertainty is important in optimal decision-making, holds even if there is environmental stochasticity. Hedging is a response to uncertainty, albeit only uncertainty of a particular type. If there is no uncertainty, there should be no hedging, since the value of the population is reduced by some animals choosing predictably suboptimal strategies. By assuming pure demographic stochasticity, I make the weakest possible assumption. Yet, even in this case, accounting for uncertainty allows better decisions.

How uncertain is the environment?

The inference that a worm will make better decisions if it takes uncertainty into account depends on the assumptions that (1) the environment experienced by a lineage of worms is uncertain, and (2) uncertainty varies a lot from one environment to another. The first assumption is not controversial. For instance, Felix and Braendle [10] refer to worms' “boom-and-bust lifestyle exploiting ephemeral resources”, and state, “Critical life-history choices likely reflect adaptations to the fluctuating and ephemeral natural habitat of C. elegans.” Considering the size of food concentrations in the wild relative to the worm, uncertainties of the level at the right edge of Figure 3, i.e. rare worms experiencing environments in which their lineage can expand 1,000-fold, do not seem unrealistic.

However, there is almost no information on how much uncertainty varies. Certainly, there must be some variation. For instance, viability, fertility, and growth rate are all affected by temperature, and C elegans, a cosmopolitan species, is commonly found in temperate climates where temperatures fluctuate seasonally above and below the optimum. Similarly, rotting organic matter is more reliably found at some seasons and locations than others. However, as shown in Figure 3, the effects of uncertainty will only make a big difference if there are also relatively predictable environments, with uncertainties on the order of 1 or less (i.e., where future value typically varies by less than a factor of 2). Furthermore, uncertainty is unimportant in very good or very poor environments (Figure 3, Figure 4). We have no information on the frequency or even existence of conditions of medium environment quality and varying uncertainty, and the prospect of measuring them in field studies is daunting.

I estimated that the use of uncertainty information might increase the value of a worm by as much as 5%. This is an upper bound, and the typical value benefit will be smaller. Reproductive value is not the same as fitness, but they are related: relative fitness is proportional to the per-generation difference in value. If, for instance, a lineage of worms finds itself in a situation where the use of uncertainty information confers a 5% advantage once every ten generations, the fitness benefit is 0.5%. Compared to known selective effects acting on survival traits, this is a respectable number and might well drive evolution [17]–[19]. (For example, Hoekstra et al. [18] find that reported values of , the relative fitness difference corresponding to one standard deviation of a quantitative trait, are distributed exponentially with a median of 8.8%. From this it can be estimated that 33% of reported effects have and about 4% have . includes the combined selective effect of variation caused by the environment and all loci controlling a trait. Recent results suggest that quantitative traits are often controlled by many genes with small individual effects [20]. This suggests that, at the level of an individual gene, even very small fitness differences can be important.) If, on the other hand, such conditions occur only once in a thousand generations, the fitness benefit is only 0.005%, a tiny effect (though still capable in theory of being important in a large population over long times). It is also true that there are many other options in C elegans development—other decisions whose accuracy may be improved by accounting for environmental uncertainty. Thus the value of sensory and computational engines capable of estimating uncertainty might well go beyond the specific circumstances in which they aid the L2/L2d decision.

Do worms use uncertainty?

These considerations suggest that worms might estimate uncertainty and use these estimates in the L2/L2d decision and others. How might they do this? Clearly, the worms can't directly measure future uncertainty. If they are to estimate uncertainty, they must do so using a proxy, such as past uncertainty or spatial variation, that is correlated with future uncertainty. There are at least four possibilities.

The first is the trivial answer: the estimate may be genetically fixed, and optimized by evolution. This would correspond to drawing a fixed vertical line on Figure 3, and making the L2/L2d decision based only on estimated environment quality. This is the “Dumb worm” strategy of Figure 4. The remaining three strategies all involve some active method of evaluating uncertainty.

The second method was hinted at in Results. The worm may estimate uncertainty based on its correlation with environmental characteristics. For instance, a worm in a rotting apple in which the food is about to be exhausted can foresee its future with considerable precision. A starving worm searching for food in a target-rich environment may have the same value, but it is much less able to predict it. The second worm should therefore favor L2d more than the first.

The third method is the most obvious: a worm can estimate its uncertainty of the future based on the volatility of its past. That is, an animal that has grown up in a varying environment might favor L2d, while a worm that grew up in a constant environment would favor L2. These decisions might depend on the worm's own experience, or (epigenetically) on its mother's.

The fourth method is communication with others. Worms communicate using ascaroside pheromones, which affect the L2/L2d and L3/dauer decisions, as well as other behaviors [2], [21]. We know now that the hypothesis of Golden and Riddle [2] that dauer pheromone is a constitutively secreted population density signal, while correct, is an oversimplification. C elegans and other nematodes release several different ascarosides, the ascarosides released depend on the worm's condition and environment, and they respond differently to different pheromones [21]–[23]. This suggests that pheromones may serve not just to detect crowding, but to pool information about the environment. Suppose, for instance, that a worm that experiences a good environment releases one pheromone, while a worm that experiences a bad environment releases a different pheromone (e.g. C9 ascarosides released by well-fed L1 cultures, compared to C5 released by starved cultures [22]). An L1 would interpret good pheromone as evidence of a good environment and the bad as evidence of a bad environment. A mixture of good and bad would suggest a variable and uncertain environment, and bias the L2/L2d decision towards L2d. It is also possible to combine the third and fourth mechanisms: a worm that experiences a variable environment might release a pheromone that directly signals volatility.

The first of these hypotheses, that the L2/L2d decision is based on a genetically fixed estimate of uncertainty, is essentially untestable, since it is negative. The second, that uncertainty is estimated based on its correlation with environmental characteristics, does make an experimental prediction: that the L2/L2d decision will depend differently on environment than do the L3/dauer and dauer recovery decisions. Unfortunately, this prediction is not specific to the uncertainty hypothesis; one can easily propose alternative explanations.

Hypotheses three and four, however, can be experimentally tested. Hypothesis three implies that a worm that has experienced a volatile past (or whose mother has experienced a volatile past) will be more likely to choose L2d than a worm that has experienced a constant past. This prediction can be tested by manipulating the L1's experience of temperature, food, and pheromone (factors known to influence the dauer decision) between hatching and the L1 molt. For instance, one group of worms might be given a constant, low density of food from hatching, while a second group received, in alternation, no food for time , followed by high density food for time , with and chosen so that, at the L3/dauer decision, the fraction of worms that choose dauer is the same in the two groups. Hypothesis three then predicts that the second group of worms will choose L2d more often than the first group. By choosing different food densities and different alternation regimes, the phase diagram of Figure 3 could be explored.

The clearest evidence for hypothesis four would be identification of the proposed good and bad pheromones. There is however an experimental test short of this, a mixing experiment. It begins by isolating conditioned medium from worms in good conditions (well-fed, optimal temperature, uncrowded) and from worms in bad conditions (starved, high temperature, crowded). The first prediction is that these conditioned media would influence the L2/L2d and L3/dauer decisions: good CM would favor reproductive development, bad L2d or dauer. The second, more important prediction is that a mixture of good and bad CMs would bias the L2/L2d decision more strongly towards L2d than it does the L3/dauer decision towards dauer.

These two tests are not trivial, but seem feasible with current technology.

Reproductive Value

To determine the best way to make the dauer decision, I follow other authors (e.g. Houston and McNamara [14]) in assuming that every worm has a reproductive value, and that the optimal decision is the one that maximizes this value. The concept of reproductive value (henceforth simply “value”) was introduced by Fisher [3], who, considering the effects of age on selection in human populations, defined it as the answer to the question, “To what extent will persons of this age, on the average, contribute to the ancestry of future generations? The question is one of some interest, since the direct action of Natural Selection must be proportional to this contribution.” Since C elegans reproduces primarily by self-fertilization, so that each animal has only one parent, a worm's value may be defined more simply as a number proportional to the average number of its descendants at some distant future time. The focus on the distant future overcomes a weakness of simply counting the worm's children: their contribution to future generations depends not just on their number, but also on their age, condition, and access to resources. If, however, one waits long enough for descendants to disperse and the environment to revert to its mean, these transient factors should average out.

The words “on the average” in Fisher's definition are important. The number of future descendants of a single worm does not approach a finite limit: it is well-known that if you wait long enough in a uniparental population, the lineages of all but one animal will die out, and the descendants of one particular animal will take over the entire population [24]. It is only by averaging over animals that are similar in some way that one can hope to define value. Since Fisher was concerned with age, he averaged over all persons of a given age and sex, and the value he calculated was therefore a function of age and sex.

Here I am concerned with developmental or behavioral decisions. Any decision must be based on the information available to the worm at the time of the decision. Therefore value is based on the average expected number of future descendants of an animal, with the average taken over all animals in the population that have the same information.(1)

(for clan) is the number of descendants of a worm at time T. is a σ-algebra representing the information available to the worm. It includes information about the worm's internal state, e.g. its age and condition, as well as information about the present and past environment gathered by the senses. , a conditional expectation, is thus the expected or average number of descendants at time T of a worm with the information . Definition (1)assumes that the population eventually reaches steady-state, so that the limit as is defined. Non-steady-state populations present no essential difficulty, but a more complicated definition is necessary. For simplicity, this paper deals only with populations at steady-state.

Value so defined is a function of information. Since information can change rapidly, as rapidly as the senses operate, value can change rapidly. In this way the value of an animal is like the value of a financial security such as a share of stock, which is also an estimate of future growth, depends on information, and can change suddenly in response to news.

The information available to a worm changes with time, and is therefore represented by a filtration . Substituting for in (1) gives(2) so defined is a martingale—the best estimate of its future average is its current value.

Binomial model

The analogy between reproductive value and the value of a financial security suggests that models used to value securities might also be applied to worms. In particular, an L2d, which may become either an L3 or a dauer larva, is like a financial option, which confers the right but not the obligation to purchase a stock (the L3) at a predetermined strike price (the value of one dauer) at some future time (the L2d molt). (To be more precise, the L2d is like the combination of a call option on a stock plus cash equivalent to the strike price.) The binomial model [25], a simple illustrative model often used to demonstrate the properties of options, can also be applied to the L2d. Although it is too simple to be quantitatively accurate, it is easy to understand and reproduces the qualitative features of more realistic models. In this model there are only two possible future environments: good and bad. The good environment ensues with probability p, the bad with probability . The value of an L3 is v in the good environment and 0 in the bad. The value of the dauer is 1 in both. The L2d, which can choose to become either a dauer or an L3, has value . If there is some cost to becoming an L2d, its value is instead , where is a discount factor.

Fixed-time European model

Black and Scholes [11] developed a model for valuing financial options that is realistic enough to make useful quantitative predictions. In this model stock price fluctuations are modeled as a geometric Brownian motion, and the option can only be exercised at a fixed time (a European-style option). An analogous model can be developed for the L2d by measuring environment quality by the ratio of the value of a reproductive L3 larva to the value of a dauer. This makes sense, since being relatively insensitive to environment is the whole point of the dauer larva. Following Black and Scholes [11], is modeled as a geometric Brownian motion with volatility , governed by the stochastic differential equation (SDE)(3) is a Brownian motion. (The Black-Scholes SDE usually includes an additional term. r is the rate of population growth, zero in a population at steady-state.)

This model leads to the following formula for L2d value, similar in form to the Black-Scholes formula:(4)where(5)

As in the binomial model, the cost of becoming an L2d may be modeled by multiplying by a discount factor d.

Variable growth rate models

European model

American and Hybrid European/American models

Future prospects

The model for growth and value described here has two obvious flaws. First, environment quality is modeled as a geometric Brownian motion (3). This means it can grow or diminish without bound, and in fact is expected to do one or the other in the long term. The real environment, however, is more stable. If things are very good, they will probably get worse; if things are very bad, they will probably get better. Second, growth and environment quality vary independently—they are governed by uncorrelated Brownian motions and (eqs (3) and (6)). In reality the worms will tend to grow faster in a good environment, slower in a poor environment. It is mathematically straightforward to modify the model so that the two Brownian motions are correlated, but this gives biologically absurd results, because it correlates the environment not with growth rate, but with worm age and condition .

These problems are related, and they can be fixed in the same way, by modeling the environment not as a geometric Brownian motion, but as the exponential of an Ornstein-Uhlenbeck process [27]. The Ornstein-Uhlenbeck process is mean-reverting and over long periods has a normal distribution. In the simplest version of such a model the growth rate of the worm (i.e., the rate of change of ) is directly proportional to environment quality, which is an Ornstein-Uhlenbeck process. This model produces the product form (10) for value, not as an assumption, but as a derived consequence. in this case depends on the difference between current environment quality and its long-term mean value. This model unfortunately is too simple, since it has no room for information about the environment with no immediate effect on growth (for instance, the smell of food, or pheromones that measure population density). It can be improved by modeling environment quality as the product of two factors, one that directly determines growth rate, and one that depends exponentially on a second Ornstein-Uhlenbeck process representing environmental information that doesn't immediately affect growth.

I have not fully developed these models and do not use them in this paper for two reasons. First, the PDEs they produce are more complicated, and I have not yet found an efficient way to solve them. Second and more important, these more complicated models have more free parameters, which I have no good way of estimating. For instance, the two time constants for mean-reversion of the two Ornstein-Uhlenbeck processes are important. Over times that are short compared to the mean-reversion time constant, an Ornstein-Uhlenbeck process looks like a Brownian motion. The Brownian motion models used here are reasonable approximations to the Ornstein-Uhlenbeck process models if mean-reversion times are longer than the duration of the L2d stage.

Smart and Dumb worm strategies

The Smart and Dumb strategies are evaluated in a world where, over long times, volatility is zero 50% of the time and infinite 50% of the time. The Smart worm knows the current volatility and makes the optimal decision based on both volatility and environment quality. Its decisions and value can be understood based on Figure 2. At zero volatility, the L2 curve intersects the L2d curve at , so the L1 chooses L2d if and L2 if . Its value at the L1 molt is(40)For this reduces to , the discounted value of the dauer, and for it is , the discounted value of the L3.

At infinite volatility, the intersection and hence the cutoff occurs at . Its value is(41)In this expression the second term of the first argument is the value of a worm that chooses L2d at the L1 molt, then, at switches to the L2 pathway.

Finally, the value of the smart worm is(42)This is piecewise linear with breakpoints at 0.3 and 2.4.

The Dumb worm doesn't know the current volatility, and must base its decision solely on q. If it chooses L2d and the volatility is 0, its value is given by (40); . If the volatility is infinite,(43)

Thus, the long-term average value of the Dumb L2d is(44)This is piecewise linear with a single breakpoint at 0.34. The breakpoint is different from that for the Smart worm because the Dumb worm, not knowing the volatility, may choose the L2d pathway at zero volatility even when it is not optimal, i.e. at . By the time it reaches it will discover its mistake, and, if , will switch to the L2 pathway, cutting its losses. If, however, , the optimum is now to become a dauer. This cutoff is different from the cutoff the smart worm makes at zero volatility, because the dumb worm has paid part of the price for becoming a dauer. Its remaining cost to become a dauer is now a 4 h delay rather than a 7 h delay, so the dauer option is more attractive.

The value of the L2 is by definition independent of volatility(45)

Finally, the value of the Dumb worm is(46)The Dumb worm cutoff, the intersection of and , is at . The difference between and , plotted in Figure 4B, thus has breakpoints at 0.3, 0.34, 1.2, and 2.4(47)

Parameter estimation

While the qualitative conclusions of this analysis are robust to numerical assumptions, it is necessary to insert specific numerical values for the parameters to estimate quantitative consequences. These parameters are not fixed. Some are known to be subject to genetic control, and the same can be presumed of the rest. To estimate the consequences of uncertainty on value, I attempted to estimate likely values of these parameters in the current C elegans population.

Software and calculations

Calculations were done in Mathematica (Wolfram Research, Inc). All calculations are included in Dataset S1 as Mathematica notebooks.