Reader Comments
Post a new comment on this article
Post Your Discussion Comment
Please follow our guidelines for comments and review our competing interests policy. Comments that do not conform to our guidelines will be promptly removed and the user account disabled. The following must be avoided:
- Remarks that could be interpreted as allegations of misconduct
- Unsupported assertions or statements
- Inflammatory or insulting language
Thank You!
Thank you for taking the time to flag this posting; we review flagged postings on a regular basis.
closePrecision of survival estimates is overstated
Posted by WardTesta on 01 May 2012 at 20:14 GMT
This difference explains the comparable uncertainties for these two distinct methods with sample sizes that differ by almost one order of magnitude.
http://plosone.org/article/info:doi/10.1371/journal.pone.0030173#article1.body1.sec3.sec1.p2
Horning and Mellish claim “comparable uncertainties” for their survival estimates and those from brand resighing data with an order of magnitude larger sample size. That claim is false, and the confidence intervals reported by Horning and Mellish appear to be substantially underestimated. Horning and Mellish used the Mayfield (1971) survival estimator that produces a daily survival probability which is extrapolated to annual (or longer) survival rates by raising the estimate and it’s confidence limits to the exponent of period length (Johnson 1979). Bias can result if the deaths are not distributed evenly over the interval of estimation, but I was more concerned with the variance. The variance derived by Johnson (1979), based only on exposure-days and number of deaths, is exactly equivalent to that for samples drawn independently from a binomial distribution, with exposure-days (de) as the sample size. This is inappropriate because the series from a single animal does not consist of independent observations of survival/mortality; it can contain, at most, only one death. Intuitively, it is easy to see that one’s confidence in an annual survival estimate based on identical numbers of deaths and exposure-days from 100 individuals over the course of a year would be much higher than the outcome of studying 5 animals for 20 years that might give identical input data. The proper sampling unit for these data is the individual. It’s unclear why the authors chose the Mayfield/Johnson approach, which was developed to allow for uncertain intervals between observations while their data has no such uncertainties. To demonstrate the likely bias in variance estimates, I constructed a dummy dataset that approximated Horning and Mellish’s account of animals at risk, exposure-days, and timing of deaths. I compared the Mayfield/Johnson method used by Horning and Mellish to the same Mayfield estimator using bootstrap resampling of individual sea lions, and to a simple Kaplan-Meier estimator with staggered entry/exit (Pollock et al. 1989. J. Wildlife Manage 53:7-15) that is appropriate for telemetry data sets without covariates. Applied to the dummy dataset, the estimator cited by Horning and Mellish produced confidence intervals for annual estimates that were similar in the annual estimates but generally narrower for longer intervals than those produced by either the bootstrapped Mayfield or Kaplan-Meier methods, and skewed upward by the use of an exponent to correct for period length (see Table). In contrast, confidence intervals reported by Horning and Mellish were much narrower than those I estimated, either by the methods they described or the two alternatives I used (Table 1). That I could not replicate their results is surprising, given that the estimator and its variance requires only the exposure days and number of deaths, which are given in the paper. If they calculated their multi-year estimates from the product of annual estimates, my own estimate of the variance for that approach was much closer to the alternatives in my table than those in their table, though without the upward skew. In any event, more appropriate confidence intervals for the 13-36 and 13-60 month survival intervals were ~80% and 200% greater than those reported by Horning and Mellish, suggesting their survival estimates have rather limited utility as corroboration of more precise survival estimates from ongoing mark-recapture studies.
Table. Comparison of Steller sea lion survival rates reported by Horning and Mellish to estimates of a simulated data set of their telemetry results with the same Mayfield/Johnson method, a bootstrap analysis of the Mayfield estimator with resampling at the level of individuals, and an ordinary Kaplan-Meier estimate with staggered entry/exit (Pollock et al. 1989).
Ages (mo): 13-24 25-36 49-60 13-36 13-60
Horning & Mellish 0.64 (none) 0.83 (none) 0.92 (none) 0.53 (0.40-0.63) 0.49 (0.40-0.54)
Mayfield (approx.) 0.64 (0.46-0.90) 0.83 (0.69-1.00) 0.92 (0.79-1.00) 0.55 (0.39-0.79) 0.48 (0.32-0.73)
Bootstrap Mayfield 0.64 (0.43-0.84) 0.83 (0.67-0.96) 0.93 (0.74-1.00) 0.55 (0.34-0.77) 0.49 (0.27-0.71)
Kaplan-Meier 0.64 (0.41-0.87) 0.82 (0.67-0.98) 0.92 (0.76-1.00) 0.53 (0.31-0.74) 0.48 (0.27-0.70)
RE: Precision of survival estimates is overstated
TheTechnologist replied to WardTesta on 01 May 2012 at 23:02 GMT
In a first response to one of several comments made by Ward Testa on this publication, here is a clarification on how we calculated the precision estimates, and why they are correct and more accurate than the estimates provided by Testa:
As Testa correctly points out, the Mayfield method when applied to longer time frames is sensitive to changing survival over the interval of estimation. This is clearly the case in juvenile Steller sea lions (see annual rates for sequential age classes). Thus, the cumulative survival rate (CSR) calculated by simply raising a single mean daily survival rate (DSR) derived from all age-years to the power of period duration (as per Testa’s table) does give incorrect results in terms of both the CSR and its CIs. This is evident from the fact that such a CSR is not the product of sequential ASRs. This barely shows in Testa’s results table with 2 decimals, it is more evident in further decimals, or in a smaller data set or simulation. There are several ways to deal with this, but one very simple and effective way is to first calculate separate DSRs for each age-class, from which one can derive a product SR for multiple days pooled from successive years (one from each year). In the example of 3 successive years, one would get a 3-day pooled SR. From this 3DaySR one may calculate a 3 year CSR by raising to the power of 365 (and not to the power of 3*365 as per Testa). The same applies to variance and SE from which the confidence intervals are derived. This is how we obtained the CIs listed in the paper. In our case, the CSR is exactly the product of sequential ASRs. The CIs are indeed lower than when using a single mean, multi-year DSR. The CIs reported by Testa would be more comparable to the predictive uncertainty when attempting to extrapolate to a 3 year CSR from a single ASR (as derived from a dataset covering only a single age-year) under the assumption of a uniform distribution in survival. Thus, our CIs accurately reflect the precision of a 3 year CSR estimate based on observations over 3 age-years, under a scenario of increasing survival from year to year.
More details on the choice and appropriateness of the Mayfield method to follow.
We will shortly also post updated survival rate estimates as well as confidence intervals incorporating recent returns.
RE: RE: Precision of survival estimates is overstated
WardTesta replied to TheTechnologist on 03 May 2012 at 00:13 GMT
I've tried to replicate the approach described by B041 to the multiyear CSR estimator, and I get the same point estimate and SE that I got when estimating the product of the 4 years of annual estimates (and variance, Goodman LA, 1962, The variance of the product of k random variables, J. Am. Stat. Assoc. 57:54-60) (table below). They produce identical estimates and SE’s because they both reach the CSR by multiplying the same daily SR’s, just in a different order. I’ve now tried 4 methods which produce similar results, and I can’t produce confidence intervals on CSR’s approaching those reported for LHX juveniles in their Table 1. Other indications that something is wrong is that the confidence interval in Table 1 for years 1-3 is less than that for 1-2, even though the point estimate hasn't changed and no information on survival variability was added in year 3. A Kaplan-Meier approach would have identical results for the 2 CSR's, as does the method of multiplying ASR’s (table below). Horning and Mellish did not report them, but confidence intervals on the annual rates obtained by the Mayfield method (table below) are also much larger than the multi-year CSR’s reported in Table 1. There is an underlying binomial process that dictates the limits in precision to any survival estimator, and the authors have no sampling process to account for because they've assumed that the fate of all animals is known with certainty every day. There is no substantial gain to be had over simple binomial estimators in a known-fate experiment like this without covariates. Horning and Mellish’s method somehow produces 2-3 times the precision of several more conventional approaches without covariates. The authors need to fully describe their method so others can determine its validity.
ASR ASR ASR ASR CSR CSR CSR
months: 13-24 25-36 37-48 49-60 13-36 13-48 13-60
survival 0.641 0.828 1.000 0.923 0.531 0.531 0.490
SE 0.108 0.078 0.000 0.074 0.103 0.103 0.095
CI 0.43-0.85 0.68-0.98 1-1 0.78-1.00 0.33-0.72 0.33-0.72 0.30-0.68