Metrics Used for Quality Control
Last modified on Mar 05, 2019
Although percent agreement (PA) is the traditional index used to examine precision in age estimations, it can vary widely among species and ages within a species (Beamish and Fournier, 1981; Chang, 1982; Kimura and Lyons, 1991; Campana et al., 1995). Although PA is not a strong estimate of precision, it provides useful information for comparing readers. Other metrics often used to assess precision include coefficient of variation (CV) and average percent error (APE). Both CV and APE are widely used. All measures of precision will be artificially inflated by any bias that exists among readers. In the absence of bias, CV and APE are equally sensitive to differences in precision. Coefficient of variation is considered more rigorous and flexible than APE (Chang, 1982; Campana, 2001), thus it is reported more frequently. Chang (1982) showed that CV exceeds APE by a predictable quantity. Many age studies have reported a CV of <7.6%, corresponding to an APE of 5.5% (Campana, 2001).
Buckmeier (2002) found that many readers produced biased estimates. Any set of age estimates contain error (Beamish and McFarlane, 1983), and if the error is random, the true age distributions of the population can be estimated. Lai et al. (1996) found that biased errors altered estimates of mortality, growth, and production. Campana (2001) noted that quality control could be used to monitor bias and ensure that estimations remained consistent. Known-age reference structures allow readers to verify their age estimates are accurate and unbiased. Acceptable levels of accuracy and bias need to be established, and readers should be periodically tested. Readers failing to meet standards would need to be re-trained prior to independently estimating age. Often, known-age fish are unavailable, and when this occurs, readers cannot verify that they are accurately estimating fish age. Buckmeier (2002) found that the use of multiple readers reduced error and bias in most instances, although there was still a tendency to underestimate the age of older fish. (#SK results here?).
Beamish, R.J. and Fournier, D.A. 1981. A method for comparing the precision of a set of age determinations. Canadian Journal of Fisheries and Aquatic Sciences 38: 982-983.
Beamish, R.J. and McFarlane, G.A. 1983. The forgotten requirement for age validation in fisheries biology. Transactions of the American Fisheries Society 112: 735-743.
Buckmeier, D.L. 2002. Assessment of reader accuracy and recommendations to reduce subjectivity in age estimation. Fisheries 27: 10-14.
Campana, S. 2001. Accuracy, precision and quality control in age determination, including a review of the use and abuse of age validation methods. Journal of Fish Biology 59: 197-242.
Campana, S.E., Annand, M.C., and McMillan, J.I. 1995. Graphical and statistical methods for determining the consistency of age determinations. Transactions of the American Fisheries Society 124: 131-138.
Chang, W.Y. 1982. A statistical method for evaluating the reproducibility of age determination. Canadian Journal of Fisheries and Aquatic Sciences 39: 1208-1210.
Kimura, D.K. and Lyons, J.J. 1991. Between-reader bias and variability in the age-determination process. Fishery Bulletin 89: 53-60.
Lai, H.-L., Gallucci, V.F., Gunderson, D.R., and Donnelly, R.F. 1996. Age determination in fisheries: methods and applications to stock assessment. In: Gallucci, V.F., Saila, S.B., Gustafson, D.J., and Rothschild, B.J. (Eds.), Stock assessment: quantitative methods and applications for small-scale fisheries. CRC Press, New York, pp. 82-178.