Last modified on Mar 05, 2019
To maximize precision, it is important to use more than one reader to be able to calculate agreement measures, such as:
- percent agreement (PA),
- average percent error (APE; Beamish and Fournier 1981),
- and coefficient of variation (CV; Chang 1982).
When using these agreement levels, a lab needs to set agreement levels and test for them.
- For example, most sites re-read ~20% of each sample and maintain >80-90% percent agreement or a CV of 6-10.
Training is key, but it is important to periodically test for bias.
- Don't just test once a year, because bias can creep in during a season as readers see that year's patterns and possibly alter their criteria.
Although testing for bias requires known age samples and these are not always possible to find, samples read by multiple readers (consensus-aged) can replace known age samples.
- When testing for bias, include some known-age or consensus-aged fish in the sample to be read. This will permit determination of bias.
- Include known-age or consensus-aged fish from other years (not just prior year) to determine whether criteria are the same over time.
Bias can be examined using age-bias plots and/or agreement matrices (see examples).
- If bias is found, re-training is necessary.
Overall, most authors note a tendency to underestimate the age of older fish, but this study found issues with the age of younger fish being overestimated.
- Thus, we recommend that new readers and returning readers study ages from young fish (ocean age 1 and 2 fish) as well as older fish (ocean age 5+) and not just the common ages.
- An estimate of bias would be best to be under 10%.
To determine accuracy, it is key to standardize procedures and equipment and set acceptable levels of accuracy.
- Reference structures (known- or consensus-aged) are necessary to this process, but they can be shared via digital images.
- Readers must be trained and periodically tested.
- Tests for accuracy can be combined with tests for precision and bias.
- When testing for accuracy, a reader's age estimations would be compared against the consensus or known-ages.
- When testing for precision, one reader's age estimations would be compared with the second reader’s age estimations.
Although many readers incorporated knowledge of a fish's length, date sample was collected and other ancillary biological data in the past, the current study demonstrated that knowledge of length during the age estimation process contributed to variability among estimates.
- Thus, length can be misleading, especially if length-at-age is changing over time.
- Length could be used during the final comparison, such as plotting age estimations versus length and examining the outliers.
We encourage biologists/readers to remember that this is a check and base age on the pattern not the length.