Vitamins Health: The mystery of the vanishing nonagenarians

The 2011 census revealed 15% fewer men in their nineties in England and Wales than had been projected by rolling forward those aged 80+ from the 2001 census. The figures for women were also reduced, but to a lesser degree.

Why does it matter?

These population estimates are relied on by decision-makers, for example in planning for elderly care needs, in reserving for future pension costs and now in the growing world of longevity risk trading. The estimation error is another example of the 21st century phenomenon of model risk.

As Richard Willets recently highlighted the ‘missing’ old people have a material effect on the calculation of mortality rates for the oldest people in our society. Mortality improvement rates are even more sensitive, with the annual rate of improvement (the fall in mortality rates at particular ages) now half of its previously believed level at these oldest ages (at around 1% a year, rather than the previous 2-2½%).

The revised down figures for the over 90s are more consistent with the lower rates of improvement that Club Vita has been tracking in its community of occupational pension funds.

Don’t we count how many people are in the UK?

It may come as a surprise, but the UK Government is only able to estimate the size of the population. Because we aren’t able to count the number of residents (remember the ID card furore), Government statisticians can only ‘model’ the figures.

The Office for National Statistics (ONS) has always presented its population numbers – which are derived from its ten yearly censuses - as estimates. The ONS also emphasises that its counts of older people are less reliable and has not made its figures for individual ages above 90 publicly available. Between censuses, annual population estimates are created by rolling forward the estimated population from the last census. The estimation error at the time of the census is likely to grow over the following decade. These warnings may have gone unheeded by some downstream users.

Is this a one-off correction or could it be repeated?

To answer that, let’s understand the origins of the data or, more accurately, the population estimates.

Importantly, unlike the records held by pension funds and insurance companies, information on alive people and deaths come from separate sources. ONS population estimates come from the census and are rolled forward using actual deaths notified to local registry offices and migration estimates. Because the population is estimated, so calculations of death rates are also estimates. Rates of change in mortality rates (improvement rates) are even more sensitive to the estimation error in the underlying population size.

So, back to the mystery. Where could all these elderly men have gone?

Did more men die in their 80s than expected?

No. The population estimates use actual deaths and do not require assumptions about future deaths. The numbers for deaths registered in the UK are believed to be “hard” (ie narrower error bars). It’s unlikely that deaths go unreported (next of kin can’t bury or cremate a loved one without a death certificate). But it is not essential to present a birth certificate, so there may be some misstatement of dates of birth, meaning that the deaths get deducted from the rolled forward population at the wrong ages.

Did they move elsewhere? Possible, but again probably unlikely.

The estimates relate to the English and Welsh population. The reduction in over 90s would be consistent with the men moving abroad in their 80s. For this purpose, Scotland and Northern Ireland counts as abroad. But mass migration at such advanced ages seems unlikely.

Was there some double counting in 2001?

This is a possible cause, as older people are more likely to be in hospital or a care home, rather than at their home address. The ONS has several controls to minimise the potential for double counting.

Did some people exaggerate their ages?

Possible but probably unlikely – both the 2001 and 2011 census forms required dates of births to be entered, not just ages. It takes more thought to exaggerate dates of birth. But the ONS does not appear to validate dates of birth against a third-party source, so it’s not possible to be certain.

Could dates of birth be mis-transcribed?

This is a possibility, with electronic scanning not being perfect. The ONS says that the character recognition software can confuse 0s, 6s and 8s. (The technology seems to have improved between the 2001 and 2011 censuses.) Even if the mis-scanning is randomly wrong, the errors don’t necessarily cancel out, as the dates of birth of the population are not uniformly distributed. An error ascribing people born in 1969 to 1909 has a bigger proportionate impact than people born in 1909 being put into 1969, because there are far fewer survivors left from 1909. This would suggest that the population estimates for those aged 92 to 101 in the 2001 census (born between 1900 and 1909) were overstated. But it does not explain the missing men in their 80s in the 2001 census.

Is a small error in the 2001 census enough to explain the 15% fall in men aged 90+?

Yes. The removal of 20,000 people – or 15% - from the estimate for men aged 90+ is only equivalent to a 2.5% change in the 2001 estimate of the population aged 80+. It would seem unlikely that the ONS’s error bars would be as narrow as +/- 2.5% for people in their 80s (particularly as they would be likely to be wider at older ages). The potential for +/-15% change in the size of the population aged 90+ was always there, and probably ought not to be regarded as an extreme event. A 100% change in improvement rates at oldest old ages could easily be repeated given the frailty of the population ‘data’.

So, who dunnit?

Although the case remains open, it seems likely that there is more than one culprit contributing towards the estimation error. The practices of not validating dates of birth, or indeed the very existence of the person and collecting deaths from a different source all contribute sources of uncertainty.

But it would require a particularly persistent statistical sleuth to get to the bottom of it. It is questionable whether that would be a great investment in time, since it will not be possible to eradicate estimation error without a major re-engineering of the origins of the data or a bigger investment in independent verification through linkage to other datasets.

The data capture system all sounds very Heath Robinson in today’s electronic information age, where virtually instantaneous identity and credit checks are performed for small purchases.

In any case, the 2011 census seems likely to be the last. Further information on the ONS’s Beyond 2011 consultation is available here. Sadly, a robust method of counting is not on the agenda. Several of the options on the table involve much smaller sample sizes, with the likelihood that estimation error will rise.

And the moral of this tale?

It is a salutary reminder of the importance of understanding the data that you are using, and the materiality of any estimation error inherent in the data. If you are looking for reliable data on improvement rates in oldest people, think twice before using ONS population estimates.