Need a population profile adjusted version for excess deaths
jabelar opened this issue · comments
Hi, greatly appreciated your efforts and they were very useful in the first days of the pandemic to understand the full impact. However, I noticed people are now getting confused about excess deaths for countries that have significant population profile differences in the recent few years. For example, Japan has a massive spike (> 40%) more people in age of 70 to 80 than they did during the reference range.
This is now distorting the conversation and prominent influencers are using this to spread misunderstanding. For example, you're reporting excess deaths in Japan at a high rate, but I believe that adjusted for population profile changes that in fact the death rate is lower than expected.
Maybe you're already adjusting for this, but I think you are not?
I believe the right technique for the expected deaths would be to multiply it by the change in population in each age group since the middle of the reference range. For example, if Japan has 40% more people from age 70 to 80 and has 30% more deaths in that age range than in the reference range, then in fact they have a lower than expected death rate.
Maybe I did the math wrong, but I think that in the four years since this project started that most countries have an aging population that is significant enough to need adjustment.
Cheers!
I just realized that you closed a similar issue, because you said you can't get the data for many countries? I think that most countries have a published pyramid, but I understand that maybe the quality isn't always good.
Perhaps you can have a very big disclaimer on your charts? Right now your data is scaring people by showing a spike in excess deaths in countries like Japan when I think their excess death is actually low right now. Maybe you can also do a couple examples for countries with reliable data, so people can understand how the math works?
But a prominent disclaimer would really help that says something like "The excess death rate can be very inaccurate for countries where the demographic age profile has changed significantly since the middle of the reference range. Countries with rapidly aging populations, the excess death rate will be significantly over-reported."
Thanks for raising this issue. I was not aware of the debate around Japan, and of course I don't want to provide misleading analysis.
Can we discuss the Japan case in more detail?
Japan has a massive spike (> 40%) more people in age of 70 to 80 than they did during the reference range.
Where does this claim come from? Are you saying that in 2023 there are 40% more 70+ people than there were in 2020? This seems very unlikely to me. Here is World Bank estimates for the fraction of 65+ people in Japan: https://data.worldbank.org/indicator/SP.POP.65UP.TO.ZS?locations=JP, and it barely changed from 2020 to 2023 (was 30% and is now still 30%). The total population size of Japan even slightly went down during this time btw.
Here is raw Japanese data in WMD:
And here it is summed by year:
I see very a smooth linear increase from 2015 to 2021 (with a small dip in 2020, probably due to lockdown), and then a large jump upwards in 2022 and 2023. This looks very much like excess mortality to me.
For example, you're reporting excess deaths in Japan at a high rate, but I believe that adjusted for population profile changes that in fact the death rate is lower than expected.
Can you point me to some analysis claiming this?
Both our statements are true because the spike is very sharp and you looked at 65+ range and my comment is about 70+ range, -- the spike was already in the 65+ range at the time the excess death reference period was happening, but the spike only entered the 70+ range right when the pandemic started.
You can see the spike on the Wikipedia population profile here: https://upload.wikimedia.org/wikipedia/commons/thumb/9/9c/Japan_Population_Pyramid.svg/700px-Japan_Population_Pyramid.svg.png
Note that that profile is for data at the end of 2021. Now consider the spike at age 72 which is almost double the number of people at age 74. I assume this is because they did not have many babies in the years right after World War 2 but hen had a baby boom.
So hopefully you can see that a very large number of excess births happened in a very small three-year window, causing a big spike in the population.
And that spike is now at age 72 which is when in most modern countries, death rates usually start to climb, so I think that if the rate of death is climbing quickly in Japan right now (compared to five years earlier), that is expected.
I hope my point makes sense.
You are right that there is a clear baby boom population bump peaking at age 70 in 2019, so birth year 1949, plus minus a couple of years. And a WWII dip.
However, these baby boomers started crossing 70 years around 2017. So I don't really understand why you expect a sudden rise of mortality in 2022. Note that in my plot posted above we see steady mortality increase from 2015 onwards. This increase is already modeled in my excess calculations because we do linear extrapolation of the mortality trend.
The "jump" in the number of deaths only happened in 2022. I don't see anything in the population histogram that suggests that such a jump should have happened in 2022.
To really analyze this further, we would need mortality rates for Japan for each age group separately, and not just in 65+, but in narrow age brackets (e.g. 5-year). I don't know if these data are available, and it's definitely not in the WMD.
Just to clarify: of course linear extrapolation is just an assumption here... It's very well possible that the deaths in Japan were expected to curve upwards, so excess in 2022-23 should be smaller than what I compute.
I am just not sure what I can do about it... Maybe the bigger point here is that at some point I should stop providing any estimates at all, as the extrapolation becomes too risky. Not sure if this point is reached already or not yet.
After another 5 years or so, we will be able to do interpolation instead of extrapolation, and retrospectively estimate pandemic excess deaths by measuring the "bump" in yearly deaths.
However, these baby boomers started crossing 70 years around 2017. So I don't really understand why you expect a sudden rise of mortality in 2022.
Because every year after 70 or maybe it is something like 72 or 74, mortality generally rises significantly. In a modern country like Japan, I wouldn't expect hardly anyone to die at age 70 (that would be an early death).
In 2021 there was almost 1M people at age 72. Two years earlier, in 2019 there was only 500k. The boom is so big that it doubled the number of people at age 72 within two years.
Basically there are much more people at older age today than there were just five years ago. So the expected "natural" deaths will be proportionally higher. So the excess death needs to be compared against that.
I am just not sure what I can do about it... Maybe the bigger point here is that at some point I should stop providing any estimates at all, as the extrapolation becomes too risky. Not sure if this point is reached already or not yet.
This linear extropolation was extremely useful immediately after the reference range, but will become increasingly inaccurate. I believe it may be on the threshold where it may not be good to report it this way. I can see a few solutions:
a) At least put a strong disclaimer that "The excess death calculation does not factor in changes in age distribution since the reference period; therefore the excess death will be overreported (even significantly) for countries with rapidly aging populations"
b) If you had the data, I think you could do an adjustment. You would look at the average deaths per age range and then multiply them by the change in demographics at each age range. But I understand that might be a lot of work.
c) Yes, it might make sense to stop reporting this as it appears to be approaching increasing inaccuracy.
I really enjoy the conversation. Thanks for entertaining my feedback!
I just found out that the World In Data site has tried to make an adjusted version. So there is another option to continue your report but advise people about the inaccuracy and refer them to the adjusted version.
Compare the unadjusted version here: https://ourworldindata.org/grapher/excess-mortality-p-scores-average-baseline?country=JPN~USA
With the adjusted version here: https://ourworldindata.org/grapher/excess-mortality-p-scores-projected-baseline?country=JPN~USA
You can see that even the USA is affected. Unadjusted we have 9% excess, but adjusted we have -1% (we're doing better than projected). It is a big difference.
So maybe what you can do is to provide the disclaimer and then recommend people look at the adjusted version, or maybe at least educate on how drastic the difference could be.
I'm sorry, but you have it all wrong. Your second link goes to the OWID analysis that is IDENTICAL to mine! They use linear extrapolation of the 2015--2019 deaths trend. That's exactly what I do. Your first link uses the average of 2015--19 data as the baseline, and that's clearly not a good idea.