Weekend reporting effect

[Peter Mancini]

There is a structural issue in the data because weekend cases are often reported on Monday. You can see the structure in the data. You might get more accuracy if you take it into account.

[KS]

Do you have data or a notebook you can provide that shows this effect and perhaps suggest a way to correct for it?

[Scott Bartell]

I've noticed this too--the new case counts routinely appear to drop on Sundays. Your smoothing should already be limiting any weekend effect to some extent, but down-weighting weekend counts in the smooth might be reasonable here.

[Peter Mancini]

Treat Saturday through Monday as one total and divide by 3. Not perfect but is spreads the issue out. If you do totals by days you will see a persistent weekend dip. This may not be true of every state, but Tennessee where I live, and Virginia have it in their data for sure.

[KS]

I'm in the middle of changing data sources right now to covidtracking.com - separately, the DOW effect should be mitigated because of a centered smoothing function (though it's not perfect). I'll have a look at this in the next rev

[Ben Scherrey]

Covidtracking.com is the data source I've been using. Rates of testing/cases doesn't have a dramatic weekend effect but reported deaths absolutely does. New York & NJ are really driving the curve nationally and they seem to be the main culprits. California's reporting is crazy land in terms of "consistency".

[Peter Mancini]

In terms of accuracy it just adds a little bit more uncertainty. Again I think that the Saturday Sunday Monday smoothing is probably a good idea but not necessary. The data is never going to be perfect. Sometimes you just simply have to accept it. Another way of looking at it though is to look at the entropy. The virus doesn't know our calendar system, one would hope! Therefore any scheme to maximize entropy would probably produce a good result. Basically you would shift counts from one day that had too many to other days that had too few. This type of smoothing would not affect the counts and correct for the fact that staff are being affected by the disease at hospitals and are unable to be consistent in their reporting. They're eventually catching up to the reporting as we can see, but they are not able to be consistent all the time. Sent from Outlook Mobile<https://aka.ms/blhgte>

…

________________________________ From: Ben Scherrey <[e-mail]> Sent: Sunday, April 19, 2020 4:00:36 AM To: k-sys/covid-19 <[e-mail]> Cc: Peter Mancini <[e-mail]>; Author <[e-mail]> Subject: Re: [k-sys/covid-19] Weekend reporting effect (#9) Covidtracking.com is the data source I've been using. Rates of testing/cases doesn't have a dramatic weekend effect but reported deaths absolutely does. New York & NJ are really driving the curve nationally and they seem to be the main culprits. California's reporting is crazy land in terms of "consistency". — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#9 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAET3474SMPXCD4OIOELSKDRNK4TJANCNFSM4MIYLR6Q>.

[kmedved]

Smoothing should mostly account for the day-of-the week effects. If you want to get fancy, you could pass the data through something like fbprophet which captures weekly 'seasonality' effects, and then back out of those effects.

I'm not sure it's worth the effort however - some basic smoothing should mostly handle this.