response errors cause apps to never be updated again
falkben opened this issue · comments
Steam API will sometimes return errors / null data without much explanation.
This is for appids that often don't have data, are demos, or dedicated servers, placeholders, duplicate appids, or who knows what.
E.g.: https://store.steampowered.com/api/appdetails/?appids=788590&l=english
We currently have about 13K appids in this table. Once added into the table, we never try to retrieve data about these appids ever again (this was to skip any bad entries so we could populate our database).
However, we might want to retry these every so often (every few months?) just in case their status ever changes. I suspect some of these appids are placeholders for games that aren't released yet, so we'd want to go back and retry them. And some of them might have just been transient errors.
E.g. appid 1196310
is present in the appid_error
table, but it appears to have metadata: https://store.steampowered.com/api/appdetails/?appids=1196310&l=english
This requires a created_at or updated_at column in this table which we can filter against (e.g. > 2 months ago) before querying for any of these data. See #40 for some issues with simply adding this column w/ pre-existing data.