falkben / steam-to-sqlite

Public Steam app and achievement data in a sqlite database

Home Page:https://steam-to-sqlite.fly.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

response errors cause apps to never be updated again

falkben opened this issue · comments

Steam API will sometimes return errors / null data without much explanation.

This is for appids that often don't have data, are demos, or dedicated servers, placeholders, duplicate appids, or who knows what.

E.g.: https://store.steampowered.com/api/appdetails/?appids=788590&l=english

We currently have about 13K appids in this table. Once added into the table, we never try to retrieve data about these appids ever again (this was to skip any bad entries so we could populate our database).

However, we might want to retry these every so often (every few months?) just in case their status ever changes. I suspect some of these appids are placeholders for games that aren't released yet, so we'd want to go back and retry them. And some of them might have just been transient errors.

E.g. appid 1196310 is present in the appid_error table, but it appears to have metadata: https://store.steampowered.com/api/appdetails/?appids=1196310&l=english

This requires a created_at or updated_at column in this table which we can filter against (e.g. > 2 months ago) before querying for any of these data. See #40 for some issues with simply adding this column w/ pre-existing data.