Rework the earlier API and launch as a v0.1 effort
gbinal opened this issue · comments
#152 contains the original work for the API
At this point, this is what I'm thinking:
We can add a flag to the command (similar to --publish
) that tells the reporter to write the reports to a postgres db. The db will have one table for the data points described in the data array. Each entry element in the array will be an array in the table. I'm thinking something like this:
id
report_name
: The name of the report the data is associated withagency_name
: The name of the agency the data is associated with. Null if gov-widedate
: The date associated with the datadata
: jsonb column with the data point, minus thedata
prop if presentcreated_at
updated_at
For reports that are not realtime, we can check for a ga:date
dimension. For each report with one, we can merge into the existing data for that report. This means that for each point we look up whether a record already exists. If it does, we update it. If it doesn't we insert it. I'm a bit concerned about performance here b/c some of the reports are long, but we can try it and see what happens.
Realtime reports are easier b/c we do not need to check for existing records. We can simply set the date
to the current date and insert.
We can build an API app that ties into the same database. I'm a fan of allowing that app to manage the db schema b/c a web app framework will likely have better tooling for managing schema.
This is done.