Export statistics
jhaff opened this issue · comments
Jacob Haff commented
We need to generate csv files with information on all of our users, those users' pull requests, and those pull requests' repositories.
- Rake task that calls a job that finds all users in our db in batches and enqueues
FetchMetadataJob
s for each of them -
FetchMetadataJob
: A job that takes a user_id and calls a service objectGatherMetadataservice
to get all metadata we need -
GatherMetadataService
calls the following three jobs which call service objects-
ImportUserMetadataJob
callsImportUserMetadataService
makes a GitHub api call to gather the user's metadata, then adds it to the db tableUserStatistics
-
ImportPRMetadataJob
callsImportPRMetadataService
makes a GitHub api call with the user's id to find all prs opened by a user during the month of October, then loops through those prs, adding their metadata to the db tablePRStatistics
-
ImporttRepoMetadataJob
callsImportRepoMetadataService
either follows the above process to get the prs or tries to retrieve the prs from thePRStatistics
table, then makes a Github api call to fetch the metadata for the PR's repository, and adds that to the db table `RepoStatistics
-
Additional steps
- Use the GitHub retryable API client on api calls so that in case the user token isn't working, we try backups before the job ultimately dies or succeeds
- Protect against users who have revoked our github app (and therefore our access to their token) from the list of allowed
- Create a a rake task to go through each of the 3 statistics tables and put each of their rows into a csv