Hacktoberfest / hacktoberfest-2020

Hacktoberfest - App to manage the annual open-source challenge, used for the 2019 & 2020 seasons.

Home Page:https://hacktoberfest.digitalocean.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Export statistics

jhaff opened this issue · comments

We need to generate csv files with information on all of our users, those users' pull requests, and those pull requests' repositories.

  • Rake task that calls a job that finds all users in our db in batches and enqueues FetchMetadataJobs for each of them
  • FetchMetadataJob: A job that takes a user_id and calls a service object GatherMetadataservice to get all metadata we need
  • GatherMetadataService calls the following three jobs which call service objects
    • ImportUserMetadataJob calls ImportUserMetadataService makes a GitHub api call to gather the user's metadata, then adds it to the db table UserStatistics
    • ImportPRMetadataJob calls ImportPRMetadataService makes a GitHub api call with the user's id to find all prs opened by a user during the month of October, then loops through those prs, adding their metadata to the db table PRStatistics
    • ImporttRepoMetadataJob calls ImportRepoMetadataService either follows the above process to get the prs or tries to retrieve the prs from the PRStatistics table, then makes a Github api call to fetch the metadata for the PR's repository, and adds that to the db table `RepoStatistics

Additional steps

  • Use the GitHub retryable API client on api calls so that in case the user token isn't working, we try backups before the job ultimately dies or succeeds
  • Protect against users who have revoked our github app (and therefore our access to their token) from the list of allowed
  • Create a a rake task to go through each of the 3 statistics tables and put each of their rows into a csv