Query for the data

[Chaitanya Prakash Bapat]

What's the query being used for retrieving the data about the PyPi packages?

[Hugo van Kemenade]

Data is queried using the https://github.com/ofek/pypinfo/ tool, with these commands:

top-pypi-packages/generate.sh

Lines 6 to 8 in 44e2c77

 # Generate the files 
 /usr/local/bin/pypinfo --json --indent 0 --limit 4000 --days  30 "" project > top-pypi-packages-30-days.json 
 /usr/local/bin/pypinfo --json --indent 0 --limit 4000 --days 365 "" project > top-pypi-packages-365-days.json 

Running the same commands with the --test option to prints the queries:

$ pypinfo --test --json --indent 0 --limit 4000 --days  30 "" project
SELECT
  file.project as project,
  COUNT(*) as download_count,
FROM
  TABLE_DATE_RANGE(
    [the-psf:pypi.downloads],
    DATE_ADD(CURRENT_TIMESTAMP(), -31, "day"),
    DATE_ADD(CURRENT_TIMESTAMP(), -1, "day")
  )
WHERE
  details.installer.name = "pip"
GROUP BY
  project,
ORDER BY
  download_count DESC
LIMIT 4000
$ pypinfo --test --json --indent 0 --limit 4000 --days  365 "" project
SELECT
  file.project as project,
  COUNT(*) as download_count,
FROM
  TABLE_DATE_RANGE(
    [the-psf:pypi.downloads],
    DATE_ADD(CURRENT_TIMESTAMP(), -366, "day"),
    DATE_ADD(CURRENT_TIMESTAMP(), -1, "day")
  )
WHERE
  details.installer.name = "pip"
GROUP BY
  project,
ORDER BY
  download_count DESC
LIMIT 4000

[Chaitanya Prakash Bapat]

Thanks!