bug: rate limit exceeded when a query returns many repos
caleb15 opened this issue · comments
➜ a git:(master) ✗ mp init "org:15five"
2022/03/26 20:43:46 GET https://api.github.com/search/code?page=5&q=org%3A15five: 403 You have exceeded a secondary rate limit. Please wait a few minutes before you try again. []
When iterating through the results the program should sleep to avoid hitting a rate limit.
Workaround: using a more specific search. Although if a very large organization tried microplane they could run into the issue even with a specific search.
Thanks for reporting!
Agreed, there should be better backoff/retry logic here.
If it's possible that a search can take several minutes, we may need to explore other UX.
Turbolift solves this problem elegantly by offloading search to another specialized CLI and being composable with that. It think microplane would do well to use this approach, too. https://github.com/Skyscanner/turbolift#identifying-the-repos-to-operate-upon