Build failed with GitHub ratelimit error on Markdown

[Simon Willison]

https://github.com/simonw/til/actions/runs/7673134644/job/20915103157

Relevant code:

til/build_database.py

Lines 73 to 104 in 5f7591a

 while retries < 3: 
     headers = {} 
     if os.environ.get("MARKDOWN_GITHUB_TOKEN"): 
         headers = { 
             "authorization": "Bearer {}".format( 
                 os.environ["MARKDOWN_GITHUB_TOKEN"] 
             ) 
         } 
     response = httpx.post( 
         "https://api.github.com/markdown", 
         json={ 
             # mode=gfm would expand #13 issue links and suchlike 
             "mode": "markdown", 
             "text": body, 
         }, 
         headers=headers, 
     ) 
     if response.status_code == 200: 
         record["html"] = response.text 
         print("Rendered HTML for {}".format(path)) 
         break 
     elif response.status_code == 401: 
         assert False, "401 Unauthorized error rendering markdown" 
     else: 
         print(response.status_code, response.headers) 
         print("  sleeping 60s") 
         time.sleep(60) 
         retries += 1 
 else: 
     assert False, "Could not render {} - last response was {}".format( 
         path, response.headers 
     ) 

I don't have a MARKDOWN_GITHUB_TOKEN set in GitHub Actions so it's using the default anonymous rate limit which is very low.

[Simon Willison]

I'll try fixing that with:

      env:
        MARKDOWN_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

https://docs.github.com/en/actions/security-guides/automatic-token-authentication

[Simon Willison]

That fixed it.