[self-hosted] 500 error on status_event

[Riley Mulford]

Hey there, looking forward to utilizing this app! We're trying to get it set up on our own architecture but are running into this error on status events:

INFO uvicorn:httptools_impl.py:436 (REDACTED, REDACTED) - "POST /api/github/hook HTTP/1.1" 500
ERROR uvicorn:httptools_impl.py:371 Exception in ASGI application
Traceback (most recent call last):
  File "/var/app/.venv/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 368, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/var/app/.venv/lib/python3.7/site-packages/fastapi/applications.py", line 140, in __call__
    await super().__call__(scope, receive, send)
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/applications.py", line 134, in __call__
    await self.error_middleware(scope, receive, send)
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 178, in __call__
    raise exc from None
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 156, in __call__
    await self.app(scope, receive, _send)
  File "/var/app/.venv/lib/python3.7/site-packages/sentry_sdk/integrations/asgi.py", line 51, in _run_app
    raise exc from None
  File "/var/app/.venv/lib/python3.7/site-packages/sentry_sdk/integrations/asgi.py", line 48, in _run_app
    await callback()
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/exceptions.py", line 73, in __call__
    raise exc from None
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/exceptions.py", line 62, in __call__
    await self.app(scope, receive, sender)
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/routing.py", line 590, in __call__
    await route(scope, receive, send)
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/routing.py", line 208, in __call__
    await self.app(scope, receive, send)
  File "/var/app/.venv/lib/python3.7/site-packages/starlette/routing.py", line 41, in app
    response = await func(request)
  File "/var/app/.venv/lib/python3.7/site-packages/fastapi/routing.py", line 133, in app
    raw_response = await dependant.call(**values)
  File "./kodiak/main.py", line 99, in webhook_event
    await handle_webhook_event(event_name=github_event, payload=event)
  File "./kodiak/event_handlers.py", line 227, in handle_webhook_event
    await push(PushEvent.parse_obj(payload))
  File "./kodiak/event_handlers.py", line 166, in push
    prs = await api_client.get_open_pull_requests(base=branch_name)
  File "./kodiak/queries.py", line 985, in get_open_pull_requests
    headers = await get_headers(installation_id=self.installation_id)
  File "./kodiak/queries.py", line 1190, in get_headers
    token = await get_token_for_install(installation_id=installation_id)
  File "./kodiak/queries.py", line 1183, in get_token_for_install
    assert res.status_code < 300
AssertionError

Events such as requested and synchronize seem to be working (200 response) but any status check events are throwing this error. I'm looking for information on understanding this error and seeing if we can get it resolved. Please let me know if anything else is required from me to help debug this issue.

[Christopher Dignam]

@rdmulford I'd double check that you have all the GitHub App permissions correctly configured: https://kodiakhq.com/docs/permissions#github-app-permissions

The error you've found hard to work from alone. If you setup https://sentry.io via the SENTRY_DSN environment variable, you'll probably be able to get more information about the errors.

If that issue persists you could try reproducing with the GitHub Marketplace Kodiak.

[Riley Mulford]

@chdsbd Hi, thanks for the quick response! After digging in a little more I actually think I know what the issue is. Im trying to run this on our enterprise instance of github, so when kodiak makes an api call to api.github.com (for example here -

kodiak/bot/kodiak/queries.py

Line 1177 in c8de897

 f"https://api.github.com/app/installations/{installation_id}/access_tokens", 

) it fails since its not hitting our internal github instance (git..com).

So I think in order to make this work id need the github host to be a configurable value. We also have an authentication token header that needs to be there to auth requests to our internal git api. Any idea how much it would take to implement that as a feature? I'd definitely be willing to help contribute if we could find a way to make it work

[Christopher Dignam]

@rdmulford This is the config file the bot uses: https://github.com/chdsbd/kodiak/blob/2fdd161bfd955de8218dd79804e28f9fff59d4e0/bot/kodiak/app_config.py

I think we'd need to add a couple new options to app_config.py to meet your use case:

# change the API root for self-hosted GitHub instances
GITHUB_API_ROOT = config("GITHUB_API_ROOT", default="https://api.github.com")

# an extra header to send with API requests.
# Example: `GITHUB_API_HEADER_NAME: GITHUB_API_HEADER_VALUE`
GITHUB_API_HEADER_NAME = config("GITHUB_API_HEADER_NAME", default=None)
GITHUB_API_HEADER_VALUE = config("GITHUB_API_HEADER_VALUE", default=None)

We'd need to update our API calls to use the new api root config in bot/kodiak/queries.py, bot/kodiak/cli.py, and bot/kodiak/refresh_pull_requests.py.

For the auth header, this is probably where the setting would need to be added:

kodiak/bot/kodiak/queries.py

Lines 743 to 746 in c30e68b

 self.session.headers[ 
     "Accept" 
 ] = "application/vnd.github.antiope-preview+json,application/vnd.github.merge-info-preview+json" 
 self.log = logger.bind( 

[Riley Mulford]

@chdsbd Awesome thanks for the breakdown! It doesn't seem to be too big of a change, I'll see if I can implement it and verify that it fixes the problem and adds support for github enterprise.