Kinto / kinto-fxa

Firefox Accounts authentication support in Kinto

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Don't instantiate OAuthClient on every request

glasserc opened this issue · comments

@autrilla reported that kinto-webextensions is periodically failing to resolve access to FxA. This is due to a "contract table" filling up, which is due to UDP traffic, which seems to be coming from DNS resolution. There's no DNS cache at the OS layer so every[1] incoming request to kinto-webextensions turns into a request to FxA to verify a token. The Linux kernel doesn't offer a DNS cache; most user-oriented distros ship with dnsmasq or systemd-resolved, which do provide a DNS cache, but we're in Kubernetes so we don't have anything.

[1] Not exactly "every" request because there's a cache, but the vast majority of clients only make two requests at a time with a period of at least 15 minutes so the cache doesn't really help us that much.

Empirically, we're generating an incredible quantity of DNS traffic and that seems bad. @autrilla is going to investigate some way of implementing a DNS cache, for example running dnsmasq alongside the service as a cache. Another question is why we need to make so many DNS requests in the first place; when talking to other services (such as e.g. Postgres) we don't generally reconnect so much, but instead keep a single long-lived connection. The PyFxA library seems set up to support this -- it explicitly instantiates a requests Session. This is kind of speculative, but maybe we aren't seeing the benefit of this keepalive-ing because we are instantiating a new OAuthClient on every _verify_token call (so, every request?). If so, we might be able to get some benefit to maintaining OAuthClient for longer -- either instantiating it in the constructor (if that isn't too early), or maybe using a lazy getter (per @leplatrem's suggestion).