`POST /api/grants` and `POST /api/tokens` results in high latency
jmorganca opened this issue · comments
When making post requests to /api/grants, the requests can take anywhere from 3-5s
As far as I can tell, there are no POST calls taking more than 1s. When I filter on POST (to block out the long-polling GET requests that are supposed to take 5s), there are no log results matching this criteria. Have a link to a relevant log entry?
Some improvements have been made to the UI which should mitigate this issue. The UI, when creating grants for multiple namespaces, sends a request for each namespace. This can lead to a flood of requests which ironically if is received by the server too slowly will cause a backlog of requests and each request is blocked by the preceding request.
The improvement is to send a bulk create request so no matter how many namespaces are selected, a single request will be sent.
A second minor improvement focuses on deleting multiple namespace grants. Similar to the earlier improvemnt, this action sends a DELETE request for each selected namespace. Additionally, each DELETE request is followed by a GET to update the UI state. Therefore, the number of requests per selected namespace is actually not 1 but 2 with all of the side effects mentioned earlier. This has been updated to a bulk delete request with a single GET once the bulk delete request has been handled.
To add some more details on what might taking a long time in a request for POST /api/tokens, here's a breakdown for a request that took ~16s:
This database query took ~3s:
SELECT count(*) FROM identities_groups WHERE group_id = $1
This database query took: ~13s:
UPDATE access_keys SET created_at = $1, deleted_at = $2, expires_at = $3, inactivity_extension = $4, inactivity_timeout = $5, id = $6, issued_for = $7, key_id = $8, name = $9, organization_id = $10, provider_id = $11, scopes = $12, secret_checksum = $13, updated_at = $14 WHERE deleted_at is null AND id = $15 AND organization_id = $16;