[quirrel.dev service] Occasional 400 from api.quirrel.dev
jakobo opened this issue · comments
Bug Report
Current Behavior
Occasionally, a queue fails to post a handler success response back to api.quirrel.dev. The lack of a response causes the Quirrel service to think the call failed.
Expected behavior/code
The communication with api.quirrel.dev has built in retries, or at least on failure throws a useful error since the 400 shows up in quirrel.dev as an empty string ""
Environment
| package | version |
|---|---|
| Quirrel | 1.7.4 |
| next.js | 12.0.7 |
| node | lts (16) |
| yarn | 3.0.2 |
- Quirrel version: [e.g. v1.0.3]
- Node/npm version: [e.g. Node 14/npm 6]
Possible Solution
As a workaround, I can pass my own fetch implementation into the route, wrapping with fetch-retry or an equivalent library. However, it feels like some amount of basic retrying should be baked into the client library itself instead of throwing on a 400.
I prototyped a few API alternatives to get fetch exposed at the point of Queue creation. I think the most reasonable place to put this if we don't want to change the way the Quirrel client works is to expose fetch in EnqueueJobOptions ref which defaults to (options) => fetch(options), allowing for a custom fetching implementation.
Or we can add retries to makeRequest, enqueue, enqueueMany, etc. In that case, I would strongly recommend using something like fetch-retry with sensible defaults.
Additional context/Screenshots
2022-01-03T21:04:04.230Z e3ee4716-20fd-452c-91b2-3d153315278f
ERROR Unhandled Promise Rejection
{
"errorType": "Runtime.UnhandledPromiseRejection",
"errorMessage": "FetchError: request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
"reason": {
"errorType": "FetchError",
"errorMessage": "request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
"code": "ECONNRESET",
"message": "request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
"type": "system",
"errno": "ECONNRESET",
"stack": [
"FetchError: request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
" at ClientRequest.<anonymous> (/var/task/node_modules/node-fetch/lib/index.js:1461:11)",
" at ClientRequest.emit (events.js:400:28)",
" at TLSSocket.socketErrorListener (_http_client.js:475:9)",
" at TLSSocket.emit (events.js:400:28)",
" at emitErrorNT (internal/streams/destroy.js:106:8)",
" at emitErrorCloseNT (internal/streams/destroy.js:74:3)",
" at processTicksAndRejections (internal/process/task_queues.js:82:21)"
]
},
"promise": {},
"stack": [
"Runtime.UnhandledPromiseRejection: FetchError: request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
" at process.<anonymous> (/var/runtime/index.js:35:15)",
" at process.emit (events.js:412:35)",
" at processPromiseRejections (internal/process/promises.js:245:33)",
" at processTicksAndRejections (internal/process/task_queues.js:96:32)"
]
}
[ERROR] [1641243844254] LAMBDA_RUNTIME Failed to post handler success response. Http response code: 400.
RequestId: 0eac4614-0a52-4a16-b13e-f33138398502 Error: Runtime exited with error: exit status 128
Runtime.ExitError
