Node.js STATUS_STACK_BUFFER_OVERRUN crash
aaronadamsCA opened this issue · comments
This is more of an FYI regarding an upstream bug, but it appears to be causing server-side crashes for us right now, so I wanted to be sure you and other potentially affected users know about it.
- We use your library to send data from our GraphQL server, which runs on Node.js 14 on Azure Functions.
- You use Axios to send HTTP requests with basic authentication:
Lines 278 to 280 in 974f8db
- This appears to be intermittently triggering this Axios crash: axios/axios#3396
- Which is apparently due to some as-yet-undiagnosed Node bug: nodejs/node#36325
Specific error message: node exited with code -1073740791 - this apparently means STATUS_STACK_BUFFER_OVERRUN.
Apparently Node.js 16 is unaffected, but Azure Functions does not support it: https://docs.microsoft.com/en-us/azure/azure-functions/functions-reference-node?tabs=v2#node-version
For us, then, the only remaining workaround on the server side would be to remove analytics-node.
Given we don't want to do that, we plan to look at client-side workarounds that involve an auto-retry on failed requests. But I wanted to get you looped in as a stakeholder here at least, since it's ultimately our installation of your package that seems to be causing this.
If it's possible to use something other than HTTP Basic auth, that could apparently be another effective workaround here.
Let me know if I've got any of this wrong or if you have any ideas, thanks.
Good morning @aaronadamsCA, thank you for your contribution to the library.
I have been testing this problem in Node.js 14 (without Azure Functions) and everything seems to be working correctly there. Maybe it is some integration problem with Azure Functions?
Since it is a specific issue for certain use cases we will be evaluating the impact of making these changes in the authorization of axios, but for now in almost all use cases this works correctly and for now we will keep it that way.
We also save it in the backlog to do it in the future, but at the moment it is not a priority.
Regards, Segment team
To be clear, this crash has nothing to do with Azure Functions, you can see that from the Axios issues I linked. As long as that issue remains in Axios, Segment is going to intermittently crash Node.js 14.
But we were able to work around the issue with a client-side retry mechanism, and then resolve it permanently by upgrading to Node.js 16 as soon as Azure Functions added preview support. At that point the crashes stopped.
And we're moving away from Node on our backend anyway, so I don't need anything else for us here.
Thanks.
Feel free to re-open if necessary. Thank you.
For the record, we’re still seeing this in our Azure Functions running on Node 16 (which is now supported)
@FlipABit If you are experiencing STATUS_STACK_BUFFER_OVERRUN errors with Node.js v16, that's almost certainly a different bug, so I'd suggest you file a new issue.