[Bug]: Crash from Stack overflow due to AQL fails beyond certain operand counts in FROM syntax
Kelerchian opened this issue · comments
Product
Actyx
Operating System
None
Current behavior
Let us formulate a term first: Tagchain*[n], which stands for an operation of multiple tags with n number of operand.
For example FROM [Tagchain*2]
stands for FROM 'tag1' | 'tag2'
Problem: Tagchain*[n] fails with a certain limit of n
. The limit decreases following this condition:
- The Tagchain is a nested operand (-1 for each level of nesting)
- The Tagchain is a left operand
Limit example:
FROM [Tagchain*1006]
does not result inFROM [Tagchain*1007]
results in stack overflow for exceeding the base limit
Limit decrease behavior example:
FROM [Tagchain*[limit = 1006]]
// 1006 is base limit
FROM 'sometag' & ( [Tagchain*[limit = 1005]] )
// -1 to being nested operand
FROM 'sometag' & ( [Tagchain*[limit = 1004]] ) & 'someothertag'
// -1 to being nested operand
// -1 to being left operand
`FROM ( [Tagchain*[limit = 1004]] ) & ( [Tagchain*[limit = 1004]] ) & ( [Tagchain*[limit = 1004]] ) & ( [Tagchain*[limit = 1005]] )
// right most: -1 being nested operand
// rest: -2 for being both nested operand and left operand
FROM ( [Tagchain*[limit = 1004]] ) & ( ( Tagchain*[limit = 1003] ) & ( Tagchain*[limit = 1004] ) )
// left most: -2 for being both nested operand and left operand
// mid: -1 for being left operand, -2 for being a level-2 nested operand
// right: -2 for being a level-2 nested operand
Expected behavior
Should not overflow?
How to reproduce
Use Actyx["queryAql"] to call AQL with the samples from current behavior
section.
For example:
const sdk = await ActyxSDK.Actyx.of({
appId: "com.example.trial",
displayName: "com.example.trial",
version: "0.1.0",
});
const tc = (x: number) =>
new Array(x)
.fill(null)
.map((_, i) => `"tag:${i}"`)
.join(" | ");
await sdk.queryAql({
query: `FROM (${tc(1005)}) & ((${tc(1003)}) & (${tc(1004)}))`,
});
sdk.dispose();
Additional notes
-
UX important consideration: While this behavior is not advertised in Actyx docs, this behavior is a potential stand-in for the absent equivalent of
WHERE IN
syntax in SQL. -
There might also be limit decrease factor that I did not catch since these rules are derived from observation of phenomena with limited samples.
-
This limit has only been tested in linux-amd64 build of Actyx
Partial mitigation is done in this PR #623.
Now databank does not entirely crash because of stack overflow, but the related query fails with an error/diagnostic event.
I'm revisiting the stack overflow thing to see if I can use stacker::grow and then I remembered that async works differently:
async {}.boxed()
only constructs the future "state machine"- The recursion happens ONLY when .await of the future is called
- Wrapping the
async {}.boxed()
inside a stacker::grow only put the construction process and not the recursion itself into the new stack - Meanwhile
.await
is not available becausestacker::grow
accepts a sync-closure