Too many invocations cause WasmtimeException "sgen-stw.c:77, condition ... not met"
warappa opened this issue · comments
Setup
I wanted to try out the limits of DotNetIsolator and changed the ConsoleSample.Program.cs file to have call runtime.Invoke(...) many more times (because you don't know how often a 3rd party code would call Invoke(...)). I also emptied the lambda body to speed up the invocation and to not have parameters, captured variables or return values have any influence.
Any non-empty lambda body causes the error to happen at an even lower invocation count.
Issue
Up to 35317 invocations there is no problem: The final output of the host is printed in the console (with impressive 0,04ms/call!).
But when incrementing numCalls by just 1 to 35318, it breaks with a WasmtimeException (see below).
Maybe this is an issue of Wasmtime itself, but I'm not sure as I'm not experienced in this kind of magic 😉.
Edited ConsoleSample
using var host = new IsolatedRuntimeHost().WithBinDirectoryAssemblyLoader();
using var runtime = new IsolatedRuntime(host);
//int numCalls = 35317; // GOOD
int numCalls = 35318; // BAD
var sw = new Stopwatch();
sw.Start();
for (var i = 0; i < numCalls; i++)
{
runtime.Invoke(() =>
{
});
}
sw.Stop();
Console.WriteLine($"Done in {sw.ElapsedMilliseconds:F0}ms ({(double)sw.ElapsedMilliseconds / numCalls:F4} ms/call)");Raised Exception
[wasm_trace_logger] * Assertion at /home/runner/work/dotnet-wasi-sdk/dotnet-wasi-sdk/modules/runtime/src/mono/mono/metadata/sgen-stw.c:77, condition `info->client_info.stack_start >= info->client_info.info.stack_start_limit && info->client_info.stack_start < info->client_info.info.stack_end' not met
Unhandled exception. Wasmtime.WasmtimeException: error while executing at wasm backtrace:
0: 0x40754b - <unknown>!<wasm function 7705>
1: 0x405089 - <unknown>!<wasm function 7618>
2: 0x407a16 - <unknown>!<wasm function 7714>
3: 0xe2e68 - <unknown>!<wasm function 709>
4: 0x2d2c58 - <unknown>!<wasm function 4879>
5: 0x3f7937 - <unknown>!<wasm function 7321>
6: 0x3f77e3 - <unknown>!<wasm function 7319>
7: 0x3f7a2b - <unknown>!<wasm function 7323>
8: 0x3f7ab6 - <unknown>!<wasm function 7324>
9: 0x2b7fce - <unknown>!<wasm function 4637>
10: 0x2b7d14 - <unknown>!<wasm function 4635>
11: 0x30b42c - <unknown>!<wasm function 5606>
12: 0x30ade8 - <unknown>!<wasm function 5605>
13: 0x30ac21 - <unknown>!<wasm function 5604>
14: 0x30ab95 - <unknown>!<wasm function 5603>
15: 0x3003b6 - <unknown>!<wasm function 5444>
16: 0x2bd5e5 - <unknown>!<wasm function 4725>
17: 0x2292bf - <unknown>!<wasm function 3284>
18: 0x229110 - <unknown>!<wasm function 3283>
19: 0x92199 - <unknown>!<wasm function 333>
20: 0x62d24 - <unknown>!<wasm function 332>
21: 0x367621 - <unknown>!<wasm function 6309>
22: 0x2110a7 - <unknown>!<wasm function 3085>
23: 0x212d90 - <unknown>!<wasm function 3101>
24: 0x21c3f8 - <unknown>!<wasm function 3187>
25: 0xe3202 - <unknown>!<wasm function 714>
26: 0xe3427 - <unknown>!<wasm function 715>
27: 0x3cd2 - <unknown>!<wasm function 43>
28: 0x4477 - <unknown>!<wasm function 46>
29: 0x419242 - <unknown>!<wasm function 7924>
Caused by:
Exited with i32 exit status 1
at Wasmtime.Function.Invoke(Span`1 argumentsAndResults, StoreContext storeContext)
at Wasmtime.Function.<>c__DisplayClass187_0`3.<WrapFunc>b__0(T1 p0, T2 p1)
at DotNetIsolator.IsolatedRuntime.CopyObject[T](T value) in C:\source\DotNetIsolator\src\DotNetIsolator\IsolatedRuntime.cs:line 115
at DotNetIsolator.IsolatedRuntime.Invoke(Action value) in C:\source\DotNetIsolator\src\DotNetIsolator\IsolatedRuntime.cs:line 308
at Program.Main(String[] args) in C:\source\DotNetIsolator\sample\ConsoleSample\Program.cs:line 17
Expected Result
There should be no upper limit in invocation count.
Additional Information
Tested on 6dbe1a3
This error also happened on previous versions of DotNetIsolator.
Runtime Environment:
OS Name: Windows
OS Version: 10.0.22621
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.100-preview.3.23178.7\
.NET workloads installed:
There are no installed workloads to display.
Host:
Version: 8.0.0-preview.3.23174.8
Architecture: x64
Commit: 47bad717bd
.NET SDKs installed:
3.1.426 [C:\Program Files\dotnet\sdk]
5.0.408 [C:\Program Files\dotnet\sdk]
5.0.416 [C:\Program Files\dotnet\sdk]
6.0.311 [C:\Program Files\dotnet\sdk]
7.0.100-preview.7.22377.5 [C:\Program Files\dotnet\sdk]
7.0.100-rc.1.22431.12 [C:\Program Files\dotnet\sdk]
7.0.105 [C:\Program Files\dotnet\sdk]
7.0.302 [C:\Program Files\dotnet\sdk]
7.0.400-preview.23225.8 [C:\Program Files\dotnet\sdk]
8.0.100-preview.3.23178.7 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0-preview.7.22376.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0-rc.1.22427.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0-preview.3.23177.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0-preview.7.22375.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0-rc.1.22426.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0-preview.3.23174.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.0-preview.7.22377.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.0-rc.1.22427.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.0-preview.3.23178.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Other architectures found:
arm64 [C:\Program Files\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\arm64\InstallLocation]
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]
Environment variables:
Not set
global.json file:
Not found
Personal Note
That's some awesome stuff your doing here! Please keep it on!
I've observed that this is not only caused by too many invocations, but also by too much data transfer.
By limiting to only 1 invocation, using the following code I manually binary searched the maximum amount of bytes I could return.
I cause the same error by either: Incrementing that amount by 1; Or the sum of multiple invocations exceeds that number.
WasiConfiguration wasiConfiguration = new WasiConfiguration().WithInheritedStandardOutput().WithInheritedStandardError();
using var host = new IsolatedRuntimeHost().WithAssemblyLoader(LoadAssembly);
using var isolatedRuntime = new IsolatedRuntime(host);
// Maximum bytes without crash: 0b0111_0110_0100_1010_1110_0011 = 7752419 = 7.39 MiB
const int bytesToGenerate = 0b0111_0110_0100_1010_1110_0011;
IsolatedObject byteGenerator = isolatedRuntime.CreateObject<ReturnManyPseudoRandomByteGenerator>();
byteGenerator.Invoke<int, int, byte[]>("GenerateBytes", 1, bytesToGenerate);
class ReturnManyPseudoRandomByteGenerator {
public byte[] GenerateBytes(int seed, int count) {
Random random = new(seed);
byte[] bytes = new byte[count];
random.NextBytes(bytes);
return bytes;
}
}