[Bug] Local time skipping test server throws `Event set unexpectedly empty`

[Michael Plaisted]

What are you really trying to do?

Use local test server with time skipping to unit test workflows.

Describe the bug

When attempting to run tests using the local test server the workflow throws System.InvalidOperationException : Event set unexpectedly empty on calling GetResultAsync() on the WorkflowHandle. Additionally calls to WorkflowEnvironment.DelayAsync() wait real time which I would expect to finish immediately and advance worfklow time.

Minimal Reproduction

public class WorkflowTests
{
    private static TemporalWorker GetWorker(IWorkerClient client)
    {
        return new TemporalWorker(client, new()
        {
            TaskQueue = $"tq-{Guid.NewGuid()}",
            Activities = { },
            Workflows = { typeof(ExampleWorkflow) },
        });
    }
    [Fact]
    public async Task It_Dedups_Inside_Window()
    {
        await using var env = await WorkflowEnvironment.StartTimeSkippingAsync();
        
        using var worker = GetWorker(env.Client);

        var id = "unit-test";

        await worker.ExecuteAsync(async () =>
        {
            var wfA = await StartWorkflow("A");

            // this waits real time, not skipped,
            // I have tried combinations of time skipping or WithAutoTimeSkippingDisabled for the call but always get a real wait
            await env.DelayAsync(TimeSpan.FromSeconds(10));  

            var wfB = await StartWorkflow("B");
            
            var rA = await wfA.GetResultAsync(); // this throws `System.InvalidOperationException : Event set unexpectedly empty`
            var rB = await wfB.GetResultAsync();

            Assert.Equal(rA, rB);
        });

       

        async Task<WorkflowHandle<string>> StartWorkflow(string req)
        {
            return await env!.Client.StartWorkflowAsync(
                ExampleWorkflow.Ref.RunAsync,
                (string?)null,
                new WorkflowOptions
                {
                    ID = "dedup:" + id,
                    TaskQueue = "demo-tasks",
                    StartSignal = "AddValue",
                    StartSignalArgs = new List<object> { req }
                });
        }
    }
}


[Workflow]
public class ExampleWorkflow
{
    private static TimeSpan DedupDelay = TimeSpan.FromSeconds(15);
    private static TimeSpan MaxDelay = TimeSpan.FromSeconds(30);
    public static readonly ExampleWorkflow Ref = WorkflowRefs.Create<ExampleWorkflow>();

    private string? value = null;
    private DateTime lastMessage;

    [WorkflowRun]
    public async Task<string> RunAsync(string? trigger)
    {
        var firstTime = lastMessage = Workflow.UtcNow;
        value = trigger;

        var max = Workflow.DelayAsync(MaxDelay);

        while (true)
        {
            var initial = lastMessage;
            var waitWindow = DedupDelay - (Workflow.UtcNow - lastMessage);
            Workflow.Logger.LogInformation("[{EventName}] {Seconds} seconds", "DedupWait", waitWindow.TotalSeconds);
            var window = Workflow.DelayAsync(waitWindow);
            var finished = await Task.WhenAny(window, max);
            if (finished == max)
            {
                Workflow.Logger.LogInformation("[{EventName}]", "MaxDurationExceeded");
                break;
            }
            else if (initial == lastMessage)
            {
                Workflow.Logger.LogInformation("[{EventName}]", "WaitPeriodFinished");
                break;
            }
        }

        if (value == null)
        {
            throw new ApplicationException("No value");
        }
        var used = value;
        value = null;

        // simiulate work
        await Workflow.DelayAsync(5000);

        // catch signal that happend when work happening
        if (value != null)
        {
            Workflow.Logger.LogInformation("[{EventName}] Starting new WF", "ExtraMessages");
            throw Workflow.CreateContinueAsNewException(Ref.RunAsync, value);
        }
        return used;
    }

    [WorkflowSignal]
    public Task AddValue(string signal)
    {
        Workflow.Logger.LogInformation("[{EventName}] {DateTime}", "NewValue", Workflow.UtcNow);
        value = signal;
        lastMessage = Workflow.UtcNow;
        return Task.CompletedTask;
    }
}

Environment/Versions

• Windows 11, x64
• Temporal Version: 0.1.0-alpha4 (sdk-dotnet)

[Chad Retz]

Thanks! I will replicate and try to fix soon.

// this waits real time, not skipped,
// I have tried combinations of time skipping or WithAutoTimeSkippingDisabled for the call but always get a real wait

We only auto-skip when you wait on workflow result and no activities are running. If you want to start auto-skipping, start the GetResultAsync earlier (can be backgrounded or whatever). You can call DelayAsync on the env to manually skip time though.

[Michael Plaisted]

Okay, that makes sense. The DelayAsync on env was waiting the real time in this case so may be a problem there as well.

[Chad Retz]

👍 Will attempt to replicate then fix (may be a bit due to conflicting priorities)

[Chad Retz]

The unset-event issue will be solved with #78, so I have opened #77 for the fact that delay is real time and not properly skipping.