Hot Loop/Hot Call Jitting Causing Exceptions With lua_resume

[atom0s]

I've also posted this issue on LuaJIT's issue tracker but since the development there has basically haulted, I am also referencing it here. I am interested in using LuaJIT via MoonJit if I can get this issue resolved. :) See here: LuaJIT#582

Hello, I am running into an issue with LuaJIT that is not present on stock Lua. The code also runs fine in LuaJIT until I trigger the hot tracing/Jitting features. (As far as I can tell/guess, it appears that I am hitting the hot loop/hot call count and causing the trace/jitter to trigger with the following example.)

Quick summary of this example, I am creating a Lua state that loads a simple script and then runs said script in a new thread that can be yielded. Afterward, the state remains open and additional tasks can be created and called on the state, using additional new threads. Further, tasks can be created from the Lua side which will create new C-sided coroutines. This allows for individual event calls to all be able to be yielded during runtime.

This is a cut down chunk of code from a much larger project but enough to get the jist across, and enough to trigger the exception.

/**
 * LuaJIT Trace Exception Example
 */

#pragma comment(lib, "lua51.lib")
extern "C"
{
#include "lua.h"
#include "lauxlib.h"
#include "lualib.h"
};

#include <Windows.h>
#include <string>
#include <vector>

enum class TaskStatus : uint32_t
{
    Ok,
    Yield,
    Dead,
    Error
};

struct task_t
{
    lua_State* L;
    int32_t Ref;
    uint32_t ArgCount;
    uint32_t StartTime;
    uint32_t StartDelay;
    TaskStatus Status;
};

std::vector<task_t> g_TaskQueue;  // Main task queue.
std::vector<task_t> g_TaskQueue2; // Newly queued tasks while processing the task queue.

/**
 */
static int32_t lua_Print(lua_State* L)
{
    const auto top = lua_gettop(L);
    lua_getglobal(L, u8"tostring");

    for (auto x = 1; x <= top; x++)
    {
        lua_pushvalue(L, top + 1);
        lua_pushvalue(L, x);
        lua_call(L, 1, 1);

        if (!lua_isstring(L, -1))
            return luaL_error(L, u8"Unable to convert stack data to string for 'print' function.");

        const auto msg = luaL_checkstring(L, -1);
        if (msg)
            printf_s(u8"%s\r\n", msg);
    }

    return 0;
}

/**
 */
static int32_t lua_Task(lua_State* L)
{
    // Expect at least 2 params..
    auto delay = luaL_checknumber(L, 1);

    if (!lua_isfunction(L, 2))
        return luaL_error(L, u8"expected function in param 2!");

    // Create a new thread for the task..
    auto LT = lua_newthread(L);
    auto LR = luaL_ref(L, LUA_REGISTRYINDEX);

    task_t task{};
    task.L          = LT;
    task.Ref        = LR;
    task.ArgCount   = lua_gettop(L) - 2; // All remaining args minus our delay and function..
    task.StartTime  = ::GetTickCount();
    task.StartDelay = delay;
    task.Status     = TaskStatus::Ok;

    // Move the function and args into the task thread..
    lua_xmove(L, task.L, lua_gettop(L) - 1);

    // Queue the task..
    g_TaskQueue2.push_back(task);

    // Yield the call that created the task..
    return lua_yield(L, 0);
}

/**
 */
void processQueue(lua_State* L)
{
    // Move newly created tasks to the main queue..
    if (!g_TaskQueue2.empty())
    {
        g_TaskQueue.insert(g_TaskQueue.end(),
            std::make_move_iterator(g_TaskQueue2.begin()),
            std::make_move_iterator(g_TaskQueue2.end()));
        g_TaskQueue2.clear();
    }

    // Cleanup all dead tasks..
    auto iter = g_TaskQueue.begin();
    while (iter != g_TaskQueue.end())
    {
        if ((*iter).Status == TaskStatus::Error || (*iter).Status == TaskStatus::Dead)
        {
            // Cleanup the task reference..
            luaL_unref(L, LUA_REGISTRYINDEX, (*iter).Ref);
            iter = g_TaskQueue.erase(iter);
        }
        else
            ++iter;
    }

    // Process the task queue..
    for (auto i = g_TaskQueue.begin(), iend = g_TaskQueue.end(); i != iend; ++i)
    {
        // Check if the current task is ready..
        if (i->StartTime > 0 && i->StartDelay > 0)
        {
            const auto delay = i->StartTime + (i->StartDelay * 1000);
            if (delay > ::GetTickCount())
                continue;
        }

        // Resume the current task..
        auto status = lua_resume(i->L, i->ArgCount);
        switch (status)
        {
            case 0: // Ok
                i->Status = TaskStatus::Dead;
                break;
            case 1: // Yield
                i->Status = TaskStatus::Yield;
                break;
            default:
            {
                i->Status = TaskStatus::Error;

                const auto err = luaL_checkstring(i->L, -1);
                if (err)
                    printf_s(u8"%s\r\n", err);

                break;
            }
        }
    }
}

/**
 */
int32_t __cdecl main(int32_t argc, char* argv[])
{
    // Prepare the Lua state..
    lua_State* L = luaL_newstate();
    luaL_openlibs(L);
    lua_register(L, u8"print", lua_Print);
    lua_register(L, u8"task", lua_Task);

    // Load the main script..
    auto status = luaL_loadfile(L, u8"main.lua");
    if (status != 0)
    {
        const auto err = luaL_checkstring(L, -1);
        if (err)
            printf_s(u8"%s\r\n", err);
        lua_close(L);
        return 0;
    }

    // Create a new thread and reference it for the main script execution..
    auto LT = lua_newthread(L);
    auto LR = luaL_ref(L, LUA_REGISTRYINDEX);

    // Move the current top function to the new thread..
    lua_xmove(L, LT, 1);

    // Insert the main script execution into the task queue..
    task_t task{};
    task.L          = LT;
    task.Ref        = LR;
    task.ArgCount   = 0;
    task.StartTime  = ::GetTickCount();
    task.StartDelay = 0;
    task.Status     = TaskStatus::Ok;

    g_TaskQueue.push_back(task);

    // Run the application..
    while (true)
    {
        ::Sleep(1);

        // Process the queue..
        processQueue(L);

        // Hotkey helpers..
        if (::GetAsyncKeyState(VK_F1) & 1)
        {
            auto LTT = lua_newthread(L);
            auto LRR = luaL_ref(L, LUA_REGISTRYINDEX);

            task_t task;
            task.L          = LTT;
            task.Ref        = LRR;
            task.ArgCount   = 0;
            task.StartTime  = ::GetTickCount();
            task.StartDelay = 0;
            task.Status     = TaskStatus::Ok;

            lua_getglobal(LTT, "main");
            auto res = lua_resume(LTT, task.ArgCount);
            if (res > LUA_YIELD)
            {
                const auto err = luaL_checkstring(L, -1);
                if (err)
                    printf_s(u8"%s\r\n", err);
                throw;
            }
            else if (res == LUA_YIELD)
            {
                task.Status = TaskStatus::Yield;
                g_TaskQueue2.push_back(task);
            }
        }
        if (::GetAsyncKeyState(VK_F2) & 1)
            break;
    }

    // Cleanup..
    lua_close(L);
    return 0;
}

And the main.lua Lua script:

local x = 0;
function main()
    coroutine.yield();
    print(string.format('queue task: %d', x));
    task(2, function(x)
        print(string.format('  >> task: %d', x));
    end, x);
    x = x + 1;
end

Tested against:

• Lua 5.1.5 stock.
• LuaJIT 2.0.5 stock.
• LuaJIT 2.1.0-beta 3 stock.
• LuaJIT (via MoonJIT)

Lua 5.1.5 stock works fine and has no issues with this setup at all. However, LuaJIT has a handful of features (especially FFI) that I wish to make use of and want to make the move to LuaJIT. (I have no need/interest in Lua 5.2/5.3/etc.) but this is preventing me from wanting to make the move because I cannot figure out why this is throwing an exception.

To reproduce the exception:

• Compile my above code with LuaJIT 2.1.0-beta 3 stock or other JIT versions listed above.
• Run the code.
• Press and hold F1. You will see tasks queue up and execute.
• Once the call count to main() hits 110ish the hot loop/hot call feature of LuaJIT will trigger and begin tracing the additional future calls to main().
• Once the 112th call happens, lua_resume inside of processQueue() will throw an exception.

To reproduce the error, run this code + any stock LuaJIT listed above. On Windows, press and hold F1 with the window open, this will trigger the code to call the main function and will trigger new tasks to be generated. Around the 112th task, the hot tracing will kick in (seems to be 56*2 with this setup, however doing inner loops and a second coroutine yield will make it trigger in the stock 56.)

I can narrow it down to being an issue with my task( ... ) call and using an anonymous function. If I move the function to a global / named function, then this does not happen. But I am unsure why that is happening and assume this may be a bug with the jitter since it works fine until the jitter triggers. As mentioned this works fine until the jitter triggers and works fine on stock Lua, so I do not see why using an anonymous function would cause this issue.

[atom0s]

Forgot to include the exception information, sorry about that.

Exception:

Exception thrown at 0x771835D2 (KernelBase.dll) in test.exe: 0xE24C4A02.

Callstack:

 	KernelBase.dll!_RaiseException@16�()	Unknown
>	lua51.dll!err_raise_ext(int errcode=0x00000002) Line 504	C
 	lua51.dll!lj_err_throw(lua_State * L=0x0015c448, int errcode) Line 528	C
 	lua51.dll!lj_trace_err_info(jit_State * J=0x00140378, TraceError e=LJ_TRERR_NYIBC) Line 48	C
 	lua51.dll!lj_record_ins(jit_State * J=0x00140378) Line 2039	C
 	lua51.dll!trace_state(lua_State * L=0x0015c448, int(*)(lua_State *) dummy=0x00000000, void * ud=0x00140378) Line 670	C
 	lua51.dll!_lj_vm_cpcall�()	Unknown
 	lua51.dll!lj_trace_ins(jit_State * J=0x00140378, const unsigned int * pc=0x0014a6fc) Line 729	C
 	lua51.dll!lj_dispatch_ins(lua_State * L=0x004ff891, const unsigned int * pc=0x0014a700) Line 424	C
 	lua51.dll!_lj_vm_inshook�()	Unknown
 	ucrtbase.dll!__initterm�()	Unknown
 	e006e900()	Unknown

The exception does not seem to kill the application either. It can be ignored and the app will continue fine. It also does not seem to re-trigger the exception after it's happened the first time. When resuming, the lua_resume call will then return LUA_YIELD as expected and not show any error information on the stack, as if nothing happened.

---

I have found another way to cause this with no custom code at all. Just stock Lua on the above mentioned versions of LuaJIT. You can cause this same exception to happen just by creating and running coroutines with anonymous functions in a loop:

function main()
    local x = 0;
    while (x < 150) do
        -- Create and start a new coroutine..
        local co = coroutine.wrap(function()
            print('hi');
        end);
        co();
        
        -- Step x..
        x = x + 1;
    end
end
main();

This will trigger the same exception. This also keeps triggering it as the 'hot' trace feature is called in its normal manner. (Seems to happen in multiples of the stock 56 'hotcall' setting.)

[atom0s]

Mike responded over on LuaJIT's tracker about this and why I am seeing this exception, so this is cleared up as to what it is. See here: LuaJIT#582 (comment)