Error when running inside Docker container without sys_nice

[J. Eduardo]

When importing lanes into a Lua script that runs inside a standard (unprivileged) Docker container, it fails immediately due to an EPERM error, as below:

Script:

#!/usr/bin/env lua5.1

print('Loading and initializing lanes')

require('lanes').configure()

print('If I got here, lanes initialized successfully')

Result inside unprivileged Docker container:

Loading and initializing lanes
src/threading.c 833: pthread_create() failed, 1 EPERM

Result inside Docker container augmented with the sys_nice capability:

Loading and initializing lanes
If I got here, lanes initialized successfully

As I do not have control over the capabilities of the running container for this particular piece of code (which is running inside a hosted container scheduler), I cannot add the sys_nice capability to the Docker container, thus cannot run my script which uses Lua Lanes inside this environment.

Exploring with some test code, I faced some EPERMs when trying to set the priority of a child thread inside itself.

Looking at the relevant file in lanes, I can see the EPERM is happening when trying to create a thread after setting a specific priority for it, which leads me to think that this is the same problem.

I could find no configuration in lanes that would allow me to specify that I do not want to set any specific priorities to threads.

Is there any way I am missing for me to make it possible to run such scripts inside unprivileged Docker containers without the need of fiddling with their capabilities?

[Benoit Germain]

Sorry for the late reply.
Maybe this has to do with the way priority is set in shed_param. However, I am no pthread guru, and in any event I don't have easy access to a linux box. Maybe someone more savvy than me can come up with a pull request?

[J. Eduardo]

Hey @benoit-germain, just a friendly ping about this one. I would be fine to give it a try on fixing it myself.

I imagine this would require one of two options. The first would be to fail gracefully and print a warning when a thread cannot be created due to the absence of sys_nice. The upside is that this would be require no code to be changed, however it would change the current behavior from Lua Lanes, where any error in creating a thread is deemed fatal.

The other option would be to add a configuration flag that, when specified, would disable the capacity to create threads with different priorities, which is the one thing it needs sys_nice for. There is a downside that anyone running Lua Lanes inside an environment where sys_nice is not available would need to explicitly invoke this kind of config option.

Any preferred approach here?

[Benoit Germain]

As you say, Lanes sets priorities with pthread_attr_setschedparam only when using a non-0 priority setting. But as it happens to be the default behavior (see lane_new: int const priority = (int) luaL_optinteger( L, 3, 0);), I would have thought it's exactly what you mean by "disable the capacity to create threads with different priorities" (Unless you run as root)?
This bit of code here is not mine, so I can't say anything about it.

    bool_t const normal =
#if defined(PLATFORM_LINUX) && defined(LINUX_SCHED_RR)
    !sudo; // with sudo, even normal thread must use SCHED_RR
#else
    (prio == 0);
#endif

Is there a means for me to know what exactly pthread_create is not happy about?

[J. Eduardo]

Yep, pthread_create is not happy about not having the sys_nice capability in the container, which is required to renice the thread, which is what I reproduced here outside of the Lanes code.

Your comment helped me to understand the problem better, though. As it is written in the documentation and in your previous comment, Lanes will try to set the thread priority only when running as root. Taking this information, I found out that the problem's source was that the container (running in Gitlab CI) was running the tests as root, hence why the relevant code in Lanes was triggered. It so happens that when I ran the relevant container locally, it was also running the Lanes code as root inside of this container.

While this is easy to fix for a local container, doing so with Gitlab CI requires some clever workaround. After putting some very crude fix in place, I could get the Lanes code, which I am using for integration testing, running inside Gitlab CI but not as root anymore. By doing so, Lanes is no longer trying to renice the thread and the test passes, as you can see here.

So my problem is actually resolved, I just need to make sure I am not running the Lanes code as root inside of the CI container as it will not have the sys_nice capability which is required for Lanes to renice the running threads.

If you think it would be interesting to have an option to disable the renice codepath even when running as root I am happy to contribute with something. Otherwise, I am thankful for your assistance and I am fine with having this closed.

[Benoit Germain]

I guess I could slightly change Lanes behavior regarding the priority parameter, and distinguish the case where the user specifies 0, and the case where no value is provided at all. In the later case, I could just do nothing, and let the user live with the consequences :-)

[J. Eduardo]

Sounds to me like an excellent idea.

[Benoit Germain]

In the code I see // with sudo, even normal thread must use SCHED_RR.
Does this mean I should still pthread_attr_setschedpolicy even if not changing the priority? Or should I skip the whole if(!normal) block?
Maybe the best would be for you to test what works.
For this all you have to do is the following:

In threading.h:
Add #define THREAD_PRIO_DEFAULT (-999) at line 172.

In lanes.c
Replace
int const priority = (int) luaL_optinteger( L, 3, 0); at line 1055
with

    bool_t const have_priority = !lua_isnoneornil( L, 3);
    int const priority = have_priority ? (int) lua_tointeger( L, 3) : THREAD_PRIO_DEFAULT;

Replace
if( priority < THREAD_PRIO_MIN || priority > THREAD_PRIO_MAX) at line 1070
with
if( have_priority && (priority < THREAD_PRIO_MIN || priority > THREAD_PRIO_MAX))
Once this is done, you can determine tow to handle THREAD_PRIO_DEFAULT in THREAD_CREATE, and give me the results.

[Benoit Germain]

Currently, on LINUX/SHED_RR builds, the following happens:
when Lanes detects that it doesn't run as root, priority setting is ignored.
when Lanes detects that it runs as root, Lanes always tries to set a priority even if the user didn't ask anything (which is the cause of your issue, unless I am mistaken).
The more I think about it, the more I am tempted to drop the sudo check in the code, and let the caller decide what will happen, as follows:
specify a prio, Lanes will try to set it. It it fails because of insufficient permissions or capabilities, stop doing that.
specify no prio, Lanes won't try to set it. If the thread doesn't start, then specify a prio.