google / marl

I was looking through the code, and found something I didn't understand: why is the stack pointer register (RSP) set to stacktop - 24 bytes here

marl/src/osfiber_x64.c

Line 36 in 5e1b007

ctx->RSP = (uintptr_t)&stack_top[-3];

Also why is the return adress (indicated by the comment) in stack top - 16 bytes here

marl/src/osfiber_x64.c

Line 37 in 5e1b007

stack_top[-2] = 0; // No return target.

Hi @superbiebel ,

It's been a while since I looked at this, so apologies in advance if what I'm about to write is in any way inaccurate. With that disclaimer:

The System V ABI states:

3.2.2 The Stack Frame
In addition to registers, each function has a frame on the run-time stack. This stack
grows downwards from high addresses. Figure 3.3 shows the stack organization.
The end of the input argument area shall be aligned on a 16 (32 or 64, if
__m256 or __m512 is passed on stack) byte boundary. In other words, the value
(%rsp + 8) is always a multiple of 16 (32 or 64) when control is transferred to
the function entry point. The stack pointer, %rsp, always points to the end of the
latest allocated stack frame.

stack_top is 16 byte aligned, and so we want the stack to remain aligned after the first call by the fiber.

After the fiber context is initialised and switched to, the root of the fiber's callstack is marl_fiber_trampoline, which is a stub function to call the fiber's real entry point function.

An x64 call instruction writes the return address to the top of the stack, and decrements RSP by 8 bytes. A ret instruction uses this 8 byte address to jumps back to the caller's address, in doing so, increments RSP by 8 bytes.

So:

stack_top is allocated with a guaranteed 16 byte alignment
This is passed to marl_fiber_set_target which assigns stack_top - 24 to RSP. At this point the stack is 16 byte mis-aligned.
stack_top[-2] is set to 0, mostly as a precaution. In marl fiber should never return back to the initial marl_fiber_trampoline stack frame, but in case we do, we don't want to ret jump to some random location.
After the fiber's context has been set up, it can be started with a call to marl_fiber_swap. The assembly swaps over the various registers and the new stack is used, and the instruction pointer begins at the start of marl_fiber_trampoline.
marl_fiber_trampoline has one statement - call the target function pointer, which it does with a call instruction. As mentioned above, call starts by saving the return address of marl_fiber_trampoline to the stack, and in doing so, decrements RSP by 8 bytes.
The target function is called, and RSP is now correctly 16 byte aligned.

Does that help?

that does help, what would happen if the fiber reads that 0 as a return adress even though it shouldn't happen? Will it read junk, will it crash...?

It will most likely segfault (crash) as it will try to execute an instruction at address at 0x00.
The marl scheduler does not let fibers return to this base level, so unless you're using the fiber API directly, this should not happen.

alright thx for the explanation, closing this.

[question] why is this line there