aarch32: No easy way for a thread to modify its TPIDRURO register.
kent-mcleod opened this issue · comments
TPIDRURO is the default register used by the -mtp=auto or -mtp=cp15 arguments to gcc or clang on aarch32 when targeting armv7 CPUs for both arm-none-eabi and arm-linux-gnueabi targets. This register is read-only from user privilege and can only be written from supervisor mode. seL4 provides a way for setting this register with seL4_TCB_WriteRegisters, but this invocation cannot be called on itself by the calling thread. On ia32 and some x86_64 configurations seL4 provides a syscall, seL4_SetTLSBase, which updates a read-only register used for TLS, however this syscall isn't available on aarch32 configurations for writing the TPIDRURO register.
It would be useful for a thread to be able to set this register without requiring another thread. Currently, the sel4_libs runtimes use a different register, TPIDRURW, which requires using -mtp=soft which causes the compiler to emit a function call to lookup the tp address. Future versions of gcc appear to add an option to select tpidrurw with -mtp=tpidrurw 1 but this doesn't appear to be supported by any stable releases yet.
A quick solution would be to add a version of seL4_SetTLSBase for aarch32 that can set TPIDRURO, but the drawback of this approach is that that syscall doesn't require any authority to be provided where access to the TCB cap should probably be required.
A harder solution would be to address the drawbacks of seL4_TCB_WriteRegisters() relating to not being able to set individual registers and not being able to be called by the running thread on it's own TCB.
It would be possible to add a mask variant of seL4_TCB_WriteRegisters and seL4_TCB_ReadRegisters [1] where each bit in the mask selects the associated register in seL4_UserContext.
Why can't seL4_TCB_WriteRegisters() be called on the current TCB? The background for this restriction is summarized here [2]. In summary it seems to be for verification convenience. (Resolving this would make it possible to take out seL4_SetTLSBase which prevents user level from implementing a policy supported by hardware - preventing access to the tp segment register (on x86).
[1] - Maybe even the current invocations should be replaced with a mask variant, and legacy functions added to libsel4 that just calls the new invocation with all bits of the mask set.
[2] - https://sel4.atlassian.net/browse/SELFOUR-22
The benefits of fixing this issue for the sel4runtime set of libs would be:
- removing the need to set -mtp=soft: https://github.com/seL4/seL4_tools/blob/master/cmake-tool/helpers/environment_flags.cmake#L52
- removing the need to provide
__aeabi_read_tp: https://github.com/seL4/sel4runtime/blob/master/src/sel4_arch/aarch32/__aeabi_read_tp.S#L38 - removing the need to patch musllib to not access TLS variables through TPIDRURO:
The benefits for other runtimes, eg rust, is that they won't need to work out how to set the equivalent of -mtp=soft and provide the associated emulation.
microkit is unaffected because it doesn't require TLS storage classes (as each PD only has a single thread).
I would just add seL4_SetTLSBase(), that would be the most consistent and easiest solution. If the kernel is not using TPIDRURO for anything itself, it safe to give write access to the register to all tasks, they have that for other registers too.
Generally speaking you don't want to give a task access to its own TCB, so forcing that approach seems worse to me, especially if the goal is to let a task be able to set its own TPIDRURO.
Improving seL4_TCB_WriteRegisters() and adding mask support is a separate issue.
I think seL4 should use the default, best supported register for TLS instead of deviating from the norm, for all architectures. AFAIK aarch64 is non-standard too now. That would also make it trivial to add threading support to Microkit in the future (not for Microkit itself, but for users of Microkit). Now it's a pain because it requires a non-standard libc build, same as for Rust.
AFAIK aarch64 is non-standard too now.
Can you elaborate? On AArch64, libsel4runtime uses tpidr_el0 [1], which seems like it could be considered "standard" based on this GCC manual page [2].
That would also make it trivial to add threading support to Microkit in the future (not for Microkit itself, but for users of Microkit). Now it's a pain because it requires a non-standard libc build, same as for Rust.
For the sake of information, the Rust Microkit runtime already supports thread-local storage [3], via the sel4-initialize-tls crate [4].
[1] https://github.com/seL4/sel4runtime/blob/bb75358167fe8bdc0e3df565bb419dc6eb150a60/include/sel4_arch/aarch64/sel4runtime/thread_arch.h#L43
[2] https://gcc.gnu.org/onlinedocs/gcc/AArch64-Options.html
[3] https://github.com/seL4/rust-sel4/blob/c97687b61b50329aefdcbfcb80ef47008582a0cd/crates/sel4-microkit/src/entry.rs#L25
[4] https://github.com/seL4/rust-sel4/tree/main/crates/sel4-initialize-tls
AFAIK aarch64 is non-standard too now.
Can you elaborate? On AArch64,
libsel4runtimeusestpidr_el0[1], which seems like it could be considered "standard" based on this GCC manual page [2].
Hmm, I looked into TLS support a few years ago and I remember encountered seL4 using a non-standard register for TLS. I think I confused aarch32's register usage with aarch64's one, because as you say, TPIDR_EL0 is pretty standard for aarch64.
For the sake of information, the Rust Microkit runtime already supports thread-local storage [3], via the
sel4-initialize-tlscrate [4].
I shouldn't have said, "same as for Rust". Rust seems to recompile everything for each build anyway, so you don't need specifically build binaries for non-standard TLS registers, you just need to make sure that the Rust/llvm compilers are using the same register as the runtime code.