Roadmap: Threading Support
kateinoigakukun opened this issue · comments
Recently a lot of work on thread support has been done in WASI: https://github.com/WebAssembly/wasi-threads
After our toolchain support threading, it will unlock offloading computationally heavy operations to threads.
Here is a list of tasks to support threads in SwiftWasm:
- 1. Enable wasm-ld's
--gc-sections
to unlock wasi-libc upgrade
#5128 - 2. #5547
- 3. Update wasi-libc to the version, which includes pthread implementation based on wasi-threads
- 4. Support
wasm32-unknown-wasi-threads
target triple - 5. Build stdlib with atomics enabled
- 6. Add compiler options to enable atomics feature in LLVM code generator
- 7. Setup Swift SDK distribution for the threaded SDK
- 8. Fix unaligned or too large atomic load/store in HeapObject.cpp
- https://github.com/apple/llvm-project/blob/d7f1b250afdb37a89d02731acb26912c1fff5625/clang/lib/CodeGen/CGAtomic.cpp#L837
This libcall requires libclang_rt.builtin.a to include atomic.c with atomics and bulk-memory features enabled, but it makes it difficult to share the archive between non-threaded target, so we should fix Swift runtime side.- swiftwasm/swiftwasm-build#288 will fix this
- 9. Add required options to toolset.json in Swift SDK
- 10. Implement a transformation pass for
i32.atomic.wait
-> spin lock to work around browser limitation on main thread- See WebAssembly/threads#106 about details of this limitation
- 11. JS polyfill implementation of wasi-threads based on Web Worker
Feedback and any help are welcome :)
One more task IMO:
- Add
generic_wasi_base.h
and update the include logic here to support WASI https://github.com/apple/swift-corelibs-libdispatch/blob/5529d351e304eb2312b7b3bfb073f7d8285473e0/os/object.h#L24-L33
Started building stdlib for wasm32-unknown-wasip1-threads
target on main branch: swiftlang#72650
I have a question. Are there still blockers for compiling Swift for the wasm32-unknown-wasip1-threads
target triple?
$ which swiftc
/home/kebo/downloads/swift-wasm-DEVELOPMENT-SNAPSHOT-2024-04-03-a/usr/bin/swiftc
$ swiftc --version
Swift version 6.0-dev (LLVM 3dd3bce1adb5b65, Swift 1f09be97a890c3e)
Target: aarch64-unknown-linux-gnu
$ cat hello.swift
print("hello")
$ swiftc -target wasm32-wasip1-threads -c hello.swift
<unknown>:0: warning: libc not found for 'wasm32-unknown-wasip1-threads'; C stdlib may be unavailable
<unknown>:0: error: could not find module '_Concurrency' for target 'wasm32-unknown-wasip1-threads'; found: wasm32-unknown-wasi, at: /home/kebo/downloads/swift-wasm-DEVELOPMENT-SNAPSHOT-2024-04-03-a/usr/lib/swift/wasi/_Concurrency.swiftmodule
$ uname -a
Linux Brown-rhinoceros-beetle 6.6.3-414.asahi.fc39.aarch64+16k #1 SMP PREEMPT_DYNAMIC Sun Mar 24 19:44:17 UTC 2024 aarch64 GNU/Linux
error: could not find module '_Concurrency' for target 'wasm32-unknown-wasip1-threads'
also happens when using swift build
with the Swift SDK.
I still need some build script engineering 🙃
It's OK for now, thank you.
Tried to fix the issue around atomic.c in libclang_rt.builtin.a, but concluded that it's difficult to fix this with the current toolchain layout of clang, which requires the single builitin library for each arch.
From e3ab0445dce6d0aabdf9719742240eaafc5731a4 Mon Sep 17 00:00:00 2001
From: Yuta Saito <kateinoigakukun@gmail.com>
Date: Sun, 14 Apr 2024 16:48:59 +0000
Subject: [PATCH] [compiler-rt][wasm] Compile atomic.c with atomics and
bulk-memory features
The bulitin object file for atomic.c is only referenced by user code
when the atomics and bulk-memory features are enabled and libcall is
required. However, the atomic.c itself was compiled without those
features and it leads to a linker error because all objects have to have
the feature when `--shared-memory` is enabled.
This patch compiles atomic.c with the atomics and bulk-memory features
enabled only for the object file.
---
compiler-rt/lib/builtins/CMakeLists.txt | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/compiler-rt/lib/builtins/CMakeLists.txt b/compiler-rt/lib/builtins/CMakeLists.txt
index c139917d4f58..5af6c72a9102 100644
--- a/compiler-rt/lib/builtins/CMakeLists.txt
+++ b/compiler-rt/lib/builtins/CMakeLists.txt
@@ -839,11 +839,30 @@ else ()
set(deps_aarch64 lse_builtin_symlinks)
endif()
+ set(BUILTIN_OBJECT_LIBS_${arch})
+ # For WebAssembly, we need to compile the atomic.c with different flags than the rest
+ # to enable atomics and bulk-memory features only for the object file.
+ if((arch STREQUAL "wasm32" OR arch STREQUAL "wasm64") AND "atomic.c" IN_LIST ${arch}_SOURCES)
+ set(BUILTIN_ATOMIC_CFLAGS_${arch} ${BUILTIN_CFLAGS_${arch}})
+ list(APPEND BUILTIN_ATOMIC_CFLAGS_${arch} -matomics -mbulk-memory)
+ add_compiler_rt_object_libraries(clang_rt.builtins.${arch}.atomic
+ ARCHS ${arch}
+ DEPS ${deps_${arch}}
+ SOURCES atomic.c
+ DEFS ${BUILTIN_DEFS}
+ CFLAGS ${BUILTIN_ATOMIC_CFLAGS_${arch}})
+ # Include the atomic object file in the builtins archive
+ list(APPEND BUILTIN_OBJECT_LIBS_${arch} clang_rt.builtins.${arch}.atomic)
+ # Remove atomic.c from the main list of sources
+ list(REMOVE_ITEM ${arch}_SOURCES atomic.c)
+ endif()
+
add_compiler_rt_runtime(clang_rt.builtins
STATIC
ARCHS ${arch}
DEPS ${deps_${arch}}
SOURCES ${${arch}_SOURCES}
+ OBJECT_LIBS ${BUILTIN_OBJECT_LIBS_${arch}}
DEFS ${BUILTIN_DEFS}
CFLAGS ${BUILTIN_CFLAGS_${arch}}
PARENT_TARGET builtins)
--
2.43.2