[GR-41372] Compiler backend LLVM broken - "ImageSingletons do not contain key com.oracle.svm.core.graal.code.SubstrateLoweringProviderFactory"
shoffmeister opened this issue · comments
Describe the issue
Trying to enable the LLVM compiler backend using -H:CompilerBackend=llvm results in a native image production failure ImageSingletons do not contain key com.oracle.svm.core.graal.code.SubstrateLoweringProviderFactory
The exact same (trivial) project compiles fine without the LLVM backend - i.e.: C compiler: gcc (redhat, x86_64, 12.2.1)
Steps to reproduce the issue
git clone https://github.com/shoffmeister/graalvm-llvm-defectGRAALVM_HOME=... mvn clean package
//exp: builds a native image using the LLVM toolchain
//act: exception from the Maven plugin
========================================================================================================================
GraalVM Native Image: Generating 'defect-graalvm-llvm' (executable)...
========================================================================================================================
[1/7] Initializing...
(0.0s @ 0.18GB)
Error: ImageSingletons do not contain key com.oracle.svm.core.graal.code.SubstrateLoweringProviderFactory
com.oracle.svm.core.util.UserError$UserException: ImageSingletons do not contain key com.oracle.svm.core.graal.code.SubstrateLoweringProviderFactory
at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.UserError.abort(UserError.java:73)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageSingletonsSupportImpl$HostedManagement.doLookup(ImageSingletonsSupportImpl.java:117)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageSingletonsSupportImpl.lookup(ImageSingletonsSupportImpl.java:44)
at org.graalvm.sdk/org.graalvm.nativeimage.ImageSingletons.lookup(ImageSingletons.java:86)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.graal.GraalConfiguration.createLoweringProvider(GraalConfiguration.java:109)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.graal.meta.SubstrateLoweringProvider.createForHosted(SubstrateLoweringProvider.java:58)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.createBigBang(NativeImageGenerator.java:1090)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:882)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:576)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:536)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:403)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:580)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)
FYI, there are profiles inside the pom.xml which allow building without the LLVM backend override, and with a snapshot of the maven plugin.
Describe GraalVM and your environment:
- GraalVM version: 22.3.0-dev-20220915_2039 (had an exception with different error in 22.2-EE )
- JDK major version: OpenJDK Runtime Environment (Red_Hat-17.0.4.1.1-1.fc36) (build 17.0.4.1+1)
- OS: Fedora Linux 36
- Architecture: AMD64
More details
> ... /22.3.0-dev-20220915_2039/graalvm-ce-java17-22.3.0-dev/bin/gu list
ComponentId Version Component name Stability Origin
---------------------------------------------------------------------------------------------------------------------------------
graalvm 22.3.0-dev GraalVM Core Supported
js 22.3.0-dev Graal.js Supported github.com
llvm 22.3.0-dev LLVM Runtime Core Experimental github.com
llvm-toolchain 22.3.0-dev LLVM.org toolchain Supported github.com
native-image 22.3.0-dev Native Image Early adopter github.com
nodejs 22.3.0-dev Graal.nodejs Supported github.com
wasm 22.3.0-dev GraalWasm Experimental github.com
@fniephaus This is a follow up to the Slack conversation https://graalvm.slack.com/archives/CN9KSFB40/p1663426800036569?thread_ts=1663413641.176299&cid=CN9KSFB40 - here focusing on the most recent CE dev build off github.
The 22.2-EE download throws a different error
[1/7] Initializing... (0.0s @ 0.31GB)
Fatal error: java.lang.IllegalArgumentException: package com.oracle.svm.core.graal.llvm not in contents
at java.base/java.lang.Module.implAddExportsOrOpens(Module.java:940)
at java.base/java.lang.Module.implAddExports(Module.java:859)
at java.base/java.lang.System$2.addExports(System.java:2344)
at java.base/jdk.internal.module.Modules.addExports(Modules.java:105)
at org.graalvm.nativeimage.base/com.oracle.svm.util.ModuleSupport$Access$2.giveAccess(ModuleSupport.java:74)
at org.graalvm.nativeimage.base/com.oracle.svm.util.ModuleSupport.accessModuleByClass(ModuleSupport.java:135)
at org.graalvm.nativeimage.base/com.oracle.svm.util.ModuleSupport.accessPackagesToClass(ModuleSupport.java:122)
at com.oracle.svm.svm_enterprise.llvm/com.oracle.svm.enterprise.llvm.EnterpriseLLVMFeature.afterRegistration(stripped:36)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$setupNativeImage$14(NativeImageGenerator.java:840)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:78)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:840)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:561)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:521)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:407)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:585)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)
which I am merely posting for the sake of search engine indexing.
As a workaround, you need to install the LLVM backend (via $JAVA_HOME/bin/gu install native-image-llvm-backend). We'll add an actionable error message for this soon.
Fix with 1d794ea.
@fniephaus - while this patch obviously fixes the UX for the error message, I don't see how it will address that neither 22-2 (EE) nor 22.3.0-dev-20220915_2039 (CE) are aware of the component native-image-llvm-backend:
22.3-dev (CE):
graalvm/22.3.0-dev-20220915_2039/graalvm-ce-java17-22.3.0-dev on ☁️
❯ gu available
Downloading: Component catalog from raw.githubusercontent.com
ComponentId Version Component name Stability Origin
---------------------------------------------------------------------------------------------------------------------------------
espresso 22.3.0-dev Java on Truffle Supported github.com
espresso-llvm 22.3.0-dev Java on Truffle LLVM Java librSupported github.com
js 22.3.0-dev Graal.js Supported github.com
llvm 22.3.0-dev LLVM Runtime Core Experimental github.com
llvm-toolchain 22.3.0-dev LLVM.org toolchain Supported github.com
native-image 22.3.0-dev Native Image Early adopter github.com
nodejs 22.3.0-dev Graal.nodejs Supported github.com
python 22.3.0-dev GraalVM Python Experimental github.com
R 22.3.0-dev FastR Experimental github.com
ruby 22.3.0-dev TruffleRuby Experimental github.com
visualvm 22.3.0-dev VisualVM Experimental github.com
wasm 22.3.0-dev GraalWasm Experimental github.com
graalvm/22.3.0-dev-20220915_2039/graalvm-ce-java17-22.3.0-dev on ☁️
❯ gu install native-image-llvm-backend
Downloading: Component catalog from raw.githubusercontent.com
Error: Component "native-image-llvm-backend" is not available in catalog.
22-2 EE:
graalvm/22-2/graalvm-ee-java17-22.2.0 on ☁️
❯ gu available
Downloading: Artifacts catalog from gds.oracle.com
Downloading: Release index file from oca.opensource.oracle.com
Downloading: Component catalog for GraalVM Enterprise Edition 22.2.0.1 on jdk17 from oca.opensource.oracle.com
Downloading: Component catalog for GraalVM Enterprise Edition 22.2.0 on jdk17 from oca.opensource.oracle.com
Skipping ULN EE channels, no username provided.
Downloading: Component catalog from www.graalvm.org
ComponentId Version Component name Stability Origin
---------------------------------------------------------------------------------------------------------------------------------
espresso 22.2.0 Java on Truffle Supported gds.oracle.com
espresso-llvm 22.2.0 Java on Truffle LLVM Java librSupported gds.oracle.com
js 22.2.0 Graal.js Supported gds.oracle.com
llvm 22.2.0 LLVM Runtime Core Experimental gds.oracle.com
llvm-toolchain 22.2.0 LLVM.org toolchain Supported gds.oracle.com
native-image 22.2.0 Native Image Early adopter gds.oracle.com
nodejs 22.2.0 Graal.nodejs Supported gds.oracle.com
python 22.2.0 Graal.Python Experimental gds.oracle.com
R 22.2.0 FastR Experimental github.com
ruby 22.2.0 TruffleRuby Experimental gds.oracle.com
visualvm 22.2.0 VisualVM EE Experimental gds.oracle.com
wasm 22.2.0 GraalWasm Experimental gds.oracle.com
graalvm/22-2/graalvm-ee-java17-22.2.0 on ☁️
❯ gu install native-image-llvm-backend
Downloading: Artifacts catalog from gds.oracle.com
Downloading: Release index file from oca.opensource.oracle.com
Downloading: Component catalog for GraalVM Enterprise Edition 22.2.0.1 on jdk17 from oca.opensource.oracle.com
Downloading: Component catalog for GraalVM Enterprise Edition 22.2.0 on jdk17 from oca.opensource.oracle.com
Skipping ULN EE channels, no username provided.
Downloading: Component catalog from www.graalvm.org
Error: Component "native-image-llvm-backend" is not available in catalog.
FWIW, https://github.com/oracle/graal/blame/08a2755f8113e7238f45e790709a3a72323a3871/substratevm/mx.substratevm/mx_substratevm.py#L1056 mentions native-image-llvm-backend, but I don't know whether this is enough to get this deployed?
I think GraalVM EE 22.2 shouldn't show the warning without further ado, but you do need to install the llvm-toolchain component to use the LLVM backend. With 22.3, EE and CE will both have the native-image-llvm-backend, which will also pull the llvm-toolchain as a dependency.
It seem that the dev build you used is outdated.
Can you please retry with the latest dev build, which should have native-image-llvm-backend (e.g., this)?
I just went to https://github.com/graalvm/graalvm-ce-dev-builds/releases and downloaded "GraalVM CE 22.3.0-dev-20221004_1644". This release indeed has gu available listing
native-image-llvm-backend22.3.0-dev Native Image LLVM Backend Early adopter (experimental) github.com
FWIW, I hadn't realized that, in addition to amending the error message, some more action took place to make the machinery available, too.
I'll try to get back at actually exercising this ASAP - many thanks!
Thanks for checking, @shoffmeister! Glad this is working now!
I ran into the same issue with GraalVM 22.3 with the provided example project.
$ java -version
openjdk version "17.0.5" 2022-10-18
OpenJDK Runtime Environment GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08, mixed mode, sharing)
$ gu list
ComponentId Version Component name Stability Origin
---------------------------------------------------------------------------------------------------------------------------------
graalvm 22.3.0 GraalVM Core Supported
llvm-toolchain 22.3.0 LLVM.org toolchain Supported github.com
native-image 22.3.0 Native Image Early adopter github.com
native-image-llvm-backend22.3.0 Native Image LLVM Backend Early adopter (experimental) github.com
native-image-llvm-backend doesn't seem to be available in macOS Aarch64 (ARM/M1). Any plans for having it there?
We are in the process of supporting the LLVM backend on ARM macOS, these builds should be available soon