Error: Cannot find global function tvmjs.runtime.ArrayConcat
DavidGOrtega opened this issue · comments
If I try to compile my own wasm I have this error.
Coming from here #373
I have setup everything again and I can see undefined symbols recreating the wasms
(mlc-chat-venv) Davids-MacBook-Pro:mlc-llm davidgortega$ ./web/prep_emcc_deps.sh
+ emcc --version
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.32-git
Copyright (C) 2014 the Emscripten authors (see AUTHORS.txt)
This is free and open source software under the MIT license.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ npm --version
8.19.2
+ TVM_HOME_SET=/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm
+ git submodule update --init --recursive
+ cd web
+ make
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/picojson -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tokenizers-cpp -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tokenizers-cpp/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/cpp -O3 -std=c++17 -Wno-ignored-attributes -c -MM -MT dist/wasm/mlc_wasm_runtime.bc emcc/mlc_wasm_runtime.cc >dist/wasm/mlc_wasm_runtime.d
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/picojson -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tokenizers-cpp -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tokenizers-cpp/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/cpp -O3 -std=c++17 -Wno-ignored-attributes -emit-llvm -c -o dist/wasm/mlc_wasm_runtime.bc emcc/mlc_wasm_runtime.cc
In file included from emcc/mlc_wasm_runtime.cc:38:
In file included from /Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/cpp/serve/grammar/grammar.cc:8:
/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/cpp/serve/grammar/grammar_parser.h:26:41: warning: '/*' within block comment [-Wcomment]
* - Using # as comment mark instead of /**\/
^
1 warning generated.
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/3rdparty/picojson -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tokenizers-cpp -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tokenizers-cpp/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/cpp -O3 -std=c++17 -Wno-ignored-attributes -o dist/wasm/mlc_wasm_runtime.wasm dist/wasm/mlc_wasm_runtime.bc --no-entry -s WASM_BIGINT=1 -s ALLOW_MEMORY_GROWTH=1 -s STANDALONE_WASM=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0
warning: undefined symbol: _ZN3tvm7runtime17GetCustomTypeNameEh (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime6Object13TypeIndex2KeyEj (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime6Object26GetOrAllocRuntimeTypeIndexERKNSt3__212basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEjjjb (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime6detail12LogFatalImplERKNSt3__212basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEiSA_ (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime6detail14LogMessageImplERKNSt3__212basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEiiSA_ (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime7NDArray5EmptyENS0_10ShapeTupleE10DLDataType8DLDeviceNS0_8OptionalINS0_6StringEEE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime8Registry8RegisterERKNS0_6StringEb (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime8Registry8set_bodyENS0_10PackedFuncE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZNK3tvm7runtime6Object11DerivedFromEj (referenced by top-level compiled C/C++ code)
emcc: warning: warnings in JS library compilation [-Wjs-compiler]
+ cd -
/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm
+ [[ -z /Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm ]]
+ cd /Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web
+ make
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/../ -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/picojson -O3 -std=c++17 -Wno-ignored-attributes -c -MM -MT dist/wasm/wasm_runtime.bc emcc/wasm_runtime.cc >dist/wasm/wasm_runtime.d
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/../ -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/picojson -O3 -std=c++17 -Wno-ignored-attributes -emit-llvm -c -o dist/wasm/wasm_runtime.bc emcc/wasm_runtime.cc
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/../ -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/picojson -O3 -std=c++17 -Wno-ignored-attributes -c -MM -MT dist/wasm/tvmjs_support.bc emcc/tvmjs_support.cc >dist/wasm/tvmjs_support.d
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/../ -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/picojson -O3 -std=c++17 -Wno-ignored-attributes -emit-llvm -c -o dist/wasm/tvmjs_support.bc emcc/tvmjs_support.cc
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/../ -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/picojson -O3 -std=c++17 -Wno-ignored-attributes -c -MM -MT dist/wasm/webgpu_runtime.bc emcc/webgpu_runtime.cc >dist/wasm/webgpu_runtime.d
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/../ -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/picojson -O3 -std=c++17 -Wno-ignored-attributes -emit-llvm -c -o dist/wasm/webgpu_runtime.bc emcc/webgpu_runtime.cc
emcc -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/../ -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dlpack/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/dmlc-core/include -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/compiler-rt -I/Users/davidgortega/Documents/projects/kunzite/z-mlc/mlc-llm/3rdparty/tvm/web/..//3rdparty/picojson -O3 -std=c++17 -Wno-ignored-attributes -o dist/wasm/tvmjs_runtime.js dist/wasm/wasm_runtime.bc dist/wasm/tvmjs_support.bc dist/wasm/webgpu_runtime.bc --no-entry -s WASM_BIGINT=1 -s ALLOW_MEMORY_GROWTH=1 -s STANDALONE_WASM=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 --pre-js emcc/preload.js -s ASYNCIFY=1
warning: undefined symbol: TVMWasmPackedCFunc (referenced by top-level compiled C/C++ code)
warning: undefined symbol: TVMWasmPackedCFuncFinalizer (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime9threading10NumThreadsEv (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN3tvm7runtime9threading15ResetThreadPoolEv (referenced by top-level compiled C/C++ code)
emcc: warning: warnings in JS library compilation [-Wjs-compiler]
python3 emcc/decorate_as_wasi.py dist/wasm/tvmjs_runtime.js dist/wasm/tvmjs_runtime.wasi.js cjs
python3 emcc/decorate_as_wasi.py dist/wasm/tvmjs_runtime.js src/tvmjs_runtime_wasi.js es
+ cd -I can confirm that despite those undefined symbols that v0.2.33 eventually works (the majority fails 🤷
however v0.2.34 and above fails always
not sure whats going on here 😞
This issue (Error: Cannot find global function tvmjs.runtime.ArrayConcat at Instance.getGlobalFuncInternal) also happens when trying the function calling example.