dyld lazy symbol binding fails
csapoadam opened this issue · comments
I am trying to get the basic examples for JXCore addons to work on Max OS X v10.10.2. I have tried both the example posted on github ( https://github.com/jxcore/jxcore-addon ) and the first example posted here ( http://jxcore.com/docs/addons.html ). In both cases I receive 5 warnings after running 'jx install', but the build/Release/binding.node dylib is generated. When I separately run 'node-gyp configure' and 'node-gyp build', the example on github does not compile on account of some errors, while the example in the jxcore docs compiles without even warnings.
However, in both cases there is a problem with exported symbols, e.g.:
dyld: lazy symbol binding failed: Symbol not found: __ZN5MozJS5Value11FromIntegerEP9JSContexti
While the binding.node file does contain the following symbols (among many others):
__ZN4node7commons21getInstanceByThreadIdEi__ZN5MozJS12EngineHelper12CreateObjectEP9JSContextPKcPNS_5ValueE__ZN5MozJS14ThrowExceptionC1ENS_9Exception5ErrorE__ZN5MozJS5Value11FromIntegerEP9JSContexti__ZN5MozJS5Value11SetPropertyEPKcN2JS6HandleINS3_5ValueEEE__ZN5MozJS5Value17SetStaticFunctionEPKcPFbP9JSContextjPN2JS5ValueEEt
the symbol in question is indeed undefined (as confirmed by nm).
In the case of the example on the documentation page I have had similar results, with the difference that the undefined symbol referred to String::NewSymbol() and String::New().
Based on this experience, I have tried modifying the binding.gyp files a bit, by adding: 'OTHER_LDFLAGS': ['-ljx']
(hoping that libjx.dylib would contain the missing symbols)
In this case, the example on the docs page still fails with the same linker problem, while the example on github yields
TypeError: binding.sum is not a function (/Users/csapoadam/Documents/Development/Cpp/JXCore/AddonOnlineExample/test.js 3:25)
at (/Users/csapoadam/Documents/Development/Cpp/JXCore/AddonOnlineExample/test.js:3:26)
I've previously installed both the static, dynamic libraries for JXCore, as well as the executable version (all for the SpiderMonkey engine). Basic examples for embedded JXCore in C++ are running without a problem.
Thank you in advance.
Just as an update, for the case where the test.js program actually runs (github example), it seems that binding.sum is seen by the interpreter as an 'object' rather than a function, i.e. running
var binding = require('./build/Release/binding');
console.log(Object.getOwnPropertyNames(binding));
console.log(Object.getOwnPropertyNames(binding).map(function(p){
return typeof binding[p];
}));
console.log(Object.getOwnPropertyNames(binding.sum));
console.log(Object.getOwnPropertyNames(binding.sum).map(function(p){
return typeof binding[p];
}));
prints:
[ 'sum' ]
[ 'object' ]
[ 'length', 'name' ]
[ 'undefined', 'undefined' ]
... the result being that binding.sum(1,2) does not work.
Does anybody have any ideas as to what might be the issue?
it seems that binding.sum is seen by the interpreter as an 'object' rather than a function
Interesting, let me check this. Same for both V8 and SM builds ?
I'm not sure as I only tried it using SM. I may try v8 later.
Thanks for looking into this.
try
sudo rm -rf ~/.node-gyp/jxb300
and also you may need to delete 'build' folder for a clean install. It should work fine.
Thanks for the suggestion. Unfortunately it doesn't work. I deleted the jxb300 folder with all of its contents from my home folder... then removed build from the project folder (i.e. the one that contains binding.cc, binding.gyp and test.js), then, I rebuilt the dylib by running "jx install" in the project folder. When running "jx test.js" I still get:
[ 'sum' ]
[ 'object' ]
[ 'length', 'name' ]
[ 'undefined', 'undefined' ]
TypeError: binding.sum is not a function (/Users/csapoadam/Documents/Development/Cpp/JXCore/AddonOnlineExample/test.js 15:25)
... as it thinks that binding.sum is an 'object'.
As a sidenote, when building the dylib, the jxb300 folder is re-created, and I receive warnings such as:
/Users/csapoadam/.node-gyp/jxb300/src/jsfriendapi.h:2025:40: warning: offset of on non-standard-layout type
'JSJitMethodCallArgs' [-Winvalid-offsetof]
static const size_t offsetOfArgc = offsetof(JSJitMethodCallArgs, argc_);
and
/Users/csapoadam/.node-gyp/jxb300/src/jx/btree.h:919:10: warning: class 'btree_internal_locate_compare_to' was
previously declared as a struct [-Wmismatched-tags]
friend class btree_internal_locate_compare_to;
However, when I try to do "node-gyp configure" and "node-gyp build" instead, no jxb300 folder is created and (perhaps for this reason) I receive the following errors:
gyp info it worked if it ends with ok
gyp info using node-gyp@2.0.1
gyp info using node@0.10.38 | darwin | x64
gyp info spawn make
CXX(target) Release/obj.target/binding/binding.o
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
../binding.cc:6:1: error: unknown type name 'JS_LOCAL_METHOD'
JS_LOCAL_METHOD(sumMethod) {
^
../binding.cc:7:3: error: expected expression
if (args.Length() < 2 || !args.IsInteger(0) ||
^
../binding.cc:15:2: error: expected ';' after top level declarator
}
^
;
3 errors generated.
Perhaps these warnings and errors can provide some clues.
PS. I've also tried the example with V8. I have the following symbolic links now on the path in /usr/local/bin and /usr/local/lib:
jx -> ./jxv8
jxsm -> /Users/csapoadam/Documents/Development/Install/JXCoreBinSM/bin/jx
jxv8 -> /Users/csapoadam/Documents/Development/Install/JXCoreBinV8/bin/jx
libjx.dylib -> ./libjxv8.dylib
libjxsm.dylib -> /Users/csapoadam/Documents/Development/Install/JXCoreSM/bin/libjx.dylib
libjxv8.dylib -> /Users/csapoadam/Documents/Development/Install/JXCoreV8/bin/libjx.dylib
I've also reinstalled node-gyp for the v8 version. I wasn't sure if this was necessary.
Now running jx install on the example still gives 3 warnings, but finally binding.sum is seen as a function! And yet, calling it results in a segmentation fault:
[ 'sum' ]
[ 'function' ]
[ 'length', 'name' ]
[ 'undefined', 'undefined' ]
Segmentation fault: 11
here is the full script:
var binding = require('./build/Release/binding');
console.log(Object.getOwnPropertyNames(binding));
console.log(Object.getOwnPropertyNames(binding).map(function(p){
return typeof binding[p];
}));
console.log(Object.getOwnPropertyNames(binding.sum));
console.log(Object.getOwnPropertyNames(binding.sum).map(function(p){
return typeof binding[p];
}));
console.log("1 + 2 = " + binding.sum(1,2));
and here is the binding.gyp file:
{
'targets': [
{
'target_name': 'binding',
'sources': [ 'binding.cc' ],
'xcode_settings': {
'OTHER_CPLUSPLUSFLAGS': ['-std=c++11', '-stdlib=libstdc++'],
'OTHER_LDFLAGS': ['-ljxv8'],
}
}
]
}
@csapoadam If you are using a precompiled JXcore binary, we are releasing a new one today. Otherwise please sync to latest and it should work. I'm testing it with latest and don't see any problem.
I downloaded the sources 2 or 3 days ago, and compiled them myself. Anyway, I will give a fresh install a try. Thanks.
Hi Obastemur, I downloaded the new source code, compiled it again and tried running jx install node-gyp. I receive a python TypeError (bad argument in native:jxcore_js_object 5:11, thrown at evalmachine.:84:12).
Luckily I kept the older jx executable and tried the same install with that one, in which case node-gyp was installed successfully... so the difference really seems to be in the new executable.
Then again, after I install node-gyp with the old excutable, I get the same problem (i.e. addon function seen as object).
Maybe I am missing something evident, and I would be glad to give your precompiled binaries a try (on Mac) once they are made available.
(BTW I was using python version 2.6.9 just now because 2.7.x did not work earlier, not to mention python3.4)
Something is not right there. I'm using the latest also. Maybe you could try deleting ~/.jx folder and give it another shot. steps below;
assuming you are under the addon folder;
sudo rm -rf ~/.jx
sudo rm -rf ~/.node-gyp
sudo rm -rf build
then jx install and it should work just fine
I made 4 screenshots to show what I am doing. I think I am doing exactly these steps. The first screenshot (red background) shows that with the new version of the jx executable, I am unable to install node-gyp. So I revert to the older version (after cleaning everything again as you suggested), install node-gyp, and have the same problem. (the second case has a green background, but the output was longer so I split it into 3 screenshots, sorry about the length).
Can you please use the jx binary from downloads page ? I just tested it on another mac and it went straight.
Other things:
Python 2.6 should be fine but I don't know if it somehow breaks something in node-gyp. It require 2.7.
I've downloaded it and tried it. Unfortunately, same result.
Then I went back to the source code I downloaded on June 1, and just to make sure something hasn't gone broken since I last compiled it, I compiled the older code once again. Using the newly generated binary, the older problem (object instead of function) re-appeared. This shows that unless I am really misunderstanding something (which is always a possibility), something really has changed in the past few days. But no matter, because the old version is having problems with the addons, too.
Below I attach screenshots to show the whole process I went through:
Test with the fresh binary:
OK, let's go back to the older sources:
sudo make install worked fine, here's the end of it:
node-gyp install throws some warnings but otherwise it's fine.
same problem as before. Let's go back to the newest binary (one last try):
I have a colleague with a MacBook that had no JXCore installed yet, so we may give it a try in the coming days. In the meantime, please let me know if you have any ideas. Thanks!
@obastemur I made a screencast of an effort at a clean install. My colleague and I are both having the same python ValueError problem when jx tries to install npm. Please see the screencast here:
http://www.coginfocom.hu/documents/JxcoreAddon.mov
Interestingly, the details on the error talk about some files in ~/.jx/node-modules/..., whereas no such directory is created by the installer.
@csapoadam Thank you for your all effort. I was trying to follow the conversation and I may say, that I'm reproducing your problems with SM build (with both JXcore 3.0.1 and latest 3.0.2), while I didn't have any problem with V8 build. The following one-liner works for me:
$ cd jxcore-addon; rm -rf ~/.jx ~/.node-gyp/ build/; jxv8 install; jxv8 test.js @csapoadam I've updated one of the files downloaded by jx install and SM build started to work for me, could you please try also this one (I mean exactly the steps from your screencast)?
@ktrzeciaknubisa Hi, thanks for letting me know about that. Yes, I tried it with v8 just now and it works perfectly as you said. That's good news!
However, with the SM build I am still having the same TypeError when it tries to install npm. In the long run I am planning to use the SM build to ensure compatibility with iOS.
Oh, sorry, let me try. I didn't notice the second message.
@ktrzeciaknubisa Unfortunately I still get the same TypeError. I made a screencast just in case I'm missing something:
This is weird. Could you also remove just for test jx from $PATH, if you have one? (I saw it on your screenshots). Let's find out if that matters, even if that didn't matter for me.
Unfortunately it doesn't matter. I removed it earlier but now I double checked and also removed references to the older dylibs.
Is there a way to check that jx install is actually downloading the newer version of the file that you switched?
I was thinking about the same. So the file is taken from https://s3.amazonaws.com/nodejx/npmjx300.tar.gz and saved into ~/.jx/npm.tar.gz. It looks like SM has problem with extracting it (V8 doesn't).
V8:
Preparing NPM for JXcore (v Beta-0.3.0.2) for the first run
Downloading NPM for JXcore
..
NPM for JX is ready.
executing... please wait.SM (in your case, since for me it works now as V8 above):
Preparing NPM for JXcore (v Beta-0.3.0.2) for the first run
Downloading NPM for JXcore
..
ERRORMy suggestion is that you can compare the file taken with jxv8 install and jxsm install. Please remember about rm -rf ~/.jx before each of them.
Are you using any proxy server that might interfere?
OK, I have two folders: npmv8 and npmsm. Running a diff on them, I see that there are 348 files that are in npmv8 but are not in npmsm.
The error message mentions a file with path /Users/csapoadam/.jx/node_modules/tar.gz/node_modules/fstream/lib/writer.js.jx:287:7
... but in .jx/node_modules I have no tar.gz folder... so jumping directly to ./jx/node_nmodules/fstream takes me to an empty folder in the case of SM, while there are several files inside the V8 folder, i.e.
"Only in npmv8/node_modules/fstream: .npmignore
Only in npmv8/node_modules/fstream: .travis.yml
Only in npmv8/node_modules/fstream: LICENSE
Only in npmv8/node_modules/fstream: README.md
Only in npmv8/node_modules/fstream: examples
Only in npmv8/node_modules/fstream: fstream.js
Only in npmv8/node_modules/fstream: lib
Only in npmv8/node_modules/fstream: package.json"
Is this the problem? Or do you think the folder is empty because of an earlier issue in the install process? Here is the full output of diff -rq : http://www.coginfocom.hu/documents/diff.txt
I don't think I'm using a proxy or anything that would cause this. Also, if I take the npm.tar.gz beside the ~/.jx/npm folder, and unpack that, it turns out that there are very few differences compared to the v8 version:
csapo3:Desktop csapoadam$ diff -rq npmv8/ npmunpacked/
Files npmv8/.DS_Store and npmunpacked/.DS_Store differ
Only in npmv8/node_modules: .DS_Store
Only in npmv8/node_modules/fstream: .DS_Store
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp: init.pyc
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp: common.pyc
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp/generator: init.pyc
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp/generator: make.pyc
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp/generator: xcode.pyc
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp: input.pyc
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp: xcode_emulation.pyc
Only in npmv8/node_modules/node-gyp/gyp/pylib/gyp: xcodeproj_file.pyc
Only in npmv8/: npmrc
This seems to show that it is not a problem of downloading, but rather, a problem of unpacking the tar.gz archive...
PS. And when I run ./jx install, and it fails... if I then delete the ~/.jx/npm folder and unpack the npm.tar.gz folder manually in its place... then re-run ./jx install, then everything works! So it seems that it has to be an issue with the unpacking part.
Yeap, that's what I was suspecting too ("It looks like SM has problem with extracting"), however I cannot guess why is that. I'm testing it locally and I cannot reproduce the problem any more. I've tried to remove npm and node-gyp from a global path, too. That didn't help.
I'm slowly out of ideas, but I've refreshed another file on the server. Could you please try one more time? Maybe for both jx sm: 1. downloaded and 2. compiled on your machine?
$ cd jxcore-addon; rm -rf ~/.jx ~/.node-gyp/ build/; jxsm install; jxsm test.js With the precompiled binary, I'm getting the same TypeError.
So I downloaded the sources anew, and built the executable with ./configure --engine-mozilla. Still, no difference. But when I go to ~/.jx/, delete npm and manually extract the .tar.gz, it works afterwards.
Anyway, I guess this is a minor issue given that at least the example is working now and I was able to begin experimenting with it. Thanks for all your help until now! Let me know in case any solution turns up.
I'm wondering, if any of the following npm_modules you have available in global path, and whether they are used instead of expected ones. Below is the list of modules used to extract the contents.
└─┬ tar.gz@0.1.1
├─┬ commander@1.1.1
│ └── keypress@0.1.0
├─┬ fstream@0.1.31
│ ├── graceful-fs@3.0.2
│ ├── inherits@2.0.1
│ ├─┬ mkdirp@0.5.0
│ │ └── minimist@0.0.8
│ └── rimraf@2.2.8
└─┬ tar@0.1.20
├── block-stream@0.0.7
└── inherits@2.0.1Could you please see if you have some of them in global path /usr/local/lib/node_modules? Or better to test if the problem vanishes once you'll rename the folder into e.g. /usr/local/lib/node_modules_old ?
Pre-compiled binaries are built with ./configure --engine-mozilla --compress-internals. This shouldn't matter, but since we're walking in a dark, might be also worth to check it (from what you've mentioned I saw, that you didn't use --compress-internals).
I did have some modules installed, some of them may be similar. But I renamed the folder now and the problem hasn't gone away:
Yes, I usually only used the --prefix and --engine-mozilla options. Now I tried it with:
./configure --prefix=/Users/csapoadam/Documents/Development/Install/jxcoresm-jun8 --engine-mozilla --compress-internals
... and received an error at the end of make install, the end of which was:
/bin/sh: jx: command not found
make[1]: *** [/Users/csapoadam/Documents/Development/Install/jxcore-master-jun8/out/Release/obj/gen/jx_natives.h] Error 127
make: *** [jx] Error 2
however, if I leave out --compress-internals, the build is successful.
Yeap, I forgot to mention, that jx needs to be available in $PATH in order to compile with --compress-internals. Sorry for that.
In order to compile with compressed build, you should have a jx binary ready on your platform.
@ktrzeciaknubisa OK, I built it with jx already in the path. Then I switched the path link to the newly built binary... but I am still having the same result.
On a different note, I am still trying to compile and use the examples provided here: http://jxcore.com/docs/addons.html
I know that you have built an API around this, but I am trying to better understand the link between the two. Right now, I am still at the first example:
#include <node.h>
#ifdef USES_JXCORE_ENGINE
#include <jxcore.h>
#endif
#include <v8.h>
using namespace v8;
Handle Method(const Arguments& args){
HandleScope scope;
return scope.Close(v8::String::New("world"));
}
void init(Handle exports){
exports->Set(v8::String::NewSymbol("hello"),
FunctionTemplate::New(Method)->GetFunction());
}
NODE_MODULE(addon, init)
and although the code compiles, when I run an example I get:
dyld: lazy symbol binding failed: Symbol not found: __ZN2v86String9NewSymbolEPKci
Referenced from: /Users/csapoadam/Documents/Development/Cpp/JXCore/addonStandardExample/build/Release/addon.node
Expected in: flat namespace
dyld: Symbol not found: __ZN2v86String9NewSymbolEPKci
Referenced from: /Users/csapoadam/Documents/Development/Cpp/JXCore/addonStandardExample/build/Release/addon.node
Expected in: flat namespace
Trace/BPT trap: 5
Do you have any idea what might be causing this?
Thanks!
Hi @csapoadam Did you see the #1 ? Your case looks similar. However honestly it doesn't happen for me anymore.
It works for me when I compile the example with the jx executable built for v8, and when I run it using that executable. In that case I don't have to include anything other than node.h.
However, if I use the SM version, I cannot compile it without including v8.h, and then even though it compiles I get these dyld problems.
On the other hand, the native jxcore API (https://github.com/jxcore/jxcore/tree/master/doc/native) seems to work for both. I wonder if this is the normal behavior. If I have to use the native API, that's fine, but I was also having some trouble understanding some of the examples. For example, in this example ( https://github.com/jxcore/jxcore/blob/master/doc/native/Macro_Class_and_Functions.md#js_class_new_instance ) the second argument to INIT_NAMED_CLASS_MEMBERS (MyClassWrap) is unknown, I don't know where it comes from.
@csapoadam 'jx install' problem with SM is finally fixed on master. I don't know why exactly v8 is needed there. Can you share me a sample ? I'm not reproducing this on sample addon.
@obastemur Hi, that's good news if fixed. Regarding the dyld problems: If I compile this on Mac, everything is fine:
JS_LOCAL_METHOD(sumMethod) {
if (args.Length() < 2 || !args.IsInteger(0) ||
!args.IsInteger(1)) {
THROW_EXCEPTION("expected arguments (int, int)");
}
int sum = args.GetInteger(0) + args.GetInteger(1);
JS_LOCAL_VALUE sum_val = STD_TO_INTEGER(sum);
RETURN_PARAM(sum_val);
}
JS_METHOD_END
JS_LOCAL_METHOD(addTextMethod) {
if (args.Length() < 1 || !args.IsString(0)){
THROW_EXCEPTION("expected arguments(string)");
}
jxcore::JXString jxs;
int x = args.GetString(0, &jxs);
//x is the length of the string...
std::cout << "length of string is " << x << std::endl;
JS_LOCAL_VALUE sum_val = STD_TO_STRING(*jxs);
RETURN_PARAM(sum_val);
}
JS_METHOD_END
void init(JS_HANDLE_OBJECT_REF target) {
JS_METHOD_SET(target, "sum", sumMethod);
JS_METHOD_SET(target, "addtext", addTextMethod);
}
... but if I use the syntax provided on the jxcore.com page, i.e. the following code, then it doesn't work for SM:
#include <node.h>
#include <v8.h>
using namespace v8;
Handle Method(const Arguments& args){
HandleScope scope;
return scope.Close(v8::String::New("world"));
}
void init(Handle exports){
exports->Set(v8::String::NewSymbol("hello"),
FunctionTemplate::New(Method)->GetFunction());
}
NODE_MODULE(addon, init)
as shown in the screenshot:
Other than that, I was able to create some useful examples with the earlier syntax, that's working fine now on Mac with SM.
On a different note, I am having some problems on Windows as well. Sorry to mention this here (this might be a long post), but somehow it seems related. I used the binary installer provided on jxcore.com, and the path is set properly. Now, when I try to do jx install in the addon folder, I have the following problem:
It seems that, once again, npm.tar.gz is not unpacked. When I unpack it manually, the installation process proceeds. However, I am having problems yet again with the symbols during the jx install process, that is, during the linking process even before I can get to running the example:
This is for the same, very simple example I posted at the beginning of this comment. Do you have any idea as to what's causing this?
Thanks!
@csapoadam I really cannot reproduce your jx install issue with extracting of npm.tar.gz. You've said:
I used the binary installer provided on jxcore.com
Which engine/architeture? (jx -jsv && jx -a)
PS. I've also tried opening the visual studio solution in the build folder, and building the addon manually. I also get linker errors, like this:
@ktrzeciaknubisa I am using this installer:
Windows Setup (ia32/x64/SM/V8) Download 51fbc39647998740662abb3d2ed9414e56ee6f71
on a 32-bit Windows 7 system, with SpiderMonkey selected.
@ktrzeciaknubisa Here's a screenshot
Thanks @csapoadam Let's move further discussion about jx install issue into jxcore/jxcore#341, to separate things here.