Question about boda install
chaeseok opened this issue · comments
Hello Matthew Moskewicz,
I'm going to test boda framework.
Can I ask a question about boda install?
I got boda from https://github.com/moskewcz/boda/,
and caffe from https://github.com/BVLC/caffe/.
When I compile the boda framework, I got an error as below:
dependencies.make:30: *** "building with caffe support, but diff of boda copy of caffe.proto with caffe version failed (because the content diffed, a file was not found, etc). run diff manually for details and/or update boda version of caffe.proto. aborting make.". Stop.
If two caffe.proto files are different, how can I solve it?
Or can I get git revisions both files are same?
Regards,
Chaeseok Im
first of all: good luck! since this is mostly a single-developer, single-machine, limited-documentation, under-constant-development grad-student project, you should expect various setup/portability issues.
for this issue, in general, you have three options:
- disable caffe support
- use a version of caffe where the files match
- use any version of caffe you like (older or newer than the version boda expects), copy its caffe.proto over the boda one, and fix any errors. generally it would only make sense to do this if you're updating boda to support a newer caffe.
you probably want to start with trying (1). you may not need caffe support at all, and this is easiest. even if you do need caffe support, it may be easier to work though any other build issues first before enabling caffe. of course there can be build issues specific to disabling things ...
for initial build testing, you can disable octave, SDL2, and caffe support by adding 'disable' to their build stanza headers in obj_list:
diff --git a/obj_list b/obj_list
index fa49ee0..5f3d8bc 100644
--- a/obj_list
+++ b/obj_list
@@ -15,11 +15,11 @@
# assume system defaults include needed include/lib dirs
LDFLAGS := $(LDFLAGS) -llmdb
-[octave]
+[octave disable]
CPPFLAGS := $(CPPFLAGS) -I/usr/include/octave-4.0.0 -I/usr/include/octave-4.0.0/octave
LDFLAGS := $(LDFLAGS) -loctave -loctinterp
-[SDL2]
+[SDL2 disable]
CPPFLAGS := $(CPPFLAGS) -I/usr/include/SDL2 -I/usr/local/include/SDL2
LDFLAGS := $(LDFLAGS) -lSDL2 -lSDL2_ttf
@@ -44,7 +44,7 @@ LDFLAGS := $(LDFLAGS) -lnvrtc
[opencl]
LDFLAGS := $(LDFLAGS) -lOpenCL
-[caffe needs=cuda needs=caffe_pb]
+[caffe needs=cuda needs=caffe_pb disable]
CAFFE_HOME := /home/moskewcz/git_work/caffe_dev
CPPFLAGS := $(CPPFLAGS) -I$(CAFFE_HOME)/include -I$(CAFFE_HOME)/build/src
LDFLAGS := $(LDFLAGS) -L$(CAFFE_HOME)/build/lib -lcaffe
after that, if you do want caffe support (i.e. for running the caffe backend for comparison/regressing testing), you'll probably want to do (2) eventually.
if you don't know that you want to do (3), you probably shouldn't do it. that's more something for me to do as-needed or every so often to track caffe progress (i.e. every ~6 months).
the caffe support in the current boda head was recently updated to caffe head, but not much has changed in caffe that boda cares about in the last ~6 months or so.
for boda head, see this commit message for the current supported caffe version / branch:
d06d6ad
for boda versions == or before this tag, see the tag message for the corresponding caffe branch/commit:
https://github.com/moskewcz/boda/releases/tag/boda-ON-fix-sign-comp-in-hpps-ON-caffe-2015-may-which-still-supports-cudnnv2
as noted, branches including my patches to caffe (which aren't strictly needed) are all in my github (forked) caffe repo.
I got an another error as below:
../src/boda_base.H: In instantiation of ‘typename T::mapped_type& boda::must_find(T&, const typename T::key_type&) [with T = std::map<std::basic_string<char>, boda::ipc_var_info_t>; typename T::mapped_type = boda::ipc_var_info_t; typename T::key_type = std::basic_string<char>]’:
../src/rtc_ipc.cc:153:82: required from here
../src/boda_base.H:101:58: error: ‘str’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
if( i == kvm.end() ) { rt_err( "missing key:" + str(k) ); }
^
In file included from ../src/rtc_ipc.cc:3:0:
../src/str_util.H:68:45: note: ‘template<class T> std::string boda::str(const T&)’ declared here, later in the translation unit
template< typename T > inline std::string str(T const & i) { // convert T i to string
^
In file included from ../src/boda_tu_base.H:7:0,
from ../src/rtc_ipc.cc:2:
../src/boda_base.H:101:60: error: ‘rt_err’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
if( i == kvm.end() ) { rt_err( "missing key:" + str(k) ); }
^
../src/boda_base.H:109:8: note: ‘void boda::rt_err(const string&)’ declared here, later in the translation unit
void rt_err( string const & err_msg ) __attribute__((noreturn));
^
looks like a typical c++ portability error; i'm guessing it's from a newer compiler than what i'm using. it looks semi-legitimate (as c++ portability errors go). as you might expect, i don't see this in HEAD on my normal build platform:
moskewcz@maaya:~/git_work/boda$ uname -a
Linux maaya 3.19.2-031902-generic #201503181436 SMP Wed Mar 18 14:37:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
moskewcz@maaya:~/git_work/boda$ g++ --version
g++ (Ubuntu/Linaro 4.6.4-1ubuntu1~12.04) 4.6.4
i also compile on ubuntu 14.04 / g++ 4.8 sometimes, i can try that and see if i get the error there. i'll follow up here if/when i do that.
otherwise, for such errors, you've got ~2 top level options:
- use a 'supported' compiler (i.e. match what i'm using)
- use any compiler you like and fix the errors -- and ideally submit a pull req. with details and your fixes
now, i'm interested in making boda more portable, so even if i don't see this error on g++ 4.8, i can make some efforts to try a yet-newer g++ and/or some other compilers, pending my time/ability to get access to them. officially, i think it's fair to expect boda to compile on current-ish gcc/clang in c++11/14 modes (c++ 98/03 should 'almost' work, but i do use a few c++11 features for convenience). but on the other hand c++ portability across compilers / standard versions isn't the most fun thing to ensure (or my highest current priority).
at a minimum, though, it'd be nice to know what compiler/setup you're using. depending on exactly what the platform is, i might need some help getting access to it. or, i might be able to work that out on my own, again pending my time/interest/etc. i think maybe github (or some github ecosystem company) has compile farms or something to help with such portability issues?
now, if you've already fixed / worked around this issue and moved on: great! again, any info on how you fixed it and/or pull req. with proposed fixes are always appreciated/welcome. but, on the other hand, if you stop and wait for a ~day at each issue (especially 'simple' ones like this), it may take a long time (weeks/months/forever) to get a working build. you may want to seek out local c++/porting/programming resources if you can't tackle issues like this yourself. it's not ideal of course, but, realistically, bearing in mind the maturity of this code and the current user base, you're probably going to need to get your hands dirty (at all levels of the code) if you want good testing results.
good news! i see the same error on g++ 4.8 -- now i just need to fix it.
moskewcz@a18:~/nwork/boda/obj$ uname -a
Linux a18 3.16.0-34-generic #47~14.04.1-Ubuntu SMP Fri Apr 10 17:49:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
moskewcz@a18:~/nwork/boda/obj$ g++ --version
g++ (Ubuntu 4.8.4-2ubuntu1~14.04) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
moskewcz@a18:~/nwork/boda/obj$ make
py_prebuild_rm_okay:
py_prebuild_hook: wrappers up to date.
makefile:55: rtc_ipc.d: No such file or directory
makefile:55: rtc_fwd.d: No such file or directory
makefile:55: rtc_compute.d: No such file or directory
makefile:55: ocl_util.d: No such file or directory
rm -f build_info.o
rm -f str_util.o
rm -f lexp.o
rm -f nesi.o
rm -f test_nesi.o
rm -f conv_util.o
rm -f caffeif.o
rm -f asio_util.o
rm -f test_dense.o
rm -f caffepb.o
rm -f upgrade_proto.o
rm -f lmdb_caffe_io.o
rm -f nvrtc_util.o
rm -f slow_cpu_fwd.o
rm -f caffe_fwd.o
rm -f ocl_util.o
rm -f rtc_compute.o
rm -f rtc_fwd.o
rm -f rtc_ipc.o
g++ -Wall -O3 -g -std=c++0x -rdynamic -fPIC -fopenmp -Wall -I/usr/include/python2.7 -I/usr/local/cuda/include -I/nscratch/moskewcz/git_work/caffe/include -I/nscratch/moskewcz/git_work/caffe/build/src -MMD -c ../src/rtc_ipc.cc
In file included from ../src/boda_tu_base.H:7:0,
from ../src/rtc_ipc.cc:2:
../src/boda_base.H: In instantiation of ‘typename T::mapped_type& boda::must_find(T&, const typename T::key_type&) [with T = std::map<std::basic_string<char>, boda::ipc_var_info_t>; typename T::mapped_type = boda::ipc_var_info_t; typename T::key_type = std::basic_string<char>]’:
../src/rtc_ipc.cc:153:82: required from here
../src/boda_base.H:101:58: error: ‘str’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
if( i == kvm.end() ) { rt_err( "missing key:" + str(k) ); }
^
In file included from ../src/rtc_ipc.cc:3:0:
../src/str_util.H:68:45: note: ‘template<class T> std::string boda::str(const T&)’ declared here, later in the translation unit
template< typename T > inline std::string str(T const & i) { // convert T i to string
^
In file included from ../src/boda_tu_base.H:7:0,
from ../src/rtc_ipc.cc:2:
../src/boda_base.H:101:60: error: ‘rt_err’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
if( i == kvm.end() ) { rt_err( "missing key:" + str(k) ); }
^
In file included from ../src/boda_tu_base.H:7:0,
from ../src/rtc_ipc.cc:2:
../src/boda_base.H:115:8: note: ‘void boda::rt_err(const string&)’ declared here, later in the translation unit
void rt_err( string const & err_msg ) __attribute__((noreturn));
^
../src/boda_base.H:261:18: warning: ‘boda::boda_magic’ defined but not used [-Wunused-variable]
uint32_t const boda_magic= *(uint32_t *)"BODA";
^
make: *** [rtc_ipc.o] Error 1
moskewcz@a18:~/nwork/boda/obj$
okay, i commited and pushed a lightly-tested fix to master. the errors seemed readable/reasonable enough.
commit: 8c36aab
When SDL2 enabled, I got an error below:
/usr/bin/ld: caffe_fwd.o: undefined reference to symbol '_ZN6google4base21CheckOpMessageBuilderD1Ev'
//usr/lib/x86_64-linux-gnu/libglog.so.0: error adding symbols: DSO missing from command line
And also when caffe enabled, I got an error below:
/usr/bin/ld: cap_app.o: undefined reference to symbol 'shm_unlink@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/librt.so.1: error adding symbols: DSO missing from command line
i think maybe you listed those errors backwards?
anyway, for the libglog error, it seems necessary to explicitly link against -lglog (under g++ 4.8 / ubuntu 14.04 anyway) when linking against caffe. like the error says: DSO missing from command line, so i added it to the command line. committed and pushed to master:
for the SDL2 error: again, it says DSO is missing from the command line ... so i added it to the command line. committed and pushed to master:
however, i don't have physical access to the Ubuntu 14.04 machine i built on, so i didn't try to test full SDL2/v4l2/camera support -- and as a bonus, it's not well tested by the automated tests in general, so it's more likely to be broken at any given random time than the 'core' parts of the code. however, at least the "boda display_test" mode seemed to work in my x2go desktop on that machine.
again i reiterate that you probably need to both be able to fix issues like this yourself and/or give me more information about your build setup if you hope to make progress on an evaluation in any reasonable time. i estimate any reasonably experienced c++ programmer could have fixed all the above build issues in ~1 hour total but it has taken ~1 week to get this far with the latency between updates.
of course, i don't actually know what you're doing on your end or if you're actually getting stuck on any of these issues, so maybe all is well. in short if it's okay with you it's okay with me, and these are all good things to fix.
mwm
I add -lglog in optional command line; add boost_system.so library.