Getting z/ip gateway running on Nerves Raspberry Pi 3B+
dvic opened this issue · comments
Hi!
First of all, let me start by thanking you (and SmartRent) for this wonderful library
I'm trying to get the z/ip gateway running on my Raspberry Pi but I'm getting the following errors:
I'm on Mac and I compiled the sdk using zipgateway-env but I had to modify the dockerfile to get it to compile:
I was struggling first to get the source code from 'Simplicity Studio 5' but I eventually found it (for reference: on mac it's in /Applications/Simplicity Studio.app/Contents/Eclipse/developer/sdks/zwave/zip_gateway/v7.16.1/Source.
After the compile step was succesfully completed i took the binary from the zipgateway-env folder zipgateway-source/7.16.01/usr/local/build/src, because I forgot to add the new version incompile.sh:
Then I added the binary to rootfs_overlay/usr/sbin and after burning the firmware to the SD-card, I ran:
iex(10)> RingLogger.attach
:ok
iex(11)> Grizzly.Supervisor.start_link(serial_port: "/dev/ttyS0")
{:ok, #PID<0.1671.0>}
20:36:46.668 [warn] [Grizzly]: Unsolicited server unable to listen
20:36:46.681 [error] /usr/sbin/zipgateway: Process exited with status 2
20:36:46.681 [error] GenServer #PID<0.1675.0> terminating
** (stop) :error_exit_status
Last message: {#Port<0.184>, {:exit_status, 2}}
State: %MuonTrap.Daemon.State{cgroup_path: nil, command: "/usr/sbin/zipgateway", log_output: :debug, log_prefix: "/usr/sbin/zipgateway: ", port: #Port<0.184>}
20:36:46.703 [error] /usr/sbin/zipgateway: Process exited with status 2
20:36:46.704 [error] GenServer #PID<0.1692.0> terminating
** (stop) :error_exit_status
Last message: {#Port<0.186>, {:exit_status, 2}}
State: %MuonTrap.Daemon.State{cgroup_path: nil, command: "/usr/sbin/zipgateway", log_output: :debug, log_prefix: "/usr/sbin/zipgateway: ", port: #Port<0.186>}
It seems that zipgateway is giving an exit 2, any idea where I can start to dig into this further :)? Is this is recommended way to get a properly compile binary for RP3?
@dvic thanks for reaching out! I am glad that you are giving Grizzly a spin!
Admittedly, I haven't updated the zipgateway-env to handle how Silabs has changed stuff over the last year and a half and I should revisit it. So much changed from their end it was hard to keep track of it all. The information you provided definitely helps inform me about the work that is needed, so thank you for providing really good details about what you are seeing.
Unfortunately, the docker container only compiles for 32-bit x86 and not ARM. To target ARM you will need a cross compiler toolchain or you can use the provided .deb package in the Simplicity Studio v5 developer path: developer/sdks/zwave/zip_gateway/v7.16.1/Binaries.
If you are using Raspbian then the .deb package should work zipgateway with your target's cross compiler toolchain.
I can walk you through the steps if you are targeting the Nerves system RPi3. I should probably make a guide anyways, so this will be a good opportunity to do this.
As a side note, earlier this year our Z-Wave hub passed the Z-Wave Alliance certification process using Grizzly! I hope once you're able to get zipgateway gets running, Grizzly will work rock solid for you!
Unfortunately, the docker container only compiles for 32-bit x86 and not ARM. To target ARM you will need a cross compiler toolchain or you can use the provided
.debpackage in the Simplicity Studio v5 developer path:developer/sdks/zwave/zip_gateway/v7.16.1/Binaries.
Ahhh clear!
If you are using Raspbian then the
.debpackage should work🤞 . If you're targeting Nerves you will have to buildzipgatewaywith your target's cross compiler toolchain.I can walk you through the steps if you are targeting the Nerves system RPi3. I should probably make a guide anyways, so this will be a good opportunity to do this.
That would be great! I'm indeed targeting Nerves RPi3. If you could give a rough outline / pointers where to start, I think I can manage from there. When I get it working I can submit a PR outlining the process in detail to get it running on RPi3.
As a side note, earlier this year our Z-Wave hub passed the Z-Wave Alliance certification process using Grizzly! I hope once you're able to get
zipgatewaygets running, Grizzly will work rock solid for you!
Wow, that's awesome! Congratulations
@dvic I haven't forgotten about this, I just was tied up with some other things. I have the notes just about done, hopefully, I can finish them up here shortly and respond to you.
No worries! I'm grateful you are willing to help, I can wait :)
Hey there, just chiming in to say that I'm just getting started with Grizzly too. I got the hello_home repo running on my BeagleBone Black, but now that I'm looking into how to compile the zipgateway binary, I think I ought to sit tight and watch this thread too.
Yes, sorry I have been trying to get notes going but I have run into a snag with the build from the source provided from Silicon Labs. Right now I only have instructions for Linux, so you might have to set up a VM or docker to build.
Attached are the work-in-progress instructions. I run into a LibUSB issue when building that I haven't had time to look into when I run make via the script in the PDF.
I know we apply some patches to zipgateway, so I might have to provide some patch files with the notes.
You can see some of the patches here (https://github.com/smartrent/grizzly/pull/245/files), but I am not sure how valid they are since they targeted an older version of zipgateway.
I really want to get provide a really easy way to compile zipgateway for Nerves devices because I believe that is the most painful part of this project - hopefully
Thanks Matt! This seems like something I can run with and see where I get.
I wonder if it's worth while to reach out to Silicon Labs about this workflow or if you've already been in touch with them?
OK, after a little work, I was able to get past the libusb issues. I created a custom nerves system for which I added libusb and json-c.
Now I have a new error:
You have changed variables that require your cache to be deleted.
Configure will be re-run and you may have to reset some variables.
The following variables have changed:
CMAKE_C_COMPILER= /root/nerves_system_br/o/bbb/host/usr/bin/armv7-nerves-linux-gnueabihf-gcc
CMAKE_CXX_COMPILER= /root/nerves_system_br/o/bbb/host/usr/bin/armv7-nerves-linux-gnueabihf-g++
-- Compiling for host on x86_64 system, loading cmake/i386.cmake toolchain for 32 bit cross compile
-- The C compiler identification is GNU 10.2.0
-- The CXX compiler identification is GNU 10.2.0
-- Check for working C compiler: /root/nerves_system_br/o/bbb/host/usr/bin/armv7-nerves-linux-gnueabihf-gcc
-- Check for working C compiler: /root/nerves_system_br/o/bbb/host/usr/bin/armv7-nerves-linux-gnueabihf-gcc -- broken
CMake Error at /root/nerves_system_br/o/bbb/host/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message):
The C compiler
"/root/nerves_system_br/o/bbb/host/usr/bin/armv7-nerves-linux-gnueabihf-gcc"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /root/shared/zipgateway-7.16.01-Source/usr/local/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make cmTC_eb4d7/fast && /usr/bin/make -f CMakeFiles/cmTC_eb4d7.dir/build.make CMakeFiles/cmTC_eb4d7.dir/build
make[1]: Entering directory '/root/shared/zipgateway-7.16.01-Source/usr/local/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_eb4d7.dir/testCCompiler.c.o
/root/nerves_system_br/o/bbb/host/usr/bin/armv7-nerves-linux-gnueabihf-gcc -m32 -o CMakeFiles/cmTC_eb4d7.dir/testCCompiler.c.o -c /root/shared/zipgateway-7.16.01-Source/usr/local/CMakeFiles/CMakeTmp/testCCompiler.c
armv7-nerves-linux-gnueabihf-gcc: error: unrecognized command-line option '-m32'; did you mean '-mbe32'?
make[1]: *** [CMakeFiles/cmTC_eb4d7.dir/build.make:69: CMakeFiles/cmTC_eb4d7.dir/testCCompiler.c.o] Error 1
make[1]: Leaving directory '/root/shared/zipgateway-7.16.01-Source/usr/local/CMakeFiles/CMakeTmp'
make: *** [Makefile:124: cmTC_eb4d7/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:29 (project)
I think this is where the cross-compilation comes in. However, I'm not really sure how to proceed.
If it helps, here are the two lines added to nerves_defconfig in my custom system:
BR2_PACKAGE_LIBUSB=y
BR2_PACKAGE_JSON_C=y
@mattludwigs Let me know if there's something else you'd recommend trying at this stage. I'll probably keep trying to make progress on this as I have time.
I went ahead and tried applying the patches, but they seemed to fail as well. I think I'm a little out of my element here:
root@8a3b5acd4659:~/shared/grizzly# mix compile
warning: use Mix.Config is deprecated. Use the Config module instead
config/config.exs:1
cd /root/shared/zipgateway-7.16.01-Source/usr/local; \
for patch in $(ls /root/shared/grizzly//patches); do \
patch -p1 < "/root/shared/grizzly//patches/$patch"; \
done
patching file CMakeLists.txt
Hunk #1 FAILED at 2.
Hunk #2 FAILED at 36.
2 out of 2 hunks FAILED -- saving rejects to file CMakeLists.txt.rej
patching file CMakeLists.txt
Hunk #1 FAILED at 129.
1 out of 1 hunk FAILED -- saving rejects to file CMakeLists.txt.rej
patching file libs2/CMakeLists.txt
Hunk #1 FAILED at 48.
1 out of 1 hunk FAILED -- saving rejects to file libs2/CMakeLists.txt.rej
patching file CMakeLists.txt
Hunk #1 FAILED at 148.
1 out of 1 hunk FAILED -- saving rejects to file CMakeLists.txt.rej
patching file CMakeLists.txt
Hunk #1 FAILED at 29.
Hunk #2 FAILED at 121.
Hunk #3 FAILED at 178.
Hunk #4 FAILED at 250.
4 out of 4 hunks FAILED -- saving rejects to file CMakeLists.txt.rej
patching file src/ZWFirmwareUpdate.c
Hunk #1 FAILED at 79.
Hunk #2 FAILED at 126.
2 out of 2 hunks FAILED -- saving rejects to file src/ZWFirmwareUpdate.c.rej
patching file CMakeLists.txt
Hunk #1 FAILED at 62.
Hunk #2 FAILED at 281.
2 out of 2 hunks FAILED -- saving rejects to file CMakeLists.txt.rej
patching file CMakeLists.txt
Hunk #1 FAILED at 2.
Hunk #2 FAILED at 18.
2 out of 2 hunks FAILED -- saving rejects to file CMakeLists.txt.rej
patching file src/command_handler.h
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file src/command_handler.h.rej
make: *** [Makefile:122: /root/shared/zipgateway-7.16.01-Source/usr/local/.patched] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
@jasonmj thanks for looking into that more! I am curious, have you pushed your Nerves system yet, just so I can see and maybe test against it once I can jump back into looking at this?
In regards to the patches, it would probably be best to ignore those as they are for a really old zipgateway version.
If you're following the script that was in the PDF one of the CMake flags disables testing: -DSKIP_TESTING=ON. Maybe double-check to make sure that flag is still being applied? I might have some time to soon jump into this more.
Thank you for testing things out and looking into this, it really helps to get more information. Hopefully, long term there can be an elegant solution to this.
@mattludwigs Thanks for the reply. I have not pushed my nerves system for the beaglebone yet, but I think I may have time to do that this weekend. I also have an rpi 3, so maybe that would be better to try? (My end goal is to run the controller on the BBB if possible).
I did try applying some of the patches manually since the line numbers were wrong on the old patches, but that didn't seem to resolve this error:
"/root/nerves_system_br/o/bbb/host/usr/bin/armv7-nerves-linux-gnueabihf-gcc" is not able to compile a simple test program.
Hopefully, I will have a little more time for testing soon.
Just a quick update to say I've not given up here. Still working through some compilation errors, but making progress (slowly, every now and then).
I have everything in a containerized repo that I think I can share, but since it contains the zipgateway source code, I'm not sure if I should?
Anyhow, I'll keep at it a little more and hopefully report back again soon.
OK, I got it to build and now I have a zipgateway binary. I've pushed my work to a private repo. Next, I'll try to integrate this with the hello-home repo load it onto a BeagleBone Black. I'll let you know how it goes!
Awesome! Thank you for the updates!
I did start reworking https://github.com/mattludwigs/zipgateway-env to be more up to date with the current method of getting zipgateway source code. The challenge I found was needing to pass the source into the container so that I could share the environment.
I really think there is a better way for all this to happen, one of my goals this year is to make compiling zipgateway for Nerves devices less painful. I appreciate your continued work in exploring this. I use Linux as my main and I can build it against Nerves systems but the way I do it is not really cross-platform, convenient, or reproducible - as there is a bit of environment manipulation going on.
Also, this has been a blocker for exploring building something more useful to the greater community using Grizzly.
Well, I took a few more steps forward and ran into some more challenges. Here's a screenshot of hello_home running on my BBB with an error coming from /usr/sbin/zipgateway (via Muontrap):
I think I can get my work (mostly) cleaned up such that the silicon labs source directory is all you have to drop into the project to use it. Trouble is, I had to manually patch up some of the source to get it to actually compile. I personally don't know how you're supposed to write patches, so I just went through editing files and resolved all the snags that were in the way.
Maybe we can plan to check it sometime about this and come up with a plan? I'm out of the office all next week, but could probably fit something in the following week or most anytime after that.
Hey @jasonmj sorry for the delay, I was under the weather for a bit last week. Thank you for the update, I might poke around this week as I have time.
I could do a quick check-in on this if you're still willing.
@mattludwigs sorry to year you were feeling under the weather. Hope you’re enjoying n the mend now.
I’m back from a week off myself and working to catch up. I think I could fit in a chat toward the end of this week or more easily, most times next week. I’m on eastern time and could check in either during the workday or afterwards. Feel free to reach out over email to continue the convo: jason@fullsteamlabs.com.
Oh and! I’d be happy to share you on the private repo I used for compiling the zipgateway. It’s intended to work with the beagle bone black, but I think it would be pretty easy to swap that out for another nerves target.