Run dcurl on armhf based Linux distribution for DE10-Nano Kit

Question

Run dcurl on armhf based Linux distribution for DE10-Nano Kit

marktwtn opened this issue 5 years ago · comments

Base on the discussion of #174 (comment),
the old image file from LampaLab/iota_fpga should not be used since it is out-of-date and does not support armhf.

On de10-nano board, the dcurl should be run with the compiler flag -mfloat-abi=hard to avoid performance impact and hence we need an armhf image file.
An armhf image is offer by @tylerw1369 in this github repository.
After the dcurl is successfully running with the new image file, we should merge the pull request #174 and change the image file of the other de10-nano boards as well.

marktwtn · Answer 1 · Sat Aug 03 2019 06:45:17 GMT+0800 (China Standard Time)

I cross compiled the dcurl and transfer the executables to the de10-nano board.
They ran well, hence the modification of pull request #174 can be merged if we use the armhf image file.

However, there are something I need to mention:

The time for burning the image into the micro SD card is quite long. Is it reasonable?
I can not use apt command to install the tool like GNU Make. It shows the message like

Package make is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'make' has no installation candidate

For some reason, the FPGA-related executables can not be executed unless we use sudo.
Output message:

cpow-ctrl open fail: Permission denied

I have not investigated these weird behaviour yet.
I wrote it down for reminding that there are somethings we need to overcome if we are going to use the armhf image file.

Jim Huang · Answer 2 · Sat Aug 03 2019 09:16:17 GMT+0800 (China Standard Time)

@tylerw1369, can you comment for the customized image?

Jim Huang · Answer 3 · Sat Aug 03 2019 09:50:14 GMT+0800 (China Standard Time)

By the way, there is another armhf based Debian GNU/Linux image for de10-nano: https://github.com/ikwzm/FPGA-SoC-Linux/blob/master/doc/install/de10-nano.md

Jim Huang · Answer 4 · Sat Aug 03 2019 23:46:54 GMT+0800 (China Standard Time)

I followed the instructions of meta-de10-nano to build Angstrom-v2018.06 derived minimal image.

Tested on node0:

$ ls /tmp/de10-nano-build/deploy/glibc/images/de10-nano
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.cpio
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.ext3
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.manifest
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.tar.gz
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.tar.xz
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.wic
...

rootfs compressed with gzip is of size 191MB.

tylerw1369 · Answer 5 · Sun Aug 04 2019 23:40:34 GMT+0800 (China Standard Time)

Ya I believe the difference is logging in as root on lampas image file vs a usr with sudo privileges on mine. I am going to try and make an image with a standard root login and that should take care of having to always sudo. I'll get on it and let you know when finished.

tylerw1369 · Answer 6 · Mon Aug 05 2019 00:58:40 GMT+0800 (China Standard Time)

I'm also going to see if I can get the kernel mod to load automatically at boot as well

marktwtn · Answer 7 · Thu Aug 08 2019 16:37:35 GMT+0800 (China Standard Time)

Ya I believe the difference is logging in as root on lampas image file vs a usr with sudo privileges on mine. I am going to try and make an image with a standard root login and that should take care of having to always sudo. I'll get on it and let you know when finished.

@tylerw1369 I think maybe we should change the file privilege of /dev/cpow-ctrl instead of logging as root.

I can not use apt command to install the tool like GNU Make. It shows the message like

Package make is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'make' has no installation candidate

I forgot to use $ sudo apt update first, my bad.
Now it has no problem at all.

Jim Huang · Answer 8 · Thu Aug 08 2019 21:39:22 GMT+0800 (China Standard Time)

Once @marktwtn repackages armhf-based Linux image file (into node0 or similar servers for hosting), we can close this issue first and then think about further improvements and FPGA-specific CI plans.

marktwtn · Answer 9 · Fri Aug 09 2019 18:13:52 GMT+0800 (China Standard Time)

I have tried three different image files and only the one from @tylerw1369 can work.

tyler1369 repository
However, the image file is around 8.6 GB, which is too large for the 8GB SD card.
FPGA SoC Linux
The steps is unusual. It makes the kernel image as a debian package and uses it later.
However, I can not even login into the de10-nano board to do the further installation.
I powered up the board and it did not show anything.
Shouldn't the kernel be installed before trying to login?

meta de10 nano
The repository is forked from Intel.
Somehow I had build failure on my desktop due to the compiler segmentation fault.
Hence I decided to use the image file previously built by @jserv on node0.
And de10 nano board keeps rebooting and print the message like

[    9.923416] of_fixed_clk: probe of soc:base-fpga-region:clk_0 failed with error -5
[    9.931347] of_fixed_clk: probe of soc:base-fpga-region:fft_sub_clk_0 failed with error -5
[    9.940037] of_fixed_clk: probe of soc:base-fpga-region:pll_stream failed with error -5
[    9.948459] of_fixed_clk: probe of soc:base-fpga-region:custom_reset_synchronizer failed with error -5

I have not tried the method offered by Linux on ARM DE10-Nano Kit.
I'll give it a try.

tylerw1369 · Answer 10 · Sat Aug 10 2019 00:56:58 GMT+0800 (China Standard Time)

I will lower the size of my image file to suite your needs. It will be easy and then you can fit it on your cards. It doesnt need to be that big in the first place as i only am running dcurl on it as a pow node. Im at work today but ill get it done tomorrow when I am off.

tylerw1369 · Answer 11 · Sun Aug 11 2019 07:51:44 GMT+0800 (China Standard Time)

Here is the smaller image file. Its 2.1 gigs.
https://drive.google.com/open?id=11QwliFc9okIU7HgwvQam2UTkUfVfClPe
Let me know if you have issues with it.
@marktwtn
Also im sure changing the file permissions for the /dev/cpow-ctrl would work. I will have to check it out.

Jim Huang · Answer 12 · Sun Aug 11 2019 09:48:49 GMT+0800 (China Standard Time)

Here is the smaller image file. Its 2.1 gigs.
https://drive.google.com/open?id=11QwliFc9okIU7HgwvQam2UTkUfVfClPe
Let me know if you have issues with it.

Thank @tylerw1369 for this work. Is it based on Ubuntu Minimal?

tylerw1369 · Answer 13 · Sun Aug 11 2019 11:03:30 GMT+0800 (China Standard Time)

It's been a while since I made it but yes if I remember correctly it was based off ubuntu minimal. It pretty much has everything to run a dcurl pow node via node js and your dcurl software and nothing more. I didn't add anything to it to bog it down. I believe it runs 4.14.73 ltsi for the kernel. Also going with armhf really helped me get rid of the errors and problems I had with the image from lampa. It runs 100 percent better.

marktwtn · Answer 14 · Sun Aug 11 2019 22:35:43 GMT+0800 (China Standard Time)

@tylerw1369 the partition seems weird.
The whole root file system is allocated with 1.6 GB and the available space has only 307 MB remained.
And there is a 5.9 GB space which is not partitioned at all.

I ran the dcurl testing files with the cross compiler on de10-nano, and everything went well.
However, it might have problem when we try to use remote worker since RabbitMQ uses OpenSSL.
Native compiler: Normal Linux distribution comes with OpenSSL and Ubuntu Minimal does not. It needs to be installed.
Cross compiler: It prints out the message like

 In file included from .../dcurl/deps/rabbitmq-c/librabbitmq/amqp_openssl.c:31:0:
.../dcurl/deps/rabbitmq-c/librabbitmq/amqp_openssl_bio.h:26:10:
fatal error： openssl/bio.h：No such file or directory
 #include <openssl/bio.h>
          ^~~~~~~~~~~~~~~

I have not figured out how to solve the problem.

tylerw1369 · Answer 15 · Mon Aug 12 2019 01:43:16 GMT+0800 (China Standard Time)

@marktwtn Ok I hope it works for you this time. This image should be perfect to work for an 8gb card.
https://opacity.io/share#handle=8d2d66d77bde24f15adfb565168706522e5febec91e64d20ab49bc54c056f85b53d03adc0207a9965140fb883b0fc0b73846593e920d8721f671c97490962627

tylerw1369 · Answer 16 · Mon Aug 12 2019 04:52:24 GMT+0800 (China Standard Time)

@marktwtn Also I got dcurl to install and compile correctly even with the BUILD_REMOTE=1 on my image file. I cloned openssl github and compiled it on my nano then ran dcurl and it finished making with no errors.
[100%] Built target test_sasl_mechanism
make[2]: Leaving directory '/home/tyler/dcurl-update/dcurl/deps/rabbitmq-c/build'
make[1]: Leaving directory '/home/tyler/dcurl-update/dcurl/deps/rabbitmq-c/build'
CC build/curl.o
CC build/constants.o
CC build/trinary.o
CC build/dcurl.o
CC build/implcontext.o
CC build/common.o
CC build/pow_fpga_accel.o
CC build/remote_common.o
CC build/remote_interface.o
CC build/test-trinary.o
LD build/test-trinary
CC build/test-curl.o
LD build/test-curl
CC build/test-dcurl.o
LD build/test-dcurl
CC build/test-pow.o
LD build/test-pow
CC build/test-multi-pow.o
LD build/test-multi-pow
LD build/libdcurl.so
CC build/remote_worker.o
CC build/worker-curl.o
CC build/worker-constants.o
CC build/worker-trinary.o
CC build/worker-dcurl.o
CC build/worker-implcontext.o
CC build/worker-common.o
CC build/worker-pow_fpga_accel.o
CC build/worker-remote_common.o
LD build/remote-worker
rm build/test-trinary.o build/test-curl.o build/test-pow.o build/test-multi-pow.o build/test-dcurl.o
tyler@arm:~/dcurl-update/dcurl$

I havent followed to closely the remote worker part so Im not sure how I would test it to see if it indeed functions correctly.

marktwtn · Answer 17 · Mon Aug 12 2019 13:08:00 GMT+0800 (China Standard Time)

@marktwtn Also I got dcurl to install and compile correctly even with the BUILD_REMOTE=1 on my image file. I cloned openssl github and compiled it on my nano then ran dcurl and it finished making with no errors.

Thanks for the testing. I'll keep checking the functionality of the remote worker.

I tried the new image and I can not boot correctly.
It kept showing the message like:

spl: fat register err - -1
### ERROR ### Please RESET the board ###

Did boot loader have something wrong?

I burned the image with the command
$ sudo dd if=./XXX.img of=/dev/sdX bs=1M

tylerw1369 · Answer 18 · Mon Aug 12 2019 21:55:54 GMT+0800 (China Standard Time)

Odd I'll check when I get home from work and see if something is wrong with the image

tylerw1369 · Answer 19 · Wed Aug 14 2019 01:33:25 GMT+0800 (China Standard Time)

Ok I re-imaged the file. Try this
https://opacity.io/share#handle=a2da86d7c72ac6496ae454601365cf1442d568b3daa78a3eaa9ba8637a2b4222ebc3d62468a9ff9fd72c27371f6fbdec0a7d0eac3d4753f7277fff6ffb52ffa5

marktwtn · Answer 20 · Wed Aug 14 2019 02:24:12 GMT+0800 (China Standard Time)

Ok I re-imaged the file. Try this
https://opacity.io/share#handle=a2da86d7c72ac6496ae454601365cf1442d568b3daa78a3eaa9ba8637a2b4222ebc3d62468a9ff9fd72c27371f6fbdec0a7d0eac3d4753f7277fff6ffb52ffa5

It still has the same error.
Does this image file work well on your SD card?
If so, I will find another SD card to give it a try.

tylerw1369 · Answer 21 · Wed Aug 14 2019 02:45:15 GMT+0800 (China Standard Time)

yes i can boot from it on my sd card. let me try uploading it via google
https://drive.google.com/open?id=1MFkvts_zdfZuN8N_HnOimkK0NSsayABG

tylerw1369 · Answer 22 · Wed Aug 14 2019 03:11:49 GMT+0800 (China Standard Time)

Its not your card for some reason the image file wont load please hold

tylerw1369 · Answer 23 · Wed Aug 14 2019 03:54:54 GMT+0800 (China Standard Time)

@marktwtn Ok for some reason i cant get partition 1 to copy correctly. Download this partition and restore this image to partition 1 and it will work. I dont know why it wont work work with the 8gb image file but this should fix it so it boots correctly.
https://drive.google.com/open?id=12yGYPtH6dotN1z9f56BGcO3CdISQuPjU

marktwtn · Answer 24 · Wed Aug 14 2019 10:57:35 GMT+0800 (China Standard Time)

@marktwtn Ok for some reason i cant get partition 1 to copy correctly. Download this partition and restore this image to partition 1 and it will work. I dont know why it wont work work with the 8gb image file but this should fix it so it boots correctly.
https://drive.google.com/open?id=12yGYPtH6dotN1z9f56BGcO3CdISQuPjU

@tylerw1369 it works now. Thanks~

marktwtn · Answer 25 · Thu Aug 15 2019 21:06:49 GMT+0800 (China Standard Time)

The dcurl, including the remote worker works well on the armhf based Linux distribution.

Thanks for the help of @tylerw1369 again.
By the way, you can use the command $ sudo apt-get install libssl-dev to get the header files and libraries of OpenSSL instead of building them from the source code.

Once we decide what software to be included in the Linux distribution, I will create another image file for deploying.