philss / rustler_precompiled

Use precompiled NIFs from trusted sources in your Elixir code

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Include riscv64gc-unknown-linux-musl target as default

fhunleth opened this issue · comments

64-bit RISC-V is becoming more common now and Nerves supports it on the MangoPi MQ-Pro. This board is not that hard to buy and similar boards are being sold. Supporting a RISC-V target allows Bumblebee to be built since it has a dependency on tokenizers which uses rustler_precompiled. There might be other libraries, but this was the first one that I ran into.

I saw that #36 hadn't been merged yet. Aarch64/Musl seems like it would be more common than RISC-V, so I was wondering if there was a drawback to adding more default configs. If not, then I have a PR ready to go to add it once #43 gets merged.

Hi @fhunleth 👋

I let that PR open - the one adding Aarch64/Musl - because I want to add that to the next version, 0.6.

I'm really not sure if I want to add more default targets now, because that means that package maintainers would need to build for those targets. I would rather wait a little bit more to see the popularity of that target grows before adding that as default. WDYT?

cc @josevalim (do you have opinions about this?)

Yeah, we probably want to keep the environments limited but extensible if the library author desires. For production targets/builds in particular there is an argument building from scratch may be preferable for performance/security reasons.

Ok.

Building from scratch with Rustler doesn't work with Nerves. I'm not sure what's going on, but it prevents building from scratch as a work around. For better or worse, Rustler Precompiled made it possible to use Rust easily with Nerves. I'll try to dig into what's happening and submit a bug report to Rustler so that Nerves users can get past this manually.

Building from scratch with Rustler doesn't work with Nerves.

Ok, this is enough reason to go ahead with this to me. Therefore the next question is: how hard it is for maintainers to build a precompiled version for this architecture? Is it an additional build in the matrix or there are other complications?

@fhunleth perhaps a next step is to explicitly enable this target in tokenizers, we give it a try, and if it is all good we can generalize it?

@josevalim Sounds like a good idea to me. I also need to verify that the produced binary actually runs on a 64-bit RISC-V board. I'll send a PR to tokenizers soon. My sincere hope is that this turns out to be super easy to maintain.

@fhunleth I'm trying to build for that target - riscv64gc-unknown-linux-musl - but seems that cross, which is the tool that we use for cross compilation does not have an image for that.

Do you know if we could build for the GNU ABI? It seems to be more supported today. That target is: riscv64gc-unknown-linux-gnu.

I was afraid that we might find a musl issue.

This all comes from a multi-arch path issue with RISC-V and GNU when I ported Nerves to it. That was earlier this year and there wasn't a way that I could find with gcc or Buildroot to work around it. Musl libc didn't have the issue. This is the second time that glibc vs. musl libc has come up recently, though.

Let me see if the multi-arch issue still exists, since there's really no reason that the Nerves RISC-V port "has" to use musl libc. This will actually take me some time since the Nerves toolchains take a hours to build. I'll get back to you.

I'm going to re-open the issue since we are investigating this.

Quick update. This turned into a small yak shave due to a recent drop of RISC-V Linux updates. I'll release a new Nerves RISC-V glibc toolchain and then I'll be in a place to actually test this. I think switching to riscv64gc-unknown-linux-gnu will work. Just need to get the pieces in place.