FR: Mention need for setting runtime paths for libtorch

Question

FR: Mention need for setting runtime paths for libtorch

n8henrie opened this issue a year ago · comments

I ran into some errors due to missing a system install of libtorch at the expected path. I was able to trace these errors back to: https://github.com/LaurentMazare/tch-rs#libtorch-manual-install and the need to set some environment variables like LIBTORCH or LIBTORCH_USE_PYTORCH.

I'm trying to get this working with nix (on aarch64-darwin) but not having luck so far.

Gadersd · Answer 1 · Fri Jul 28 2023 09:28:23 GMT+0800 (China Standard Time)

Torch is such a painful dependency to work with so I switched the backend from tch to wgpu. Try and see if it works for you now.

Nathan Henrie · Answer 2 · Sat Jul 29 2023 21:55:31 GMT+0800 (China Standard Time)

I'd like to get torch working if I can. Looks like there is some issues with tch and MPS support though: LaurentMazare/tch-rs#777

Dilshod Tadjibaev · Answer 3 · Sat Jul 29 2023 22:24:11 GMT+0800 (China Standard Time)

@n8henrie, I am the burn team and I worked on getting torch working on Mac.

Normally, tch-rs will download the library and save it under the target directory. tch-rs gives a few option to reference libtorch, for burn by default we picked download.

I would recommend you remove all environment variables LIBTORCH and LIBTORCH_USE_PYTORCH first. This will use the downloaded version. Make sure you do not have this in your .cargo/config.toml as well. Afterwards, you can move forward using the system installed version if you'd like

If you're still having trouble, you can ping the team on Discord or file an issue ticket with Burn. We will prioritize it.

Nathan Henrie · Answer 4 · Sat Aug 05 2023 01:48:19 GMT+0800 (China Standard Time)

Thanks for the suggestion. Clearing the env and rebuilding seems to work if I run as per the example:

$ cargo run --release audio.wav tiny_en
...
Transcribed text: <|0.00|> Hello, I am the whisper machine learning model. If you see this as text then I am working properly.

but curiously the binary fails if I try to run directly:

$ target/release/whisper audio.wav tiny_en
dyld[47393]: Library not loaded: @rpath/libtorch_cpu.dylib
  Referenced from: <4C4C4450-5555-3144-A167-13D0CD0B6346> /Users/n8henrie/git/whisper-burn/target/release/whisper
  Reason: tried: '/usr/local/lib/libtorch_cpu.dylib' (no such file), '/usr/lib/libtorch_cpu.dylib' (no such file, not in dyld cache)
Abort trap: 6

Should this be a new / separate issue? Seems like it's still on the same theme of finding the libtorch bindings.

Dilshod Tadjibaev · Answer 5 · Sat Aug 05 2023 01:54:01 GMT+0800 (China Standard Time)

huh that's interesting bug. Let me try to replicate it and we probably file a ticket with tch-rs team if it's an upstream problem.

Dilshod Tadjibaev · Answer 6 · Sat Aug 05 2023 02:13:27 GMT+0800 (China Standard Time)

What I will do is to file a ticket with Burn and fix the situation with Torch dependency. We should have good documents to troubleshoot and correct tools. It's far to often people struggle (including myself with Mac installation in the past). So expect a ticket link here and we will collect issue there, including running the binary as standalone (did not test myself yet) like @n8henrie found out.

Dilshod Tadjibaev · Answer 7 · Sat Aug 05 2023 04:15:09 GMT+0800 (China Standard Time)

I have created an issue ticket with Burn: tracel-ai/burn#587 so that we can improve Libtorch set up.

Nathan Henrie · Answer 8 · Sat Aug 05 2023 04:42:37 GMT+0800 (China Standard Time)

Thanks for your attention to the matter!