Reduce parallelism
techhazard opened this issue · comments
I'm trying to run nix-index on my raspberry pi 4B (aarch64), but after a bit my load-average is 20+ and this effectively locks-up the device (even when run with nice). I can issue no commands over SSH and have to reset the power supply of the device.
Is there some way I can limit the amount of thread/jobs that are being run? I tried running it with nice -n19 nix-index --requests 4 but that still locked-up the pi (for a while there were 4 threads, but after the http requests were done it got too much).
version: nix-index 0.1.5 on NixOS 23.05
+1 -- also locks up my Pi 3b, and my Acer Chromebook, both of which are now happily running NixOS.
EDIT: Looks like the -r flag should do this. I tried in the past without success but will try again and report back.
EDIT2: Should have read more closely; OP already mentioned the --requests flag 🤦♂️. Didn't work for me either.
Not just raspberries , my Laptop froze for a couple of minutes when indexing
Would be great if we could something about this
Edit: My issue was RAM not processing power , maybe i should open a new issue
nix-index uses rayon for parallelism
After reading https://github.com/rayon-rs/rayon/blob/master/FAQ.md#how-many-threads-will-rayon-spawn I thought perhaps this would work:
$ RAYON_NUM_THREADS=1 nix-indexUnfortunately on my Pi4 (4gb ram) I eventually get:
$ RAYON_NUM_THREADS=2 result/bin/nix-index
+ querying available packages
error: querying available packages failed
caused by: nix-env failed with error: nix-env failed with signal: 9 (SIGKILL):
And on my Pi3, even as nicely as I can, I eventually get a full system lockup:
$ RAYON_NUM_THREADS=1 nice -n 19 nix-index --requests 1