Giters

sachaos / viddy

👀 A modern watch command. Time machine and pager etc.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Spawning a lot of threads

btjanaka opened this issue · comments

commented

Hi @sachaos! Whenever I run viddy, it seems to spawn a lot of threads. For example, I may run

viddy ls

Then, when I check my processes/threads with

ps -u btjanaka -T

I see something like

  32929   32929 pts/11   00:00:00 viddy
  32929   32930 pts/11   00:00:00 viddy
  32929   32931 pts/11   00:00:00 viddy
  32929   32932 pts/11   00:00:00 viddy
  32929   32933 pts/11   00:00:00 viddy
  32929   32935 pts/11   00:00:00 viddy
  32929   32936 pts/11   00:00:00 viddy
  32929   32938 pts/11   00:00:00 viddy
  32929   32939 pts/11   00:00:00 viddy

This is an issue for me because I run viddy on an HPC login node, where there is a very strict process limit and viddy is causing me to run out of processes. Is there any way I can make viddy be single threaded?

Thank you!

commented

Did you try setting $GOMAXPROCS=1 ? (env var)

commented

Hi! I just tried setting GOMAXPROCS=1. I ran

export GOMAXPROCS=1
viddy ls

And also

GOMAXPROCS=1 viddy ls

And got the same result as in my original post for both commands.

commented

Your $ ps output shows only one process but many threads.

Yet you mention a strict process limit? If your number of processes are limited, Viddy can't do much better than one process? Or are your thread amounts limited also?

FWIW, I can observe the same behaviour: if I set GOMAXPROCS=4 I can still see Viddy using 10 threads.

GOMAXPROCS is documented as follows:

The GOMAXPROCS variable limits the number of operating system threads that can execute user-level Go code simultaneously. There is no limit to the number of threads that can be blocked in system calls on behalf of Go code; those do not count against the GOMAXPROCS limit

(the bold sections were emphasized by me). So it looks like this is just Go runtime machinery using those ~9 additional threads - it looks like even if your application itself is single-threaded, the Go runtime still spawns these about nine threads to service the runtime, do garbage collection etc.

commented

Hi @joonas-fi, thank you for your insights! It looks like there is a thread limit on the system I am using. If I am understanding correctly, this seems like something I will just have to live with, so I will close this issue.