Document concurrency
dimaqq opened this issue · comments
I'm curious what model is used to bridge Python and golang concurrency models.
I wish it was documented :)
Python has threads (1:1 mapped to lightweight system processes) and asyncio, explicit and quite so different.
Go has special handling for cpu-bound and io-bound goroutines, transparent to the user.
How would someone use gopy to interface go's amazing net/http module or some go package that wraps it? As in, how should Python code look like to benefit from go's concurrency?
I'm not really sure about the python side but anything that is fully contained and managed on the Go side should function as usual -- so if you have a single-thread python "manager" that is calling into Go code that does a bunch of stuff in parallel goroutines, that should just work. We do this in our neural network software and it is fine. I ran into some issues trying to navigate python threads in relation to GUI threads, and ended up turning the whole thing into a separate exe as a result to bypass those issues (hence the exe mode for gopy) -- starting from python and trying to get python off onto another thread so the Go gui could run in the main thread didn't work (for me at least).
The basic handle code that interfaces between python and Go is protected by a mutex, so it should be robust to anything.
I'd say, give it a try and document what you discover!
I find this answer unconvincing.
How do you handle cooperative multitasking with both Python tasks and goroutines? How do you coordinate synchronization primitives?
I suspect a practical solution will also depend on #67.
I think that a more immediate solution could be to use exe mode, and have the go and python sides exchange either POSIX pipes (as os.pipe) or a POSIX socket (as socket.socketpair) for the two sides to communicate.
(Or use pkg mode and put go on the secondary OS thread.)