Should there be a spec for starting/stopping a kernel
jdfreder opened this issue · comments
On a phone conversation with @rgbkrk , I explored the idea of using RxJS streams to manage kernel life cycles. There are a few designs that can be used to achieve this, and I'm unsure of which, if any, are actually good. In a non-reactive environment, I could see this type of API being problematic.
Generally, I was thinking about a higher level enchannel spawning utility where you pass in the kernel name and your subscribe
import { Spawner } from 'enchannel-spawn'; # Made up name, not intended to be real library
const spawner = new Spawner();
// spawner is an RxJS Subject, we can subscribe for onNext, onError, onCompleted
spawner.subscribe((channels) => {
// Provides the channels from the enchannel spec
}, (err) => {
// Error launching a kernel - show error to user
}, () => {
// user should know that the kernel service is gone
});
spawner.next('python2');
// launches new python2 kernel, closing any old kernel, possibly deleting files if zmq backed?
spawner.complete();
// this is a TODONow that I've written that up, it's not what we need and spawnteract already provides this. Let me have another go at this.
What we really want is a nice wrapper that lets us create a kernel and close it. In zmq land, this is an additional function to enchannel (maybe in the same package) that would handle shutting down the kernel process, severing the connection, and deleting the connection file.
The reason we don't want enchannel to do all cleanup in the default functions for a backend is so that we don't close the running kernel for another frontend. In the case of the notebook, since we start the kernels we always close them down too.
I have a UI grid where each cell is a kernel. I can click on arbitrary cells to spawn kernels.
With rxjs this would be implemented like so:
cell.onclick = () => {
spawner.next('python2').once().subscribe(channels => {
// Yay channels
});
}Where Promises look like this:
cell.onclick = () => {
spawner.spawn('python2').then(channels => {
// Yay channels
});
}How is cleanup handled? When all channels are unsubscribed/closed?
That's not how it would look with RxJS. The next operates on a subscriber / observer, not on an observable.
cell.onclick = () => spawner.next('python2')
The subscription happens elsewhere, mapped into the state of the app.
Ok, I see the difference in thinking here. I'm wondering about what it looks like for a single kernel in an app that you would subscribe to for getting a new kernel.
The subscription happens elsewhere, mapped into the state of the app.
Well that doesn't work well in the example I describe above, where you need to know what cell the kernel belongs to.
How does state and data flow for this app normally? Do you have more example code?
In this case, I think a spawner belongs to each cell, since it's the kernel for that cell.
In this case, I think a spawner belongs to each cell, since it's the kernel for that cell.
Okay, that would work. What happens if you clean up a spawner? Does it close all of it's kernels? Or are kernels closed by closing all channels?
How does this work with multi user shared compute?
What happens if you clean up a spawner? Does it close all of it's kernels? Or are kernels closed by closing all channels?
My thinking on use cases in nteract/ick and nteract/nteract is that cleaning up a spawner would close the one kernel.
How does this work with multi user shared compute?
That's likely a bit out of bound for this discussion (which I helped in going off on this tangent 😉). However, maybe we can nail what a user experience would look like.
If a user is sharing a kernel, what they don't share is the actual connection (e.g. the websocket in their browser). They do want to share the same backing kernel though, which means that if one user changes the kernel being used, they should also get a new websocket for connecting to that kernel. Effectively, everyone is subscribed to changes on what kernel they are all interacting with.
This should live somewhere other than enchannel.
This should live somewhere other than enchannel.
Yeah, something like spawnteract or similar.