Use a persistent shell instance, preferably an interactive one

[Feraidoon Mehri]

There are two missing pieces in up:

• Persistent shell: Currently up forks a new shell for each command. This is very inefficient (especially if you have a lot of shell functions and heavy dotfiles like I do), and also doesn't persist variables. Solving this one alone is not very hard; I have a library for Python for using a persistent shell, and a similar thing should be easily implementable here. We just need a socket (I currently use stdin/stdout for brish, but I plan to change this to proper sockets) to communicate between the interpreter and the host, and a helper server written in the interpreted shell that reads the commands the host sends and sends their results back.

• Autocompletion: Getting this to work is harder. I know of this zsh plugin that does this for zsh, but my experience of it has been slow, and buggy and it doesn't return all the completions. However, I think that if we can get the shell to run in an emulated terminal thingy, we should be able to use the shell's builtin autocomplete and also get it to persist at the same time. Now, that linked plugin is kinda doing this as well, though I haven't looked at it too deeply and anyways I don't know these gritty low-level details at all. The idea I have is using tmux. Solving the first problem with tmux should be easy (we can send the needed keystrokes to tmux. Its shell API is tmux send-keys ....), however, reading the results and also capturing completions triggered by pressing tab is harder. But I think this is the most promising avenue, and it works with any interpreter.

[Mateusz Czapliński]

Hm; as a workaround for the time being, do you think it could be possible to write a wrapper for brish (in Python?), that you'd then pass as $SHELL (or with the new -e flag) to up? I.e., the wrapper could start brish in background, opening some socket, and then you could maybe start up -e=brish-client -e=$SOCKET?

Other than that, I'm wondering if that idea doesn't really grow in a direction of something like Jupyter protocol? If yes, could it make sense to reuse/piggyback on that one? This could potentially automagically open the gates to a huge library of Jupyter "kernels"...?

[Feraidoon Mehri]

Hm; as a workaround for the time being, do you think it could be possible to write a wrapper for brish (in Python?), that you'd then pass as $SHELL (or with the new -e flag) to up? I.e., the wrapper could start brish in background, opening some socket, and then you could maybe start up -e=brish-client -e=$SOCKET?

Other than that, I'm wondering if that idea doesn't really grow in a direction of something like Jupyter protocol? If yes, could it make sense to reuse/piggyback on that one? This could potentially automagically open the gates to a huge library of Jupyter "kernels"...?

Yes, I think using Jupyter is best. It should be possible; Emacs-jupyter uses Jupyter programmatically. Unfortunately, I don’t know more details.