replacements for system_command and system_command_nb

[Ryan Hileman]

Right now system_command and system_command_nb use os.system and subprocess.Popen(shell=True) respectively.

These both can have some undesirable behavior:

• os.system and Popen(shell=True) launch an OS-native shell, which will behave differently between windows and posix, and it's difficult to safely escape shell input.
• os.system leaks Talon's file handles to the child, which may cause unexpected behavior in both Talon and the launched app.
• Popen(shell=True) may close the launched process when Talon itself closes, which is likely undesirable. This behavior also depends on your platform.
• Popen(shell=True) can leak resources inside Talon with each subprocess launched.

I propose improving these in a couple of ways:

• Use shlex.split instead of shell=True. shlex will behave the same across platforms, and doesn't do shell expansion of variables and subcommands, so is generally safer.
• Use subprocess.run(check=True) instead of os.system(). subprocess.run doesn't leak Talon's file handles, doesn't invoke an extra /bin/sh process, and check=True means you'll get an exception if the subprocess fails, which is usually a good thing.
• Use talon's system.launch instead of subprocess.Popen(shell=True). This spawns the process separately from Talon, so it won't close when Talon exits, and won't leak resources.

Please test this on each platform before merging.

from talon import Module, system
import shlex
import subprocess

mod = Module()
@mod.action_class
class Actions:
    def system_command(cmd: str) -> None:
        """Run a command and wait for it to finish"""
        args = shlex.split(cmd)
        subprocess.run(args, check=True)

    def system_command_nb(cmd: str) -> None:
        """Run a command without waiting for it"""
        args = shlex.split(cmd)
        cmd, args = args[0], args[1:]
        system.launch(path=cmd, args=args)

[splondike]

Had a go at this on the branch mentioned above. The implementation is the same as lunixbochs' with an attempt to allow ~ to expand to the user's home directory. There are some issues with it though relative to the current implementation:

• shlex.split doesn't work properly with Windows it seems. system_command("WScript C:\\Users\\Administrator\\Desktop\\test.vbs hello") came out of shlex.split as something like ["WScript", "C:UsersAdministratorDesktoptest.vbs", "hello"].
• Arguments with quotes and ~ behave differently, for example ~/my-script.sh "~/.bashrc" would previously pass a literal ~/.bashrc to the script. shlex.split doesn't let us find out if an argument was quoted however, so my implementation expands it to /home/my-user/.bashrc.

The first issue I think requires a different parser to shlex.split. This would probably also let us fix the second (more minor) issue.

The following were the set of test cases (with per-platform variants) I was using to verify the behaviour:

• Run actions.user.system_command("false") (failing command) and check you get a stacktrace.
• Run actions.user.system_command_nb("false") (failing command) and check you don't get a stacktrace.
• Run actions.user.system_command("~/my-script.sh ~/.bashrc") and check the ~ are expanded correctly and the shell script is executed if it has the execute mode set.
• Run actions.user.system_command("~/my-script.sh 'with quotes'") and actions.user.system_command("~/my-script.sh '~/.bashrc'") and check the quotes are handled correctly. Also run the _nb version with same.
• Run actions.user.system_command("bash -c \"false || ~/my-script.sh 'with quotes'\"") and check the quotes are handled correctly. Also run the _nb version with same. This is the accepted way of running an ad-hoc shell command.
• Run actions.user.system_command("sleep 5") (or equivalent) and check the action doesn't return until the command exits.
• Run actions.user.system_command_nb("gedit"), check the gui comes up, and doesn't close when you exit Talon.

[David Vo]

FWIW, there only two files that use system_command, which could well use a rewrite for the affected commands anyway:

• apps/i3wm/i3wm.talon
• apps/dunst/dunst.talon

[splondike]

OK, I ended up writing my own lexer/parser. I've heard of enough people using this action in their scripts that it seems worth implementing. It was also fun to write it ;).

[Ryan Hileman]

Ok, let me think about this and review it.

[splondike]

Listing out some options for how to proceed with this issue:

1. Change the action to def system_command(cmd: str, arg1: str="", arg2: str="", arg3: str="", arg4: str=""):. You could then call it from .talon like user.system_command("echo") or user.system_command("echo", "foo") etc. up to four arguments. We could deprecate the existing system by checking if there was a space in the first argument and using the existing os.system path if so.
2. Don't change the implementation. lunixbochs has identified some theoretical issues with it, but I haven't heard of it being too problematic so far.
3. lunixbochs makes a 1st party implementation in Talon.
4. Use my implementation in #1140. Hopefully it's pretty much a set and forget situation; I doubt we'd want to change the behaviour again (unless there are bugs). I am doing some model based testing against shlex.split, so hopefully there aren't many issues.
5. Deprecate then remove this API, as identified in this comment it's only used in a couple of places in this repo. I think it is used by a fair few knausj users for custom commands though, so we'd be making them all do some work.
6. Use shlex.split for Unix and a .dll call on Windows.
7. Try and find a third party pure Python implementation with compatible licensing and copy paste it in to knausj. This doesn't seem like an improvement on #1140 though.

I've sorted them in my order of preference. The fixed args count thing seems kind of neat, and if people want more they probably want to drop in to Python anyway.

[Michael Arntzenius]

One thing we could try is to to replace all of knausj's uses with some other, safer API/implementation, and keep but deprecate the existing actions.