Escape codes not interpreted properly in /bin/ksh on macOS Big Sur

[David J. Weller-Fahy]

I'm running polyglot using ksh on macOS Big Sur v11.6.

When running it on CentOS or RHEL, the longer commands slide, so they stay on one line.

However, whether I use polyglot within wezterm or the default Terminal app, scrolling back to a long line or typing in a long line will result in the cursor jumping around. The simplest example I've been able to reproduce is executing a git command that exceeds the line length, then opening a new shell and pressing up arrow twice with the intention of re-running (or modifying) the long git command.

The result of that sequence of actions in an 80x24 Terminal is shown in the image below:

I also experience strange behavior when moving left and right on long commands, but I believe those are related to this issue, so will wait until we figure out how to solve this and see if the others go away.

[David J. Weller-Fahy]

It seems other escape codes are messing with things - for example, starting a fresh Terminal, typing 'ls' (vi mode is active), then hitting tab twice to trigger an autocomplete list results in the following:

I'll dig a bit if I have time, but let me know if you want me to do anything specific. Note I'm running off of the develop branch (as it's even with master).

[Alexandros Kozák]

Thanks for reporting this! Yes I can see those escape sequences; 31;1m in the first example, and [ in the second example.

Are all the instances of this problem occurring right after you've started a new ksh or opened a fresh terminal? That could be meaningful.

Could you test running set -o multiline to change the line-wrapping behavior -- just to see if it is the horizontal scrolling is the main problem? Let me know if that changes things, or if you still see escape sequences. You can set it back with set +o multiline later.

Also, could you run echo ${.sh.version} and let me know what it says? And run echo $TERM in both terminals. THANKS!

[Alexandros Kozák]

Also, try what I've got on the term-hack branch and see if it fixes your problem.

[David J. Weller-Fahy]

Note that the [ in the second example may be macOS Terminal being clever and surrounding any lines with a completed command with [ and ] - if you notice, they surround the historical command-line, and in my other test terminals they are always present (even in bash).

In answer to your general question about whether the issues happen after opening a new terminal: one of them does! It appears that the problem with auto-completion only happens as the first command. Multi-line settings do not have an effect on it. So, the auto-complete causing termination of the shell happens only on the first command after opening a new shell.

The second issue (cursor moving about when recalling history, and cursor doing strange things when moving left or right) does not only happen on the first command. Interestingly, when I set -o modeline I cannot reproduce the issue with the cursor moving upward. However, when I then set +o modeline I also cannot reproduce the issue. I have seen it a few times, but have been unable to reproduce it reliably.

Below is the information you requested.

KSH version:

dave ~ $ echo ${.sh.version}
Version AJM 93u+ 2012-08-01

In Terminal:

dave ~ $ echo $TERM
xterm-256color

In wezterm:

dave ~ $ echo $TERM
xterm-256color

[David J. Weller-Fahy]

Hrm... I'm looking at term-hack and there doesn't seem to be a difference between term-hack, develop, or master. Am I missing something?

dave ~/.rc/polyglot (term-hack) $ git ls-remote upstream | head -n 4
55d4044b07978b6cceb98423f99e8773c29e6659        HEAD
55d4044b07978b6cceb98423f99e8773c29e6659        refs/heads/develop
55d4044b07978b6cceb98423f99e8773c29e6659        refs/heads/master
55d4044b07978b6cceb98423f99e8773c29e6659        refs/heads/term-hack

[Alexandros Kozák]

Give the branch another pull. Thanks for your patience.

[David J. Weller-Fahy]

Alright, pulled term-hack, used that. The auto-complete on first command is still present in Terminal. I didn't try it in wezterm because of the following:

That's with the term-hack version. Going back to the develop version allows me to use wezterm.

[Alexandros Kozák]

OK.

Just to be absolutely thorough, on the develop branch, try TERM=vt100 ksh and see what happens.

[David J. Weller-Fahy]

This is with develop in wezterm:

This is the same thing in Terminal:

[David J. Weller-Fahy]

I'll keep trying to reproduce the other stuff as well.

[Alexandros Kozák]

I'm 99.99% sure my prompt couldn't be causing the memory fault! It's probably worth researching that problem elsewhere.

The business with the cursor jumping around, though, could conceivably be Polyglot's fault. I'm not sure yet. I don't have a Mac to experiment with, but I do notice a problem with horizontal scrolling on OpenIndiana that must be my fault. Give me a little while to fix that and we'll see if it improves your situation at all.

[David J. Weller-Fahy]

That sounds good - I'm certain (even though I may be wrong) that the memory fault is not polyglot - this shell is pretty ancient, after all. :)

Once you get the fixes for the horizontal scrolling in, I'll check that out and see if if fixes my other issues. Thanks for the help, and the information about multiline!

[David J. Weller-Fahy]

Oh! I just realized that this is all in vi editing mode. I have not been able to duplicate the weird cursor movement in emacs editing mode. The auto-complete memory fault does happen in either mode.

[Alexandros Kozák]

That's really meaningful, I think -- the point about vi editing mode (which I use myself, exclusively). I'll keep that in mind.

[Alexandros Kozák]

Interestingly, the tab completion problem you've run into looks a little like this -- but that would appear to be a bug more recently introduced into the shell.

[David J. Weller-Fahy]

Yes, and in that thread it is mentioned that the problems are throughout the code base, so this could just be another example. I'll do a trace of it when I get a chance.