Unicode glyphs ruin the TUI in my terminal
strk opened this issue · comments
I'm not sure this is a problem with toot itself or with my terminal, but I'm experiencing it with toot so I'm filing it here for now: presence of unicode glyphs seem to confuse toot as to how large is the text. See screenshot:
This is with version v0.28.0 as packaged on Ubuntu 22.04.3 LTS
This is a miserable problem. The issue is that different terminal emulators support different versions of the Unicode standard. Let's say your emulator supports Unicode 7.0. It knows the width of characters defined up to the release date of 7.0. But now we're up to Unicode 16. Any characters defined after Unicode 7.0 (lots of emojis!) your terminal will not know the width of, so it'll probably default to single-cell width. But maybe the emoji you are using is double-cell-width.
Whereas the toot app tries to be smart and uses a library (wcwidth) to find the correct cell widths for all characters. If toot and your terminal disagree, you'll see the kind of alignment errors you've posted above.
There's also ucs-detect that attempts to provide a way to detect your terminal's unicode support level. But even if we do that, I'm not sure we can provide a complete solution for using the latest Unicode emoji.
The TUI library we use (urwid) suffers from the same problem, people have tried to integrate wcwidth support in the past, but no patches have been accepted AFAIK.
Both of wcwdith and ucs-detect libraries were dormant since 2020, but have recently revived and have new releases published. I'm interested to see what they can do, but I'm not hopeful they'll provide a solution for us.
In the meantime, you can minimize the problem by using a terminal with support for a recent version of Unicode. Windows Terminal is pretty good, if you're on Windows.
Thanks for the in-depth explanation. Using gnome-terminal (3.44.0) does indeed fix the problem (on Ubuntu-22.04). The terminal on which I have the problem is qterminal-0.17.0
I think that moving to Textual will solve this issue.
@davidbrochart after a chat with the wcwidth author I'm pretty well convinced that the only real solution to the problem is to pressure terminal emulator authors to upgrade their support to the latest Unicode version. Textual, by itself, has no solution to this problem AFAIK.
I just tried tooi, and it seems to solve some issues. Here is toot followed by tooi (see on the right of line "It's Migrations week"):
It does not solve all the problems, e.g. here's a trans flag in foone's handle causing a black char on the next line.
My guess is Textual renders line boxes differently, in a way that is unconnected to rendering the enclosed contents - using absolute positioning of the cursor when drawing the box sides.
It may be possible to simulate this in Urwid by obtaining the screen dimensions and using a fixed size widget of appropriate size, but I'm not sure of that.