question on telnet vs terminals sequence

[Yucong Sun]

Hi,

So, recently I started to work adding websocket support (with libwebsocket) to my application, which is an vitage text-based game engine, sort of behave like an multi-user time-shared terminal. The project use to deal with telnet protocol specifically, and it automatically turns on linemode and local echo during telnet negotiation.

In searching of similar stuff over websocket, I found xtermjs and also domterm. But it seems xtermjs specifically doesn't support any "telnet" client features, which means I would have to write line-editing support directly in my application.

I found this in DomTerm

"\e[80;97u"
"\e[80;99u"
"\e[80;108u"
"\e[80;112u"
Set input editing mode. The value 99 ('c') sets character mode; the value 108 ('l') sets line-editing mode. The value 97 ('a') set automatic mode, which switches between character mode and line-editing mode based on the mode of the inferior process (when using a PTY). The value 112 ('p' for "pipe") is like line-editing mode, but the inferiors doesn’t echo the input, so we have to do it. This mode is useful when the input is a pipe or some other non-tty stream.

Which seems to be doing what I wanted, but also it seems that it doesn't really belong to any sort of standard. Also, what would be the "mode of inferior process?" since I'm not really using an PTY. My question is, what is your suggestion?

1. Should I stick to telnet protocol, and work on creating some telnet protocol handling in JS , write to transparent terminal emulators like xtermjs?
2. Should I try to incorporate readline library into my engine and support editing and remote-echoing , directly writing to terminals like xtermjs ?
3. Or should I switch to support DomTerm's non-standard line-editing mode directly in my engine? Which is better in your opinion?

Not only do I want to support raw domterm on an desktop computer, I may want to eventually support mobile clients with an webview or some other framing protocol .

Thanks

[Per Bothner]

On 12/30/19 10:04 AM, Yucong Sun wrote: Hi, So, recently I started to work adding websocket support (with libwebsocket) to my application, which is an vitage text-based game engine, sort of behave like an multi-user time-shared terminal. The project use to deal with telnet protocol specifically, and it automatically turns on linemode and local echo during telnet negotiation.

I assume you're talking about FluffOS? I don't think I heard of it before, but I google'd your name. I only had a vague knowledge of MUDs, but I think DomTerm could be very useful for it (though I'm biased of course).

Which seems to be doing what I wanted, but also it seems that [switching line mode] doesn't really belong to any sort of standard.

No, it's my own invention.

My question is, what is your suggestion? Should I stick to telnet protocol, and work on creating some telnet protocol handling in JS , write to transparent terminal emulators like xtermjs? or should I switch to support DomTerm's non-standard line-editing mode directly in my engine? Which is better in your opinion?

If you use xtermjs, you would still have to implement line editing. Basic telnet-style line-editing is fairly simple, but readline-level functionality is more work. Mouse/selection integration is also nice - DomTerm has that, but most readline-style libraries don't.

Not only do I want to support raw domterm, I may want to eventually support mobile clients with an webview or some other framing protocol

Not sure what you mean by "raw domterm" but I think it makes sense to support two modes: (1) The user is running a local domterm client (the domterm binary) (2) The user uses DomTerm in a browser window, talking directly to the remote server. DomTerm has lots of potential nice features for a MUD, including rich text and image support: http://domterm.org/Features.html If you do go the DomTerm route, I can promise you a lot of help in terms of advice, implementing/testing missing functionality, and other help. I have experimented with using xtermjs as the core terminal-emulator for DomTerm, as opposed to DomTerm's native terminal emulator: https://github.com/PerBothner/DomTerm/blob/master/README-xtermjs.md Partly because xtermjs is faster (especially with the new webgl renderer), though DomTerm is pretty fast for most use (except things like cat'ing a long file - but why do that?). Partly because xtermjs has a bigger community. However, it would take a of of work to implement DomTerm's features on top of xtermjs. -- --Per Bothner [e-mail] http://per.bothner.com/

[Yucong Sun]

well, I agree the performance matters very little and it can always be improved, so it matters little in my case.

I know every terminal emulator has its own stuff, but in general xterm compatibility is enough, if an feature doesn't work on popular terminal emulator (xterm, iterm ) , it is very unlikely to be used anyway.

What I do want from domterm though, is following:

1. distribute an consolidated and minified js and css. (built from master if release is not very often)
2. an clear instruction on how to use domterm in an browser, connecting to a custom made WS backend.
   1. repl-client.html sort of does what it takes, but domterm-client.js contains too much stuff that I need to replace.
   2. make it clear what to expect from the backend, right now domterm sends a bunch binary handshake messages to backend, (for example .focus, .version etc) , either I need to implement those handshakes or if possible, domterm can support an standardized application level protocol (for example telnet, that allows linemode and local echo control messages)
   3. some js code to handle PTY related VT functions ( on sending \r replace it to \r\n , and on receiving \n replace them with \n\r) so websocket backend can be pure ascii input/output , no need to care about PTY stuff.

[Per Bothner]

On 12/31/19 12:00 PM, Yucong Sun wrote: What I do want from domterm though, is following: 1. distribute an consolidated and minified js and css. (built from master if release is not very often)

In the past, I experimented with Closure. I dropped it as it complicated development while I concentrated on local terminal emulation, where minified code is less important. Do you recommend any particular tool? It has to handle a mix of module and non-module code.

2. an clear instruction on how to use domterm in an browser, connecting to a custom made WS backend.

There is a start here: http://domterm.org/Remote-serving.html but that assumes you use the domterm executable for your WS backend. The Kawa language implementation (https://www.gnu.org/software/kawa/) can create a console window using DomTerm, doing the serving itself, without using the domterm executable: https://www.gnu.org/software/kawa/REPL-Console.html#Running-a-Command-Interpreter-in-a-new-Window one of two ways: (1) Using the Java HttpServer class, combined with AJAX (instead of WebSockets) - see the connectAjax code in domterm-client.js. (2) Using the JavaFX WebViewer class. It uses HttpServer only to serve the js and css resources and the initial jfx-term.html. It uses the Java/JavaScript bridge (see jfx-term.html) instead of WebSockets. I mention these as examples of how the DomTerm client code can be used without the domterm server. If you're planning on using WebSockets, I agree that is the best way to go. Roughly, the client code (probably in a "load" handler) could be as simple as: let name = "term1"; let termElement = DomTerm.makeElement(name, document.body); let wsProtocol = ""; // ? Terminal.connectWS(name, PATH_TO_WebSocket_SERVER, wsProtocol, termElement);

3. repl-client.html sort of does what it takes, but domterm-client.js contains too much stuff that I need to replace.

The last few days I've been working on getting the JavaFX embedded browser to work (again). It uses jfx-term.html - which no longer uses domternm-client.js.

4. make it clear what to expect from the backend, right now domterm sends a bunch binary handshake messages to backend, (for example .focus, .version etc) , either I need to implement those handshakes or if possible, domterm can support an standardized application level protocol (for example telnet, that allows linemode and local echo control messages)

See http://domterm.org/Wire-byte-protocol.html#Special-sequences-sent-by-DomTerm-to-back-end for the various messages. The server doesn't have to handle all the events - some can be ignored. The default protocol send these event messages "inline" is 0xFX NAME DATA \n However, you can override the reportEvent method (in terminal.js) if you have some alternate channel (see jfx-term.html). However, if you're using WebSockets might as well use the default encoding.

5. some js code to handle PTY related VT functions ( on sending \r replace it to \r\n , and on receiving \n replace them with \n\r) so websocket backend can be pure ascii input/output , no need to care about PTY stuff.

See the automaticNewlineMode field in the Terminal class (in terminal.js). It is set by setInputMode which is called by the "\e[80;MODEu" escape sequence, but you can also set it in your initialization code.

…

-- --Per Bothner [e-mail] http://per.bothner.com/

[Per Bothner]

"distribute an consolidated and minified js and css."

I wrote some Makefile rules (using sed and patch) to combined the core .js files into a single file, removing the ES6 modules stuff. (Whether this is done will depend on configure options.) The resulting file can be compiled by Google closure-compiler, reducing the size from 495280 to 234523 bytes (for those files).

This still needs some polishing and testing but is promising.

Even better compression should be possible with "advanced optimizations" but getting those to work is more complicated.

[Yucong Sun]

webpack/babel is the way to go, I can help you set it up since I had some experience on frontend side as well.

see my comments on what is really need on xtermjs/xterm.js#2664 and let me know what you think

[Per Bothner]

"webpack/babel is the way to go,"

What is the advantage of webpack/babel? For me Google closure-compiler has the advantage that it's trivial to install on Fedora (sudo dnf install closure-compiler) plus I had old (commented-out) Makefile rules I could update. So once I managed to concatenate the source files and strip out es6-module stuff it was very easy.

It should be easy to modify the Makefile to make use of some other minifier, if the benefit justifies it.