Newline breaks CJK chars

[›]

the invalid char appears around the line break --- it seems that the newline char was inserted in the middle of code point.

�
��

[Aleksei Kovrizhkin]

1. How can I reproduce this behavior?
2. Here is how webtail processes lines:

line := <-tw.tf.Lines // tw.tf is a https://pkg.go.dev/github.com/nxadm/tail#Tail
msg := &TailerMessage{Channel: tw.channel, Data: line.Text}
data, _ := json.Marshal(TailMessage{Type: "log", Data: msg.Data})
client.send <- data // see https://github.com/gorilla/websocket/blob/master/examples/chat/client.go#L48

Do you have an idea what step that newline char might be inserted in?

[›]

Check this out with the default Docker image:
test.log

You can see the broken chars. And if you remove the heading 1or2, it looks normal.

[Aleksei Kovrizhkin]

@hyorigo, thank you for this sample.
The cause of such behaviour lies in read buffer limit which is 180 bytes by default.
Here this default value is set
Here nxadm/tail translates it to bufio

I think there are 2 solutions:
1, Add rune support to nxadm/tail
2. Run webtail with --skip 0 or some value which is big enough

[Aleksei Kovrizhkin]

== no feedback for 3+ months. Closing ==