When too much data written to logrus.Writer, next writes fail
karelbilek opened this issue · comments
I am not sure if this should even be considered a bug, but I guess might.
By a mistake, we logged a big chunk of data into logrus transformed to io.Writer. Next writes to the same writer then fail.
package main
import (
"fmt"
"github.com/sirupsen/logrus"
"strings"
)
func main() {
logger := logrus.NewEntry(logrus.New())
writer := logger.Writer()
fmt.Fprintln(writer, strings.Repeat("A", 20_000_000))
_, err := fmt.Fprintln(writer, "foo") // this and all following writes always fail, because the pipe is broken
fmt.Println(err)
}
I am fine with the first write not working properly, but the next writes always write io: read/write on closed pipe and never get any chance to "reset" it.
Locally, I overcame this (so it does not happen again, even when I removed the root cause of 20MB write) by wrapping logger.writer like this.
Which works; not sure if it kosher though.
package main
import (
"fmt"
"github.com/sirupsen/logrus"
"io"
"strings"
)
type wrappedLogrusWriter struct {
l *logrus.Entry
w io.Writer
}
func (wlw *wrappedLogrusWriter) Write(p []byte) (int, error) {
n, err := wlw.w.Write(p)
if err != nil {
wlw.w = wlw.l.Writer()
}
return n, err
}
func NewWrappedLogrusWriter(l *logrus.Entry) io.Writer {
w := l.Writer()
return &wrappedLogrusWriter{
l: l,
w: w,
}
}
func main() {
logger := logrus.NewEntry(logrus.New())
writer := NewWrappedLogrusWriter(logger)
fmt.Fprintln(writer, strings.Repeat("A", 20_000_000))
_, err := fmt.Fprintln(writer, "foo")
fmt.Println(err) // is nil
}
Unfortunately, I cannot use this
because we use it as an arbitrary writer; we feed stdout of a command to logrus.Writer.
hello @karelbilek, it is a bug indeed.
I guess it's more related to the fact your input does not with a \n rather than its size.
Can you post this error trace from your sample https://github.com/sirupsen/logrus/blob/master/writer.go#L63
I suspect it should be io.EOF
It's time="2009-11-10T23:00:00Z" level=error msg="Error while reading from Writer: bufio.Scanner: token too long"
This issue is stale because it has been open for 30 days with no activity.
Not stale. I might try to make a PR but not sure if I got time, and if I don't break something in the process.
Especially I cannot break existing API; existing writer exposes *io.PipeWriter directly, which I cannot break.
Similar to
(for a different issue)
This issue is stale because it has been open for 30 days with no activity.
This issue was closed because it has been inactive for 14 days since being marked as stale.
It is still an issue.
@karelbilek is this tracked anywhere else ?
I don't think so.
I wanted to make a fix and never got around to it. Well it has been marked as stale, good job stale bot. (I hate stale bots.)