un000 / tailor

Tailor, the library for tailing logs under logrotate, written in go.

Tailor, the library for tailing logs under logrotate

Tailor provides the functionality of tailing for e. g. nginx logs under logrotate. Tailor will follow a selected log file and reopen it if it's been rotated. Now, tailor doesn't require inotify, because it polls logs with a tiny delay. So the library can achieve cross-platform.

There is no plan to implement truncate detection.

Currently this library is used in production, handling 5k of opened files with a load over 100k rps per instance, without such overhead.

Install

go get github.com/un000/tailor

Features

• Tail files from any offsets
• Reopening on logrotate
• Rate limiter + support custom rate limiters
• Leaky bucket
• Performant helpers to trim or convert bytes to string
• Lag monitoring

Example

package main

import (
	"context"
	"fmt"
	"io"
	"time"

	"github.com/un000/tailor"
)

func main() {
	t := tailor.New(
		"./github.com_access.log",
		tailor.WithSeekOnStartup(0, io.SeekStart),
		tailor.WithPollerTimeout(10*time.Millisecond),
	)

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	err := t.Run(ctx)
	if err != nil {
		panic(err)
	}

	fmt.Println("Tailing file:", t.FileName())
	for {
		select {
		case line, ok := <-t.Lines():
			if !ok {
				return
			}

			fmt.Println(line.StringTrimmed())
		case err, ok := <-t.Errors():
			if !ok {
				return
			}

			panic(err)
		}
	}
}

Contributions are appreciated, feel free ✌️