zincsearch / zincsearch

ZincSearch . A lightweight alternative to elasticsearch that requires minimal resources, written in Go.

Home Page:https://zincsearch-docs.zinc.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to send logs from nginx to zinc in docker?

devig opened this issue · comments

commented

Could you add some examples to send logs from nginx?

use fluent-bit

  1. install fluent-bit:

https://docs.fluentbit.io/manual/installation/getting-started-with-fluent-bit

  1. configure with zinc:

https://docs.zincsearch.io/ingestion/fluent-bit/

use filebeat

  1. install filebeat:

https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html

  1. configure with zinc:

filebeat.yml

setup.ilm.enabled: false
setup.template.name: "nginx-log"
setup.template.pattern: "nginx-log-*"

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/nginx/*.log

output.elasticsearch:
  hosts: ["http://zinc:4080"]
  path: "/es/"
  index: "nginx-log-%{+yyyy.MM.dd}"
  username: "admin"
  password: "Complexpass#123"
commented

@hengfeiyang thank you.
fluent-bit - config does not work's for me. And as I know syslog-rfc3164 used UDP, UDP - may loose data.
I think rsyslog (with TCP) may be better.

How I can read logs from stdout if I am using docker (nginx puts logs in stdout)?

the config file of fluent-bit just an example, and the syslog just a defined parser.

fluent-bit has some components: input, parser, filter, output.

no matter, let me know how is you use nginx with docker? i'll guide you.

  1. run nginx, zinc as single container in a hosted docker (centos, ubuntu, macos).
  2. run nginx, zinc as a deployment in k8s cluster.

@devig This is more of a question on how to use log forwarders like fluent-bit, vector, fluentd, filebeat to forward data rather than zinc itself.

As @hengfeiyang mentioned depending on how you are running nginx (binary, docker, kubernetes) the method may vary. We can guide you more but would require more info from you. Also posting it on the appropriate forum of the log forwarder used will be useful.