repeatedly / fluentd

Fluent event collector

Fluent¶ ↑

Fluent is an event collector system. It’s said that Fluent is generalized version of syslogd, which can deal with JSON object for the log message.

Archtecture¶ ↑

Fluent collects events from various data sources and write them to files, database or other storages:

Web apps  ---+                 +--> file
             |                 |
             +-->           ---+
/var/log  ------>  fluentd  ------> mail
             +-->           ---+
             |                 |
apache    ----                 +--> fluentd

Fluent also supports log transfer:

Web server
+---------+
| fluentd -------
+---------+|     |
 +---------+     |
                 |
Proxy server     |    Log server, Amazon S3, HDFS, ...
+---------+      +--> +---------+
| fluentd ----------> | fluentd ||
+---------+|     +--> +---------+|
 +---------+     |     +---------+
                 |
Database server  |
+---------+      |
| fluentd ---------> mail
+---------+|
 +---------+

An event consists of tag, time and record. Tag is a string separated with ‘.’ (e.g. myapp.access). It is used to categorize events. Time is a UNIX time when the event occurs. Record is a JSON object.

Quick Start¶ ↑

$ gem install fluentd
$ fluentd &
$ echo '{"json":"message"}' | fluent-cat debug.test

Web site

fluentd.org/

Documents

fluentd.org/doc/

Source repository

github.com/fluent

Author

Sadayuki Furuhashi

Copyright

© 2011 FURUHASHI Sadayuki

License

Apache License, Version 2.0