Scribe input plugin for Fluent¶ ↑

Overview¶ ↑

This is a plugin for fluent event collector. This plugin adds the Scribe compatible interface to fluent.

What’s Scribe?¶ ↑

Scribe is a server for aggregating log data streamed in real time from a large number of servers, developed at Facebook.

It uses Thrift, cross-language RPC framework, to communicate between clients and servers.

What’s Scribe plugin for fluent?¶ ↑

The Scribe plugin for fluent enables fluent daemon, to talk the Scribe protocol by using Thrift. The following shows the protocol itself, in thrift-idl format:

enum ResultCode
{
  OK,
  TRY_LATER
}

struct LogEntry
{
  1:  string category,
  2:  string message
}

service scribe extends fb303.FacebookService
{
  ResultCode Log(1: list<LogEntry> messages);
}

The category field is used as fluent ‘tag’.

How to use?¶ ↑

To use this plugin with fluent, please add the following configuration to fluent.conf.

# Scribe input
<source>
  type scribe
  port 1463
</source>

The following options are supported.

port: port number (default: 1463)
bind: bind address (default: 0.0.0.0)
server_type: server architecture either in ‘simple’, ‘threaded’, ‘thread_pool’, ‘nonblocking’ (default: nonblocking)
framed: use framed protocol or not (default: true)

For Developers¶ ↑

To test this plugin, please execute the following command in your ‘fluent’ project directory, not scribe input plugin directory.

$ export SCRIBE_PLUGIN_DIR=../fluent-plugin-scribe                                                                   
$ FLUENT_PLUGIN=$SCRIBE_PLUGIN_DIR/lib/fluent/plugin/ ./bin/fluentd --config $SCRIBE_PLUGIN_DIR/example.conf -v -v -v

Then please execute the sample client.

$ ./bin/fluent-scribe-remote

Contributors¶ ↑

Satoshi Tagomori

Copyright¶ ↑

License: Apache License, Version 2.0

frsyuki / temp-fluent-plugin-scribe