sciyoshi / vector

Collect, transform, & route all observability data with one simple tool

Docs  •  Chat  •  @vectordotdev  •  Download v0.7.2

---

Vector is a highly reliable observability data router built for demanding production environments. Vector is designed on the following principles:

• High reliability - Built in Rust, Vector is memory safe, correct, and performant.
• Operator safety - Vector is pragmatic and hard to break. It avoids the common pitfalls in similar tools.
• All data - Logs, metrics, and traces (coming soon). A sophisticated data model enables correct interoperability.
• One tool - Deploys as an agent or service. One tool gets your data from A to B.

Vector is deployed over 100,000 times per day, and is trusted by Fortune 500 companies and engineering teams trying to tame observability pipelines.

Documentation

About

• Concepts
• Data model (Event) - log event, metric event
• Guarantees

Setup

• Installation - containers, operating systems, package managers, from archives, from source
• Configuration
• Deployment - topologies, roles
• Guides - getting started, unit testing

Reference

• Sources - docker, file, journald, kafka, socket, syslog, and 6 more...
• Transforms - json_parser, log_to_metric, logfmt_parser, lua, regex_parser, sampler, and 14 more...
• Sinks - aws_cloudwatch_logs, aws_s3, clickhouse, elasticsearch, gcp_pubsub, gcp_stackdriver_logging, and 20 more...

Administration

• Process management
• Monitoring
• Updating
• Validating

Resources

• Community - chat, @vectordotdev, mailing list
• Releases - v0.7.2 (latest)
• Roadmap - vote on new features

Performance

Test	Vector	Filebeat	FluentBit	FluentD	Logstash	SplunkUF	SplunkHF
TCP to Blackhole	86mib/s	n/a	64.4mib/s	27.7mib/s	40.6mib/s	n/a	n/a
File to TCP	76.7mib/s	7.8mib/s	35mib/s	26.1mib/s	3.1mib/s	40.1mib/s	39mib/s
Regex Parsing	13.2mib/s	n/a	20.5mib/s	2.6mib/s	4.6mib/s	n/a	7.8mib/s
TCP to HTTP	26.7mib/s	n/a	19.6mib/s	<1mib/s	2.7mib/s	n/a	n/a
TCP to TCP	69.9mib/s	5mib/s	67.1mib/s	3.9mib/s	10mib/s	70.4mib/s	7.6mib/s

To learn more about our performance tests, please see the Vector test harness.

Correctness

The following correctness tests are not exhaustive, but they demonstrate fundamental differences in quality and design:

Test	Vector	Filebeat	FluentBit	FluentD	Logstash	Splunk UF	Splunk HF
Disk Buffer Persistence	✅	✅	❌	❌	⚠️	✅	✅
File Rotate (create)	✅	✅	✅	✅	✅	✅	✅
File Rotate (copytruncate)	✅	❌	❌	❌	❌	✅	✅
File Truncation	✅	✅	✅	✅	✅	✅	✅
Process (SIGHUP)	✅	❌	❌	❌	⚠️	✅	✅
JSON (wrapped)	✅	✅	❌	✅	✅	✅	✅

To learn more about our correctness tests, please see the Vector test harness.

The Little Details Make All Of The Difference

• Zero delay start - Starts and restarts without a delay.
• Hot reload - Reload configuration on the fly without disrupting data flow.
• Configurable concurrency - All CPU cores (service) or just one (agent) via the --threads flag.
• Programmable transforms - Lua, Javascript (coming soon), and WASM (coming soon).
• Templating - Use event fields to create dynamic values, enabling dynamic partitioning and more.
• Sink healthchecks - Protection against downstream disruptions make Vector a good citizen.
• Customizable schema - A schema is not forced on you. Change Vector's schema to anything you like.
• Decoupled buffer design - Buffers are coupled with sinks; a bad sink won't bring the entire pipeline to a halt.
• Thoughtful disk buffering - Transfers data to operating system memory to reduce the risk of data loss.
• Merge split logs - Easily merge multi-line logs into one event.
• Custom DNS - Custom DNS makes service discovery possible.
• Rate-limited logging - Vector's internal logging is rate-limited and noise-free.
• Optionally fully static binary - Optional MUSL static binaries mean zero required dependencies.
• Thoughtful docs - Quality documentation that respects your time and reduces communication overhead.
• Clear Guarantees - A guarantee support matrix helps you make the appropriate tradeoffs.
• Config unit tests - Develop Vector config files like code. Avoid the frustrating dev style required by other tools.
• Config linting - Quickly lint Vector config files to spot errors and prevent bad configs in CI.

Installation

Run the following in your terminal, then follow the on-screen instructions.

curl --proto '=https' --tlsv1.2 -sSf https://sh.vector.dev | sh

Or use your own preferred method.

Latest Posts & Announcements

• Prometheus Source
• EC2 Metadata Enrichments
• Alpha Kubernetes Source
• Use Custom DNS Servers
• Unit Testing Your Vector Config Files

view all...

---

Developed with ❤️ by Timber.io