rem4ik4ever / railway-chord

railway-chord

Vector log egress for Railway.app projects.

railway-chord pipes the log stream from Railway's GraphQL API into Vector. This gives you project-wide centralized logging on Railway. For each Railway project you configure, railway-chord dumps the log stream of the project's deployed services and plugins (e.g. Postgres, Redis, etc.) in all environments into Vector.

Usage

⚠️ railway-chord depends on Railway's API. You must be a part of the Priority Boarding program to use Railway's API. You can join the program easily by following the instructions in the link.

The button above will deploy railway-chord on Railway. You will be prompted to set some required environment variables; there is no additional configuration required beyond this. Log sinks are automatically configured based on the presence of required environment variables of each provider, i.e. setting a DATADOG_TOKEN will enable Datadog, and so on.

Datadog

To enable Datadog logs, set the DATADOG_TOKEN service variable.

There is an optional DATADOG_SITE setting if your Datadog account is hosted on a different Datadog instance (defaults to datadoghq.com).

Logtail

To enable Logtail, set the LOGTAIL_TOKEN service variable.

stdout

To enable logging to stdout, set ENABLE_STDOUT=true. This will output all enabled project's logs into railway-chord's stdout.

Project IDs

The RAILWAY_PROJECT_IDS variable requires a comma-separated list of Railway Project IDs to enable railway-chord for. You can find your Project ID under Railway's Dashboard -> Project -> Settings -> General.

Example:

# Separate each Project ID with a comma!
RAILWAY_PROJECT_IDS="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX,XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"

Log Enrichment

railway-chord will inject a railway object into the logs sent to the provider containing the deployment/plugin ID and name.

Example:

{
  "railway": {
    "id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
    "name": "app.up.railway.app",
    "type": "DEPLOYMENT"
  }
}

Railway API Rate Limit

Railway's Public API currently has a rate limit of 1,000 requests daily. railway-chord will make ~100 requests every 24 hours for each project enabled. Work is underway to reduce the amount of requests railway-chord is making (see railway-chord#3).

Be careful about this if you're using Railway's API for something else - railway-chord will eat into your rate limit!

Adding Vector sinks

Request for new sinks

To request for a new Vector sink, please open a GitHub issue.

Add your own sink

See this pull request for an example.

Before adding a new Vector sink, check the authentication mechanism of the provider you're using. There is usually an API key required.

1. In src/main.ts, pass the required API key/token into configureVector():

   const PROVIDER_TOKEN = process.env.PROVIDER_TOKEN ?? null
   const vectorCfg = configureVector(
     ...,
     PROVIDER_TOKEN
   )

2. Create a new Vector sink configuration in src/vector/sinks.ts (TOML format).

   const PROVIDER = (token: string) => `
   [sinks.PROVIDER]
   ...
   token = "${token}"
   `

3. In src/vector/configure.ts, import and append the newly-created config created above, passing the required API key into it:

   import { PROVIDER } from './sinks'
   const configure = ( ..., providerToken: string | null ) => {
       ...
       if (providerToken !== null) {
           enabled.push('provider')
           cfg += PROVIDER(providerToken)
       }
       ...
   }

Configuration

Name	Value
RAILWAY_API_TOKEN	Required. Railway API token.
RAILWAY_PROJECT_IDS	Required. A comma-separated list of Railway Project IDs.
LOGTAIL_TOKEN	Optional. Logtail token.
DATADOG_TOKEN	Optional. Datadog API token.
DATADOG_SITE	Optional. Datadog site setting. Defaults to datadoghq.com.
ENABLE_STDOUT	Optional. Enable Vector logging to stdout.
RAILWAY_API_HTTP_ENDPOINT	Optional. Railway's HTTP GQL Endpoint. Defaults to https://backboard.railway.app/graphql/v2.
RAILWAY_WS_HTTP_ENDPOINT	Optional. Railway's WebSockets GQL Endpoint. Defaults to wss://backboard.railway.app/graphql/v2.
VECTOR_BIN_PATH	Optional. Path to Vector binary. Defaults to path defined in Docker build. You do not need to set this.

License

MIT