Open source, horizontally scalable Prometheus as a service

NB this is a pre-beta service. Availability is not 100%. APIs might change. Data could be lost.

Cortex is an API compatible Prometheus implementation, that natively supports multitenancy and horizontal scale-out clustering.

Cortex is a Weaveworks project that forms the monitoring backend of Weave Cloud.

Getting Started with Weave Cloud & Kubernetes

Go to https://cloud.weave.works and sign up. Follow the steps to create a new instance.

Once you have created your instance, note down the 'Service Token' listed underneath 'Probes' in the box on the right of the screen. You will use this token to authenticate your local Prometheus with Weave Cloud.

Install the Cortex agent (an unmodified OSS Prometheus) on your Kubernetes cluster using the following command:

$ kubectl apply -n kube-system -f "https://cloud.weave.works/k8s/cortex.yaml?service-token=<token>&k8s-version=$(kubectl version | base64 | tr -d '\n')"

Click Monitor to access the Cortex service in the Weave Cloud UI:

Weave Cloud gives you some example queries that displays general system information. You can of course write your own Prometheus queries.

About Weave Cortex

To learn more about Weave Cortex, consult the follow documents / talks:

Original design document for Project Frankenstein
PromCon 2016 Talk: "Project Frankenstein: Multitenant, Scale-Out Prometheus": video, slides
KubeCon Prometheus Day talk "Weave Cortex: Multi-tenant, horizontally scalable Prometheus as a Service" slides
Or join us on our Slack channel and ask us directly!

Using Cortex on Other Platforms

Cortex can be used to monitor virtually any application on any platform, as the local agent is a specially configured unmodified OSS Prometheus binary. You can download Prometheus from its website. You will need v1.2.1 or later.

When you've got Prometheus, you will need to configure it to discover your services and configure it to send its data to Weave Cloud by adding the following top-level stanza to prometheus.yml:

remote_write:
  url: https://cloud.weave.works/api/prom/push
  basic_auth:
    password: <token>

Where <token> is the Service Token you obtained from Weave Cloud.

Once your local Prometheus is running you can enter Prometheus queries into Weave Cloud. Go to https://cloud.weave.works and click Monitor from Weave Cloud header:

Using Cortex to power Grafana dashboards

To use the Prometheus Service with Grafana, configure your Grafana instance to have a datasource with the following parameters:

Type: Prometheus
Url: https://cloud.weave.works/api/prom
Access: Proxy
Http Auth: Basic Auth
User: user (this is ignored, but must be non-empty)
Password: (the same one your retrieval agent uses)

Developing

To build & test, install minikube, and run:

eval $(minikube docker-env)
make
kubectl create -f ./k8s

Cortex will sit behind an nginx instance exposed on port 30080. A job is deployed to scrape it itself. Try it:

http://192.168.99.100:30080/api/prom/api/v1/query?query=up

Experimental Bazel builds

We also have early support for Bazel builds. Once you have installed bazel, try these commands:

make bazel-test
make bazel

Bazel can be useful for running fast, local, incremental builds and tests. Currently bazel does not support cross-compiling so it is not used to produce the final binaries and docker container images.

Vendoring

We use dep to vendor dependencies. To fetch a new dependency, run:

dep ensure && dep prune

To update dependencies, run

dep ensure --update && dep prine

Getting Help

If you have any questions regarding Cortex our hosted Prometheus as a service:

Invite yourself to the #weave-community slack channel.
Ask a question on the #weave-community slack channel.
Join the Weave User Group and get invited to online talks, hands-on training and meetups in your area.
Send an email to weave-users@weave.works
File an issue.

Your feedback is always welcome.

mattbostock / cortex