Intro to Observability: OpenTelemetry in Go

This application is here for you to try out tracing. It consists of a microservice that calls itself, so you can simulate a whole microservice ecosystem with just one service!

What to do

Recommended:

Alternative:

Remix this app on Glitch.

Alternative 2: Clone and run locally. If you use VSCode Devcontainers, this repository is set up for that. Otherwise, we expect golang to be set up.

Start the App

./run

See the App

In GitPod: while it's running, click "Remote Explorer" on the left sidebar; then expand "ports" and look for a "preview" button.

Locally: http://localhost:3000

Activate the sequence of numbers by pushing Go. After you see numbers, push Stop. Try this a few times.

Stop the App

Push Ctrl-C in the terminal where the app is running.

Configure Tracing to Honeycomb

Our goal is to define a few environment variables. tracing.go reads these variables to send them to Honeycomb.

Create a .env file in the command line:

cp .env.example .env

Now open .env, and populate the environment variables. (This file will be ignored by git, so you won't commit your API key.)

export HONEYCOMB_API_KEY=replace-this-with-a-real-api-key # important
export SERVICE_NAME=fib-microsvc # can be any name

You can choose any Service Name you want.

See the Results

Run the app. Activate the sequence of numbers.

Go to Honeycomb and choose the Service Name you configured.

How many traces are there?

How many spans are in the traces?

Why are there so many??

Which trace has the most, and why is it different?

2. Customize a Span

Let's make it easier to see what the "index" query parameter is.

In the fibHandler function in main.go, after parsing the index from the query, add it as a custom attribute (search for "CUSTOM ATTRIBUTE" in main.go):

trace.SpanFromContext(ctx).SetAttributes(attribute.Int("parameter.index", i))

Restart the app, make the sequence go, and find that field on the new spans.

Can you make the trace waterfall view show the index? What pattern does it show?

3. Create a Custom Span

Make the calculation into its own span, to see how much of the time spent on this service is the meat: adding the fibonacci numbers.

In fibHandler, surround the addition statement with a span start and end (search for "CUSTOM SPAN" to find it):

	tr := otel.Tracer("calculator")
	ctx, span := tr.Start(ctx, "calculation")
  // interesting code here
	defer span.End()

After a restart, do your traces show this extra span? Do you see the name of your method? What percentage of the service time is spend in it?

cartermp / intro-to-o11y-go