Recently, for a client project, I’ve found myself wanting to see how my traces render while developing my code locally. However, to test against their non-production observability stack (Grafana and Tempo deployed on AWS) I’d need to create a PR, then have a team member review and approve said PR, before I could plan and apply some terraform code to get my container deployed and testing (yes, I have proposed improvements to this). This is quite cumbersome; I want to get fast feedback and be able to tweak and change code without going through this cycle for every single change.

So, what are our other options?

The Console Exporter prints your spans to the console window, which is great for reviewing single spans, e.g. for correct attribute values. You could also import a trace into a Grafana instance if you just want to see what it looks like visually in Grafana. But this falls down when we want to see multiple spans visualised as a distributed trace, and it is cumbersome to review text in comparison to pretty visualised traces.

Another option is Grafana Cloud, which has a generous free tier and is relatively easy to set up. But there is one down-side - I need to set up a collector to collect and forward my spans, or else they aren’t guaranteed to reach Grafana Cloud’s instance of Tempo. I also value being able to spin up a local development environment, so I’d like something I can keep on my machine.

What I really wanted was a local instance of the LGTM stack we’ve deployed in our client site along with a collector that would process and export my OpenTelemetry data. I could build a simple version of the stack we have deployed to AWS but luckily I didn’t have to - this already exists! In the form of the otel-lgtm Docker image. This includes the following components:

• OpenTelemetry Collector
• Tempo Trace Database
• Loki Logs Database
• Prometheus Metrics Database
• Grafana

Getting started with the image is as simple as running the following command (assuming you already have Docker installed on your machine):

    docker run -p 3000:3000 -p 4317:4317 -p 4318:4318 --rm -ti grafana/otel-lgtm

This brings up each of the above components in turn. At that point you can use http://localhost:3000 to connect to Grafana, and you can send your OpenTelemetry data to http://localhost:4317 (GRPC) and http://localhost:4318 (HTTP). No further configuration is needed at this point, and I can run and test my application locally.

This is all done by the Grafana team and it’s open source; for more information see their Github repository and blog post.