Introducing the Pants OpenTelemetry Plugin
We are excited to publicly announce our open source Pants OpenTelemetry Plugin which brings the power of distributed tracing and observability to Pantsbuild workflows via OpenTelemetry.
Pantsbuild (or just "Pants") is an open source build orchestration tool designed especially for today's complex software development environments, supporting such diverse ecosystems as Python, Javascript, Go, and the JVM (Java and Scala). The Pants OpenTelemetry Plugin extends Pants to support emitting tracing data for build workflows to any OpenTelemetry-compatible monitoring system.
The Challenge: Visualizing Build Performance
As your codebase scales, understanding build performance becomes increasingly critical to managing the health of your entire software development pipeline. Traditional build systems often provide limited insights into the complex dependency graphs and parallel execution patterns enabled by modern tools like Pants. This is where distributed tracing comes in.
Pants has an internal tracing system called "work units." The work units subsystem is specific to Pants and is not yet officially documented. (As part of this project, we wrote up unofficial empirically-derived documentation of the work unit system.)
The Pants OpenTelemetry Plugin adapts the work unit tracing system in Pants to work with OpenTelemetry so that the tracing data already available in each Pants run is actually sent to tools which can make sense of that data.
Use Cases
There are several potential use cases for this data:
-
Understanding cache misses. One use is to understand why certain build actions did not make use of the Pants cache. The tracing metadata includes information about each process invoked for a build action. That metadata can be compared against metadata logged from prior Pants runs to understand what changed in a process invocation to cause a cache miss.
-
Understanding rule performance. Tracing spans are emitted for Pants rule invocations. That data can be used to understand Pants rule performance for potential optimization by maintainers and plugin authors.
Installing the Plugin
Visit the plugin's GitHub page for instructions on how to configure Pants to use the plugin. The instructions include a sample configuration for integrating Pants tracing with Honeycomb.io. The plugin should work with other OpenTelemetry-compatible systems.
The plugin is published to PyPI.
Need more help?
Do you have any questions about the plugin or Pants, or just need help getting started? Contact us - we're here to help you succeed with Pants observability or any other build orchesration challenges you have!