LF AI & Data project helps developers turn agent traces into repeatable tests for more reliable agentic AI applications
LF AI & Data is pleased to announce that Monocle (sometimes referred to as “Monocle2AI”) has advanced from Sandbox to Incubating status, recognizing the project’s growing technical maturity, community development, and potential to help organizations build more reliable generative and agentic AI applications.
As AI applications become increasingly autonomous, developers face a fundamental challenge: understanding not only what an agent produced, but how it reached that result.
An agentic application may interact with multiple models, tools, data sources, vector stores, APIs, and sub-agents before producing an answer or completing a task. When something goes wrong, a final output rarely provides enough information to determine why. Even when an application appears to work correctly, teams need a consistent way to validate the agent’s actual behavior.
Monocle addresses this challenge by turning complex agent traces into repeatable tests, giving developers the visibility to verify internal logic and ensure agentic applications behave reliably at every step.
Moving Beyond Observability Dashboards
Many GenAI observability tools help teams collect traces and visualize application activity. Monocle goes a step further by turning those traces into tests.
Through its trace-based testing tools, developers can assert directly on the agent loop. Teams can validate which agents were invoked, which tool calls and sub-agent handoffs occurred, whether the expected parameters were passed, how much an interaction cost, whether errors occurred, whether an application stayed within defined performance or token limits, and embed evaluations to catch silent failures.
This enables developers to test agent behavior and not simply inspect it after the fact.
Instead of relying on a “run it and hope” approach, teams can incorporate agent-level validation into existing testing and CI/CD workflows. The same structured traces used to debug an application can also become repeatable evidence that the application continues to behave as intended.
That capability is increasingly important as agentic systems move from experiments into business-critical environments.
Open, Consistent Tracing for the GenAI Stack
Monocle provides an observability layer for GenAI applications built on OpenTelemetry. It automatically instruments key operations such as agent runs, model invocations, tool calls and retrieval queries, producing structured traces that can work with existing OpenTelemetry-compatible collectors and observability platforms.
The project’s metamodel creates a consistent representation of GenAI components (including agents, prompts, responses, tools and vector operations) across different frameworks and technology providers.
Monocle supports a growing range of agent frameworks, model providers, vector stores and application environments. Its low-code instrumentation options allow developers and platform teams to introduce tracing without extensively modifying application code.
The companion monocle-test-tools framework then allows teams to use those traces in pytest-style tests and CI/CD pipelines. Developers can validate individual interactions, multi-agent workflows and multi-turn sessions using the same telemetry generated during application execution.
Advancing as an LF AI & Data Incubating Project
Monocle joined LF AI & Data as a Sandbox project in 2024. Its advancement to Incubating status reflects the progress the project and its community have made since that initial contribution.
Incubating status recognizes projects that have demonstrated increasing maturity and are building the governance, contributor participation, documentation, release practices and technical foundation needed to support broader adoption.
Under LF AI & Data’s open and vendor-neutral governance, Monocle2AI can continue expanding its community while working with developers, platform engineers, site reliability teams, researchers and enterprises addressing the reliability of agentic AI systems.
The project’s roadmap includes support for additional programming languages, agent frameworks, model services and vector databases, along with richer test assertions and evaluation capabilities for complex and policy-driven AI applications.
Help Build More Reliable Agentic AI
As AI agents take on more complex responsibilities, observability alone will not be enough. Developers must be able to verify how agents reason, which resources they use and whether their behavior remains within expected boundaries.
Monocle provides an open source foundation for making that verification part of the development process.
LF AI & Data congratulates the Monocle maintainers and contributor community on this important milestone and invites developers, organizations and researchers to help shape the project’s next stage.
Explore Monocle on GitHub:
https://github.com/monocle2ai/monocle
Learn more about Monocle at LF AI & Data:
https://lfaidata.foundation/projects/monocle/
