We used to think that evaluating a code agent required a single bit: task passed or failed. AgentLens breaks that bit into a full spectrum of observable behaviors along the entire path—from the first instruction to the final output, covering tool usage, self-verification, error recovery attempts, and even communication style with the user.

The project, released as open source on GitHub, is not yet another model ranking. It pairs formal verification, where an objective check exists, with LLM-written trajectory reviews and side-by-side comparisons, so every run yields a readable explanation of the score. The idea is simple but powerful: people who actually use these agents don’t see just the final result—they experience everything that happens in between. And those details make the difference between a reliable agent and one that hits the target by accident.

For teams developing or evaluating agents in production, the leap is significant. It’s no longer just post-mortem diagnostics: the trajectory becomes the primary regression indicator. The AgentLens team already uses it in a nightly evaluation pipeline to compare successive versions of their own agent and catch behavioral degradations before they reach users. It’s a kind of CI/CD for interaction quality, not just for generated code.

But the paradigm shift becomes structural in on-premise scenarios. When an agent runs on proprietary code or sensitive data, the system administrator needs far more than a binary verdict: they must understand how the agent worked, whether it respected corporate policies, whether it used tools appropriately, or risked exposing data. A trajectory-based approach transforms the agent from a black box into an auditable process, with immediate benefits for compliance, incident response, and trust.

It’s no coincidence that AgentLens arrives as LLM providers push toward increasingly autonomous agents, while enterprises adopting them locally demand control guarantees. Opening up the trajectory is not just a technical choice: it incentivizes building more transparent agents that justify each step rather than hiding behind a “task completed” message. For those evaluating on-premise deployment strategies, AI-RADAR offers analytical frameworks to weigh these trade-offs (see /llm-onpremise).

The signal for the industry is clear: a code agent’s maturity will no longer be measured only on closed benchmarks, but on its ability to narrate and verify every single move. With AgentLens, evaluation becomes a dialogue between human, machine, and process—and the score is no longer a number, but a story.