The challenge in training agents is no longer just the model: it is everything around it. Microsoft Research Asia has open-sourced Agent Lightning 1.0, a reinforcement learning framework of roughly 3,500 lines that reverses the traditional paradigm: instead of forcing teams to reimplement the agent inside the training framework, it lets the harness used in production participate directly in the RL loop. The framework sits between the agent and the model through an OpenAI-compatible proxy: model calls remain the same, while the system records the prompts, responses, and log probabilities needed for training.

The change is not cosmetic. In traditional agentic systems, the training framework owns the loop with the environment: the model generates an action, the environment returns an observation, and the whole rollout is mapped to a single token trajectory. That works with relatively simple systems, but not with real harnesses such as mini-SWE-agent, OpenHands, or Codex, which manage context, tools, and execution logic on their own. Reimplementing them inside the training framework is expensive and produces an agent that no longer matches the one deployed. Agent Lightning shifts control: the harness manages the interaction with the environment, while the framework observes and collects samples. In v1.0 this is formalized as Harnessed Agentic RL.

The system has three components: an API Gateway that acts as an LLM proxy, a Rollout Controller for agent execution, and a customized trainer built on verl. This architecture introduces four concrete difficulties: retokenization, rollout-level advantage calculation, loss normalization, and scheduling on fixed GPU resources. The issue is not theoretical: if a rollout produces more samples, a sample-level average distorts the statistical contribution of that rollout. The researchers show that handling advantage and normalization at the rollout level yields a higher validation reward and more stable policy entropy.

On the infrastructure side, Agent Lightning 1.0 is Kubernetes-native. Agents run as standard Kubernetes jobs on self-managed clusters, cloud Kubernetes, or local infrastructure. Commercial sandboxes like Modal or E2B are not required: for teams that already have on-prem compute or shared clusters, this lowers the marginal cost of large-scale rollouts and keeps code and data within boundaries the organization controls. It is a structural signal: lightweight frameworks that integrate with Kubernetes lower the barrier to running agent RL without outsourcing execution.

Agent Lightning also introduces Collocated Async RL, which lets rollout and updates share the same set of GPUs. In reported tests, the end-to-end gain is about 2x over synchronous RL, using fewer GPUs than conventional asynchronous RL. Beyond efficiency, alternating rollout and update on the same hardware makes agent training more compatible with moderate local infrastructure.

The test on an end-to-end coding pipeline based on Qwen3.5-9B and the SWE-smith dataset lifted Pass@1 on SWE-bench Verified from 41.8% to 56.4%, an absolute gain of 14.6 percentage points with about 6,000 samples. The notable part is not the benchmark itself, but the data-to-result ratio: large-scale compute is not required for measurable improvements.

Who benefits? Teams that already have self-managed Kubernetes clusters, labs with moderate resources, and organizations with data residency constraints. Who loses? Commercial sandbox providers that until now offered the fastest way to scale rollouts without dedicated infrastructure. The direction is clear: agent training is moving away from closed cloud services toward open-source components that run where the organization decides, with predictable cost. For those evaluating on-premise deployment, AI-RADAR offers analytical frameworks at /llm-onpremise to assess these trade-offs, but the choice still depends on internal skills and specific goals.

The code is available on GitHub along with the technical report. Together with the Kubernetes choice, that detail shifts the comparison from benchmark numbers to reproducibility.