The bottleneck in multi-agent systems is not always the model: it is often memory. When an LLM explores in parallel by spawning sub-agents, each branch tends to duplicate prefix KV caches; findings are exchanged by re-prefilling text summaries, and cross-branch dependencies get flattened or dropped during reinforcement learning. Octopus, a single-model multi-agent reinforcement learning framework, tackles the problem at the root: all threads share one backbone and coordinate over a GPU-resident directed acyclic graph (DAG) of KV states through five primitives: fork, read_thread, wait_threads, rewind, and commit.

The first mechanism, Zero-Copy Radix Forking, creates branches as O(1) cursor views over shared physical KV pages. The second, Attention-Only KV Mounting, mounts a sibling thread’s terminal KV blocks directly into full-attention layers while bypassing linear-recurrent states of hybrid architectures such as Gated DeltaNet and Mamba-2: off-task hijacking drops by 21 times (from 57.9% to 2.7%) and full-text mutual information remains at 92.2%, with zero extra KV allocation. The third, Packed DAG-GRPO Trainer, packs all episode branches into a single block-sparse attention pass and assigns counterfactual branch credit via leave-one-branch-out marginal rewards.

The numbers tell a precise story. Trained end-to-end with GRPO on Qwen3.5-4B over a sparse BM25 corpus, Octopus reaches 49.6% accuracy on BrowseComp-Plus: a 17.5-point gain over single-agent GRPO at 143K context and a 12.5-point gain over text-summary multi-agent RL. On OpenResearcher it reaches 51.2%, outperforming dense-retrieval agents up to 250 times larger. Training throughput rises by 3.3 times and peak KV memory falls by 2.8 times.

For anyone evaluating self-hosted stacks, the structural signal is clear: competition is shifting from model size to how efficiently a system manages shared state and memory. Reducing peak KV cache changes the required hardware profile, lowers VRAM pressure, and can make complex multi-agent setups feasible on a single GPU without moving to distributed configurations. Octopus also highlights a rarely discussed constraint of hybrid attention-recurrent approaches: when full-attention and linear-recurrent states mix, cross-branch transfer can introduce interference that selective mounting contains. This matters for teams designing on-premise inference and training pipelines, where every gigabyte of VRAM and every compute pass weighs on TCO. For those evaluating on-premise deployments, the trade-offs between memory, throughput, and TCO are exactly the ground covered by AI-RADAR’s analyses at /llm-onpremise.