The reference to Socrates in the paper's title is not decorative. Sieve and Sage starts from a distinction that in the practice of Retrieval-Augmented Language Models is often flattened: a system can fail because the evidence is absent from retrieved documents, or because the evidence exists but is contaminated by conflicting, negated, or adversarial information. The paper calls the first condition unanswerable and the second distracted. Failing to separate them is expensive: the generative model is invoked even when the context is noisy, with the double risk of hallucinations and wasted compute.

The proposed approach is a two-stage pipeline. Sieve, a lightweight module, screens the retrieved document set and identifies distracting noise before Sage, the actual LLM for grounded generation and abstention decisions, is called. Evaluated across general and high-stakes expert domains, the framework improves accuracy by up to 69.4 percentage points and Macro-F1 by 55.2 points compared to one-stage baselines. The speedup reaches up to 1.99x.

The separation between screening and generation has implications deeper than speedup. In self-hosted deployments, where every call to a large LLM consumes VRAM and compute time, shifting noise control to a lightweight module reduces the number of expensive inference calls. This is not just about latency: it changes the TCO profile, because the marginal cost of abstention becomes lower. A system can afford to be cautious without driving up computational spending.

This architecture signals a structural direction: retrieval failure handling does not have to be assigned to a single monolithic model. Decomposing failures into distinct states creates a clear boundary between context quality control and generation. In regulated or on-premise settings, that can mean greater auditability: the filter that decides whether the context is trustworthy can be inspected, updated, and sized separately from the generative model. It shifts incentives for those building RAG pipelines: value is no longer only in the largest model, but also in the ability to discard unusable context early.

For those evaluating on-premise deployment, the trade-off between inference cost and data sovereignty becomes more concrete. AI-RADAR offers analytical frameworks on /llm-onpremise to compare similar architectures.

Like Socrates, a RALM should know when the context does not allow it to answer. The difference is that now it can do so without paying the full cost of the LLM every time.