The problem with generative query suggestions is familiar to anyone working on search engines or conversational assistants: a list can contain items that are individually valid but redundant. When five reformulations point to the same intent, the user gets no added value. The framework proposed in the source tackles this short-circuit with a two-stage optimization.

The first stage builds intent-aligned supervised fine-tuning data and introduces a specific diversity reward, called Intent-Aware Diversity Reward. The second stage assigns credit at the query level: quality signals are routed to the tokens of the corresponding query, while the diversity signal is shared across the entire list. In tests on a production dataset, both in online A/B testing and offline evaluation, the framework improves click-through rate, query quality, and intent coverage.

Why credit assignment is the key point

Credit assignment is not an implementation detail. In reinforcement learning applied to language models, the reward often arrives at the sequence level: the model knows whether a response is good, but not which tokens contributed to the result. Here the problem is twofold: there is a quality signal for each query and a diversity signal for the entire list. Treating everything as a single reward risks confusing the two objectives.

Routing quality to the query tokens and sharing diversity at the slate level prevents the model from learning to maximize diversity at the expense of relevance. In practical terms, the system understands that a specific suggestion was clicked because of certain words, while the list as a whole must cover distinct intents. This separation is closer to how product teams think about metrics: CTR measures the individual suggestion, coverage measures the list.

The challenge for those who keep data in-house

For those evaluating on-premise or self-hosted deployment, the framework touches a sensitive point: the quality of the first stage depends on the availability of real search logs and the alignment of intents. Interaction data is often among the most sensitive an organization owns. Keeping it outside the corporate perimeter is not always compatible with sovereignty or compliance requirements. A system that requires fine-tuning on these logs, as described, turns proprietary data into a direct competitive factor.

The TCO in this case is not limited to hardware for inference. It also includes the data preparation pipeline, training the reward model, and managing click signals. These are costs that cloud providers tend to hide inside managed services, but they resurface when deployment is local. AI-RADAR provides analytical frameworks at /llm-onpremise to evaluate these trade-offs without reducing them to simple price comparisons.

The direction is clear: as generative suggestions become part of internal search engines, the gap between those who can train close to their own data and those who must outsource will widen. The framework itself introduces no new hardware constraints, but it shifts attention to data and the ability to turn it into training signal.