In model merging, the point where you compute the directional derivative is not a mathematical detail: it separates a useful prediction from one nearly orthogonal to actual changes. The work shows that at the receiver model the derivative forecasts interference, halves the accuracy error of a no-change baseline, and identifies most examples whose predictions flip. At the pretrained backbone, however, the derivative is nearly orthogonal to realized changes in vision experiments and only partly predictive in LoRA language-model pilots.

The explanation is not parameter distance but displacement along the receiver's own task vector: the mismatch appears within the first few dozen fine-tuning steps. Freezing receiver-local responses produces an affine surrogate in the merge coefficients. On top of that, LR-Merge corrects coefficients through convex teacher matching on unlabeled probes, without backpropagation through merged networks.

For self-hosted stacks, the advantage is not only academic. Model merging combines multiple fine-tuned models on a shared backbone, reducing the number of models to serve and, in turn, pressure on VRAM and TCO. LR-Merge makes coefficient search cheap: once the surrogate is built, it answers new task-subset queries in seconds. This shifts the incentive from hosting a fleet of specialized models to recomposing behavior from available task vectors with limited compute overhead.

The critical point is that prediction at the shared backbone fails. Practitioners using standard merging recipes anchored to the backbone risk optimizing coefficients on the wrong geometry. The paper's controls link the failure to displacement along the receiver vector, not simply parameter distance. In practice, comparing weights is not enough; local responses matter. For on-prem deployments, this adds a preparation step but does not require backpropagation through fused networks, so it remains compatible with hardware that lacks large training clusters.

A less visible aspect concerns data. LR-Merge is label-free: it uses unlabeled probes. In data sovereignty or compliance contexts, this reduces dependence on annotated datasets, which are often expensive or sensitive. It does not remove the need for validation, but shifts cost from labeling to collecting unlabeled probes, easier to maintain on-premise. Structurally, it signals that merging-coefficient correction is becoming a convex optimization problem separate from training, potentially a standard component of model merging pipelines for local LLM operations.

The fact that LR-Merge matches exact teacher matching through a merged network but converges roughly three times faster at equal compute budget suggests the bottleneck is evaluation cost, not correction quality. Once built, the surrogate makes the problem practical. The open question is whether the approach holds when task vectors come from very different model families.