The assumption that a skill is a skill, independent of the model executing it, does not hold. In an analysis across seven models and three benchmarks, researchers identified 103 task-skill pairs where the same skill improves GPT-5.6-luna by 31.1 points but degrades Gemma-4-E4B by 8.8 points. The gap is not merely a scaling effect: it is an executability problem.

The most telling figure is different: in 86% of cases where weaker models fail, the violated requirement was already stated in the skill. It is not a knowledge gap. The model knows what it should do, but cannot turn that guidance into action. This changes how harness self-evolution should be designed: adding instructions or enriching skills is not enough; skills must be reformulated around the executor's capabilities.

To address this, the researchers separated failures into four areas: tool execution, context management, instruction following, and verification. From these diagnoses they distilled a fixed meta-skill that guides adaptation. A more capable external rewriter, Kimi-K3, adapted skills without execution evidence. The result for the three weaker executors was an improvement between 5.1 and 14.1 points over the original skills. In a five-round self-evolution loop, where each model revises its own skills from execution trajectories and verifier feedback, the meta-skill produced final improvements between 2.0 and 5.1 points across seven executors, from Gemma-4-E4B to Kimi-K2.5.

The emerging thesis is that skills are not portable artifacts across models, but components that need to be "compiled" for the executor. In agent frameworks, this shifts the problem from prompt engineering to an adaptation pipeline: the generic skill becomes an input, the model's capability profile a constraint, and the adapted skill the output. For teams deploying agents on smaller models in self-hosted or resource-constrained settings, the implication is direct: you cannot reuse the same skill catalog that works on a high-end model and hope that lowering the temperature or reducing the context will be enough. The difference is not just quantitative; it is qualitative in execution.

There is also a second-order effect. The described approach uses a more capable external rewriter to adapt skills. In an on-premise deployment, this raises an architectural question: must adaptation remain under your control, or is it acceptable for part of the pipeline to depend on an external model? The answer is not neutral with respect to data sovereignty, because skills and execution trajectories may contain sensitive operational details. A fixed and verifiable meta-skill, like the one described, is easier to audit than a completely free rewriting process, but it does not remove the constraint: if the rewriter is external, data still passes through it. For those evaluating these trade-offs in an on-premise context, AI-RADAR offers analytical frameworks at /llm-onpremise.

The structural message for the ecosystem is that harness self-evolution cannot be treated as a single routine that works for everyone. Variability across executors is a design problem, not noise to be removed. Those developing agent frameworks will likely need to separate the competence layer from the execution layer: the former can be shared, the latter must be adapted to the model. This distinction resembles, by analogy, the separation between intermediate representation and architecture-specific optimization in compilers. It is not an academic detail: it determines whether an agent built on a small model can perform a real task or fail even though it knows exactly what is being asked.