Imagine two doctors: one has read every textbook, the other has spent years in the ward reading patient records. Which one recognizes a rare complication first? The same question, applied to models, is at the center of a study that systematically varied the ratio of didactic to clinical data when training medical LLMs. The answer is not binary: the type of data changes the profile of capabilities, not just the average score.

Researchers ran token-matched experiments, varying the proportion of textbooks and didactic material on one side and clinical notes and EHR data on the other. The main effect is asymmetric transfer: clinical data improves clinic-oriented tasks while remaining competitive on knowledge-intensive ones, whereas didactic data mainly boosts knowledge tasks. This is not a marginal difference: error analysis reveals a knowing-doing gap, where gains in knowledge recall do not reliably generalize to clinical reasoning.

The knowing-doing gap

This is the structural point. If knowledge does not automatically transfer to practice, then the value of a medical LLM cannot be measured only by factual accuracy benchmarks. Data composition becomes a design lever: for tasks requiring reasoning over real clinical data, the findings suggest that focusing collection and curation on clinical data yields most of the gains, even in modest amounts. The optimal ratio is not fixed; it depends on the knowledge and clinical reasoning demands of the downstream task.

This shifts the center of design: you no longer start from the available dataset, but from the use case. Vendors training generalist medical models mostly on didactic texts risk promising clinical competence that the data did not build. Conversely, those who control structured clinical archives — hospital networks, health authorities, research groups with access to records — hold the hardest-to-replicate resource. The competitive advantage here is not compute power, but data quality and data governance.

For teams evaluating on-premise deployment, this has a direct consequence. Processing patient records for training or fine-tuning imposes data sovereignty and compliance constraints, making local repositories a development asset rather than just a storage cost. The source provides no hardware specifications, but the principle applies to any pipeline: data quality and relevance outweigh hyperparameter optimization. AI-RADAR offers analytical frameworks at /llm-onpremise to evaluate trade-offs between data control, infrastructure cost, and management complexity.

The knowing-doing gap is, ultimately, the gap between a medical search engine and a clinician. Data composition decides which side of that line a model stands on.