A group of researchers tested the simplest possible idea for multimodal clinical prediction: take every piece of patient data – clinical notes, vital signs, lab values, comorbidities – and serialize it into a single natural language sequence. On that sequence, no task-specific fusion architecture, just end-to-end fine-tuning of a pre-trained large language model. The results, published in a preprint, overturn the assumption that you need dedicated encoders for each modality and complex combination mechanisms: unified serialization matched or outperformed specialized multimodal baselines in three clinically distinct tasks – in-hospital mortality on MIMIC-III, graft failure prediction using longitudinal data from a German transplant center, and emergency triage classification from ambulance records.

The tests compared various models: encoder-based ModernBERT and decoder-based ones like Llama 3.1, Gemma, DeepSeek-R1-Qwen, and Qwen3. In graft failure prediction, the serialized system also outperformed the gradient boosting model currently used in clinical practice for post-transplant management. The structural message is disruptive: simplicity wins, and by winning it removes friction points that have so far slowed AI adoption in real-world healthcare settings.

For those operating in hospital or clinical research environments, the benefit isn’t just academic. Traditional multimodal systems require multiple components – feature extractors, fusion mechanisms, task-specific preprocessing pipelines – that must be maintained, updated, and reconfigured for every new scenario. This translates into development, integration, and validation costs that push many institutions toward turnkey cloud solutions, with all the compromises on data sovereignty and regulatory compliance we know. Here, a single LLM fine-tuned on text suffices: the entire pipeline collapses into one model, simplifying orchestration, reducing technical debt, and making on-premise deployment not only feasible but pragmatic. The hardware requirements remain significant – the models used are mid- to large-sized – but eliminating heterogeneous architectures eases management and opens self-hosting scenarios where the entire clinical dataset stays under direct institutional control, aligning with GDPR and data residency requirements.

The approach also signals a paradigm shift in how we think about foundation models in medicine. Instead of engineering plug-in fusions for every signal combination, we can bet on the transformer’s ability to extract cross-modal relationships if data is simply presented as language. This not only speeds up experimental iteration – from months of engineering to hours of prompt preparation – but makes the system more adaptable to new wards, new diseases, and new clinical outcome definitions without rearchitecting anything. The proof is in the three very different tasks used in the study, with diverse data sources, temporal distributions, and predictive targets.

Operational maturity is still to be built: serialization fidelity, missing data handling, calibration, and clinical interpretability remain open challenges. But the signal for those evaluating local stacks is clear: research is shifting intelligence away from architectural complexity toward the quality of textual representation, a shift that favors deployments less tied to vendors and more compatible with privacy-by-design. For practitioners, AI-RADAR offers analytical tools at /llm-onpremise to weigh trade-offs among models, hardware, and total costs in self-hosting scenarios.