Asking a large language model how long an oncology patient will live is no longer a lab experiment. It is a concrete possibility in consumer products, and the Survprompt framework turns it into a rigorous benchmark: it converts structured patient covariates into free-text clinical vignettes and asks pre-trained LLMs to produce zero-shot survival estimates. The comparison with random survival forests (RSF) on two multi-institutional pan-cancer cohorts — the public MSK-CHORD cohort and a Providence St. Joseph Health Network cohort built with an LLM-based medical abstraction framework — shows ambivalent results. GPT-5.6-Sol achieved cMAE within 10% of specialized RSF models for several cancer types and even beat it on prostate cancer in MSK-CHORD.

The c-index blind spot

The cMAE says little about the ability to rank patients. Frontier models showed a lower c-index: in practice, they can produce plausible mean times but fail the high-versus-low risk hierarchy. In oncology, that distinction is often more important than the absolute value, because it drives treatment intensity and resource allocation. A ranking error is harder to catch in clinical workflows: the estimate looks reasonable, but comparative decisions remain fragile.

Feature ablation analysis adds an important detail: the LLMs prioritized clinical variables similarly to specialized models. This is not a guarantee of reliability, but it signals that the behavior is not just a statistical artifact. Variability across cancer types and institutions remains the main limitation.

From prediction to data governance

The real cost of this scenario is not training. A zero-shot model eliminates specialized fine-tuning and lowers the entry barrier, but it shifts the burden to local validation and data sovereignty. A hospital sending clinical records to a consumer app or a cloud API moves outside its control perimeter; residency and consent constraints are not optional. The Survprompt framework can run on local infrastructure, but the Providence cohort shows that even abstracting clinical data through LLMs introduces non-neutral decisions. For those evaluating on-premise deployment, AI-RADAR offers analytical resources on /llm-onpremise to compare control, latency, and TCO trade-offs, but the choice remains tied to institution-specific requirements.

The structural lesson is that predictive accuracy is no longer the only criterion: validation on local data, with metrics such as cMAE and c-index computed locally, becomes the critical step. Generalist models can approximate a prognosis, but the infrastructure hosting them decides whether that approximation is governable.