## Deterministic inference in LLMs: a challenge Non-reproducibility in the inference of large language models (LLMs) is a problem that can compromise the reliability of results. Even under seemingly deterministic conditions, variations in batch size can lead to different outputs. ## DetLLM: the open source solution To address this challenge, a developer has created DetLLM, a tool that measures and proves the repeatability of inference at the token level. DetLLM generates detailed traces and a diff of the first divergence, creating a minimal reproduction package for each run. This package includes an environment snapshot, the run configuration, applied controls, traces, and a report. The DetLLM code is available on GitHub (https://github.com/tommasocerruti/detllm) and the developer invites the community to provide feedback and report any prompts, models, or setups that still show divergences.