The familiar weakness of static benchmarks is easy to miss: a large language model can answer a physics question correctly because it has seen similar formulas and patterns during training, without ever touching a mass, a spring, or an inclined plane. PhysMent tries to change the rules. The benchmark has models face classical mechanics scenes in the MuJoCo simulator and requires them to discover the necessary information by acting on the environment: applying forces, querying object states, advancing time, and modifying scene geometry. The initial data is not handed over; it has to be extracted through tool-mediated interaction.

The corpus spans 105 scenes organized into four difficulty regimes, three modalities, and a scene-manipulation category, scored across six dimensions. It is not a simple questionnaire: the model has to build an experimental procedure, interpret simulator feedback, and decide when it has enough elements to answer.

The results show a sharp divide. On qualitative single-concept tasks, performance remains reasonable, with accuracy up to 80 percent. But when precise quantitative measurement and multi-step sequences are required, most LLMs fall below 30 percent on the hardest category. Across the seven models evaluated, accuracy ranges from 25 to 67 percent. The most interesting finding is that the bottleneck is procedural rather than conceptual: failures come from premature answer submission, inefficient exploration, and inconsistent grounding in simulator feedback, not from gaps in mechanics knowledge.

That distinction has practical consequences for systems that couple LLMs to simulations or physical controls. A model that can state Newton's law but does not plan a sequence of experiments consumes tokens and compute cycles without converging. In a local deployment, this turns into operating cost and latency: every tool-calling step and every state readout traverses the runtime, holds memory, and lengthens the context. The problem is not solved simply by scaling model size; it requires interaction orchestration, persistent state management, and a deterministic feedback loop.

For teams evaluating on-premise deployments, the benchmark signals a shift in perspective: the LLM is no longer an isolated answer generator but an agent inside an environment. AI-RADAR offers analytical frameworks at /llm-onpremise to weigh these trade-offs, but the deeper point is that interactive benchmarks move attention away from the quality of a single answer and toward the robustness of the entire pipeline. In this sense, PhysMent does not only say which models do better; it shows which execution architectures make sense when the correct output depends on a process, not on a single insight.