A LocalLLaMA user reported an interesting behavior in the Devstral 2 123B and 24B models.

Hybrid Logical Reasoning

It appears that these models can be pushed into deeper and more consistent logical reasoning through the use of Jinja templates. By inserting the statement {%- set reasoning_content = 'High' %} into the template, a hybrid reasoning mode is activated, which improves the quality of responses.

Considerations

The user noted that the smaller version of the model (24B) may have difficulty exiting the thinking process, at least with some sampling settings. The larger version (123B) seems to work correctly. In general, the quality of responses obtained with this approach is significantly higher.

For those evaluating on-premise deployments, there are trade-offs between model size, hardware requirements, and reasoning capabilities. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these aspects.