Aggressive quantization, specialized runtimes, and local fine-tuning are exposing new failure modes, from reasoning loops and off-topic digressions to runtime-dependent incoherence and hype-driven releases. Practitioners are starting to treat hallucination metrics and reproducibility as first-class checks for local deployment.
A refactoring task on an RTX 5070 Ti with Qwen3.8-27B in IQ3_XXS quantization produces a biographical digression about Lee Kuan Yew inside the reasoning. The episode shows how aggressive compression and speed can erode internal coherence, making trac...
An RTX 5070 Ti user running a Qwen3.8-27B model quantized to IQ3_XXS through Strata reports a 2-3x speed gain, but also an odd reasoning artifact: during a code cleanup task the model inserted a biography of Lee Kuan Yew. The case highlights how aggr...
A local fine-tuning project for AliceAI-80B-A3B shows the limits of on-premise training: 5 million synthetic tokens are not enough for a general conversational model. On three 32GB V100s, checkpoint 1.0 alpha responds but is underfit; the creator acc...
A field test with IQ3_S quantization shows that the same exact model can produce a coherent answer in llama.cpp and a repetitive loop elsewhere. The report challenges the idea that choosing the right model is enough: runtime, sampling and scheduling ...
A test on Ollama with models from 1.5B to 120B shows CPU-based exact-match entropy can detect hallucinations without using VRAM. On Qwen 27B AUROC reaches 0.89; on Mistral 7B it matches DeBERTa. But the collapse to 0.09 on 120B signals Confident Mode...
In September 2024, Reflection 70B was announced as an open-source model that supposedly surpassed GPT-4o. Those who downloaded it found it was essentially Llama 3.1. The episode remains a case study in how unverified claims can inflate quickly in Loc...
