Two GGUF files of the same model can show similar benchmarks and then diverge on a single token. Inside-llm, a newly published open-source project, makes that breaking point visible: it loads one or two GGUF models through llama.cpp and shows in the browser what happens during generation, position by position.
The visualization places two columns side by side, one per model, and uses brightness to indicate the probability of each token. A magenta mark signals where the two models would choose differently. You can work in lockstep mode, with both models reading the same text for a clean comparison, or in free mode, where each model follows its own picks until the paths split. The candidates panel shows the full distribution behind each choice, with mirrored bars, entropy, effective number of options, Jensen-Shannon divergence, and KL in both directions.
Then comes the sampler chain: penalties, top-k, top-p, min-p, temperature, and GBNF grammar, traced stage by stage to show how many candidates survive each filter and how the distribution changes. The chain and seed are configurable, so a session can be reproduced. The timeline shows entropy and divergence along the whole text, along with the tree of branches. From any candidate you can branch generation; every branch stays in the tree and jumping back to a node is exact thanks to KV-state snapshots, with nothing regenerated.
The logit lens function is available with --lens and shows what each layer would predict, both as a 2D map and as a 3D relief. The model picker lets you choose two models from a folder or download two quantizations of the same Hugging Face repository; the system only offers B models with the same tokenizer and estimates whether the pair fits in VRAM.
For self-hosted deployments, this changes how quantizations are read. It is no longer only about disk space or occupied VRAM: you can see where a more aggressive quantization introduces behavioral differences, and whether those differences stay confined to low-impact tokens or emerge at critical passages. That information matters in TCO calculations, because reducing precision can help a model fit in less VRAM, but degradation is not uniformly distributed: it needs to be observed at the point where it appears.
The project is built as a single C++17 process that links directly to llama.cpp. At each step it reads the full logits of model A and, on the same context, of model B, computes metrics and the sampler chain, and draws the token with its own seeded sampler. Binary frames travel over WebSocket to a Svelte 5 and Canvas 2D page; three.js is lazy-loaded only for the 3D relief. The compressed stream can be recorded and replayed in a static page with no engine, no model, and no GPU.
The packages are self-contained for Linux and Windows with CUDA or Vulkan: Vulkan covers AMD, Intel Arc, and NVIDIA. The process listens on 127.0.0.1, requires no accounts, and sends no telemetry. The limits should be read honestly: the two models must share the tokenizer, both must fit in VRAM, only one GPU is used at a time, iGPUs are never used, and without a dedicated GPU it falls back to the CPU, which is slow. The supported format is text only, no split multi-part GGUFs, and no embedding models.
The developer asks for feedback especially on which pairs to compare first: imatrix versus non-imatrix quantization, base versus fine-tuning or abliterated, Unsloth UD versus standard quantization. It is a less trivial question than it seems: the point where two quantizations diverge is often the detail that aggregate benchmarks do not tell.
๐ฌ Comments (0)
๐ Log in or register to comment on articles.
No comments yet. Be the first to comment!