The Kyojin project has demonstrated that two MoE models in the 300-billion-parameter class can run on a single 128 GB mini PC equipped with AMD Strix Halo (gfx1151) and ROCm. Measured on a Ryzen AI Max+ 395, GLM-5.3-Flash reaches 580 tokens per second in prefill at 3.5K context and 546 at 64K; MiMo-V2.6-Flash-MOPD hits about 650 tokens per second at 4K. In decode, GLM runs between 26 and 30 tokens per second with MTP, while MiMo reaches 32 on prose, 35 on chat, and 44 on code in speculative mode, dropping to 29 without speculation.

The structural point is not absolute speed, but that the EXL3 weights take 99.7 GB for GLM and 105 GB for MiMo, so each fits into the unified memory of a 128 GB machine. For years, models of this class required multi-GPU servers with aggregate VRAM well above 100 GB. Kyojin shifts the constraint: what makes local deployment impractical is no longer VRAM capacity, but quantization quality and available memory bandwidth. In an MoE model only a fraction of experts is active per token, so the real bottleneck is not pure compute but how quickly weights are read from memory.

The trade-off is measurable. The KLD distance from the FP8 version is 0.151 for GLM and 0.0713 for MiMo, with top-1 agreement of 89.3% and 92.0% respectively. The GLM pack comes from a mix of turboderp's public EXL3 tensors at 2.05 and 3.05 bits per weight, with a layer mix and a tuning stage developed by the team. On the same 129 rows, turboderp's smaller 85 GB pack records a KLD of 0.275 and decodes about 10% faster; Kyojin's 100 GB mix drops to 0.190 but pays in speed. This is the classic size-quality-latency triangle made explicit.

The project's openness is partial but not trivial. The Kyojin engine is built on turboderp's ExLlamaV3, with ROCm contributions from sdougbrown and vcruz305, and is available on GitHub alongside the models on Hugging Face. The conversion pipeline, however, remains private. This creates tension for anyone evaluating on-premise or air-gapped deployment: on one hand, there is an open runtime and downloadable weights, so more verifiability than cloud endpoints; on the other, without the conversion pipeline it is impossible to independently reproduce or validate the quantization process. For data sovereignty, having the engine and weights locally is still a step forward, but full transparency is still distant.

The -Uncensored variants add another layer. The same weights are accompanied by a small file that the engine applies at load, with a switch to turn it off. They have not been measured yet. In an enterprise context, a removable safety switch is not neutral: it facilitates experimentation, but forces organizations to carefully track which configuration runs in their environments, especially in regulated sectors.

At the market level, the project signals that AMD's Strix Halo platform becomes interesting not for compute peaks but for its large unified memory pool. A 128 GB mini PC could reduce CapEx and footprint compared to a multi-GPU server for certain inference workloads, even if decode speed below 50 tokens per second makes it unsuitable for high-concurrency interactive loads. The fact remains that software is decisive: ExLlamaV3, ROCm, and community kernels are what make the result possible, not just the hardware. The next question, posed by the developers themselves, is which models to bring next and what tokens per second users get: an invitation to turn a technical announcement into a distributed testbed for local deployment of frontier models.