Chinese video giant Bilibili has just open-sourced a family of small language models called Index-1.9B. Plenty of tech companies release models, but Bilibili’s move stands out for its unusual transparency about training choices and variants, offering a valuable resource for anyone working on private AI infrastructure.

The family includes four versions. Index-1.9B-Base is a foundation model with 1.9 billion non-embedding parameters, trained on 2.8 trillion tokens—mostly Chinese and English. Index-1.9B-Pure follows the same recipe but strictly filters out all instruction-like data from the corpus, yielding an uncontaminated base ideal for custom fine-tuning without interference. Index-1.9B-Chat is aligned from the base model via supervised fine-tuning and direct preference optimization (DPO), ready for dialog. Finally, Index-1.9B-Character extends the Chat model with retrieval-augmented generation for few-shot role-playing customization.

Technically, the researchers employ a Warmup-Stable-Decay learning-rate schedule and substantially raise the concentration of curated data during the decay phase, while a Norm-Head output layer stabilizes training even at large learning rates. On a suite of standard benchmarks covering exams, reasoning, math, and code, Index-1.9B-Base scores an average of 64.92, competitive with or exceeding open models several times its size.

One of the most intriguing details is an unexplained surge in benchmark performance halfway through the constant-learning-rate phase. The anomaly raises scientific questions: what exactly flips inside the model at that moment? Understanding that phenomenon could lead to shorter, cheaper training cycles—crucial for anyone who trains or fine-tunes their own models.

For on-premise deployments, Index-1.9B is another step toward small yet effective LLMs. With fewer than 2 billion parameters and a sufficiently wide context window, the model can run on modest hardware—a consumer GPU with 8 or 12 GB of VRAM in FP16, depending on quantization—and low energy consumption, making sovereign AI architectures viable where data never leaves the corporate perimeter. It’s not just about privacy: it’s a TCO advantage when inference volume justifies buying dedicated machines rather than paying per token to a cloud provider.

The Pure variant, in particular, addresses a common need among enterprises that want to fine-tune on proprietary data (internal manuals, customer chat logs, corporate policies) without the model inheriting biases or response formats from public instruction datasets of variable quality. A clean base shortens the tuning cycle and reduces the risk of unwanted behaviors.

Bilibili is hardly a global LLM research powerhouse, yet this release fits into a broader movement: the proliferation of open, transparent, and efficiency-optimized models that progressively shift the innovation center of gravity from cloud-only mega-models to solutions deployable on local servers or at the edge. If the goal is to maintain full control over the AI you use, initiatives like this provide the raw materials—not just model weights but the “instruction manual” for replication. And that mysterious mid-training surge remains, a reminder that there is still much to discover in the depths of optimization.

The models and evaluation code are available on GitHub.