Architectural Innovation in LLMs: K-Splanifolds for More Efficient Decoders
A researcher has experimented with a new LLM decoder architecture, replacing traditional MLPs with discrete lower-dimensional spline manifold geometry, as described in the K-Splanifolds paper. The 18-million-parameter model, trained on 5 billion toke...