## Language Model Development: The Current Situation An AI enthusiast is documenting the process of building a language model (LM) from scratch. The latest update, regarding the sixth day of work, focuses on system stabilization and model training. ## Challenges and Solutions Initially, the use of DataParallel on Windows created bottlenecks, slowing down training compared to using a single GPU. Despite this, the developer chose to continue working on Windows to make the process accessible to beginners. Training required more resources than expected: after 25,000 steps, the model was trained on approximately 400 million tokens, an insufficient number for a model of this size. ## Preliminary Results and Future Prospects Despite the limited amount of training data, the model has shown promising results, with good sentence structure. However, more in-depth training is needed, with approximately 300,000 steps, to obtain a quality pre-trained model. The author plans to have a benchmark ready by the eighth day, to showcase the model's capabilities.