The LLM Dilemma: Knowledge vs. Agency

A user in the LocalLLaMA community raised a crucial point regarding the current direction of large language model (LLM) development. The initial interest in LLMs was linked to their ability to provide personalized and contextualized knowledge, surpassing the signal-to-noise ratio of traditional search engines.

Prioritizing Agency and the Trade-off on Knowledge

Currently, development seems to focus on increasing the agentic capabilities of models, that is, their ability to act autonomously to achieve specific goals. The user fears that this emphasis on agency, given the limited capacity of the models (number of parameters), may compromise performance in other areas, particularly in the ability to store and retrieve knowledge.

In Search of a "Knowledgeable" Model

The question posed is whether there are labs or initiatives focused on developing simple models, but with vast knowledge. The goal would be to create an offline and omniscient alternative to Wikipedia, based on LLMs, that can be consulted to obtain accurate and reliable information.