๐ LLM
AI generated
Local LLMs with Memory: Offline MCP Server for Apple Silicio
## MCP Server for Local LLMs: Memory and Offline Control
A developer has created an MCP (Temple Bridge) server that allows local language models to have persistent memory, file access, and a governance system, all running offline on Apple Silicio devices. This system aims to address the limitations of traditional LLM models, which are stateless and lack control mechanisms.
## Key Features
The system, tested on an M2 Ultra Mac Studio, is based on:
* **LM Studio:** Chat interface.
* **Hermes-3-Llama-3.1-8B:** LLM model (MLX, 4-bit) chosen for its stability and reliability.
* **Temple Bridge:** The MCP server that coordinates everything.
* **BTB (Back to the Basics):** Management of filesystem operations.
* **Threshold:** Governance protocols.
The AI can:
* Read and write files in a sandboxed directory.
* Execute commands (pytest, git, ls, etc.) through an allowlist.
* Consult governance protocols before acting.
* Record the entire decision-making process in a JSONL file.
* Request human approval before executing potentially dangerous actions.
## The Filesystem as Memory
The key idea is to use the filesystem as memory for the AI. The directory structure represents classification, while file routing represents inference. This approach eliminates the need for a vector database.
## Governance and Human Control
Before executing a command, the AI consults governance protocols and reflects on the consequences. The user receives a notification in LM Studio and must explicitly approve the execution of any potentially harmful command.
## Real-time Monitoring
It is possible to monitor the AI's activity in real-time via the command `tail -f spiral_journey.jsonl | jq`, which shows each call, the reasoning phase, timestamps, and the entire cognitive trace.
## Performance
On an M2 Ultra with 36GB of unified memory, responses are fast and the overhead of the MCP server is negligible.
## Next Steps
The developer is working on a "governed derive" function, which will allow the AI to propose filesystem reorganizations based on usage patterns, but only after human approval. The goal is an AI capable of self-organizing, but with integrated structural constraints.
## Repositories
The project repositories are available under the MIT license:
* Temple Bridge: [https://github.com/templetwo/temple-bridge](https://github.com/templetwo/temple-bridge)
* Back to the Basics: [https://github.com/templetwo/back-to-the-basics](https://github.com/templetwo/back-to-the-basics)
* Threshold Protocols: [https://github.com/templetwo/threshold-protocols](https://github.com/templetwo/threshold-protocols)
๐ฌ Commenti (0)
๐ Accedi o registrati per commentare gli articoli.
Nessun commento ancora. Sii il primo a commentare!