Local Development with LLMs: A Decentralized Approach

The use of large language models (LLMs) locally is gaining popularity among developers seeking greater control over their data and processes. This approach, an alternative to the cloud, allows inference to be performed directly on your machine, opening up new possibilities for applications that require low latency and data sovereignty.

Tools for Local Development

Several tools are available to facilitate local development with LLMs:

  • OpenCode: A mature and complete solution, comparable to Claude Code and Codex.
  • Mistral Vibe: A simpler project than OpenCode, focused on ease of use.
  • Roo-Code: Integrates with Visual Studio Code for a smoother development experience.
  • Aider: A command-line tool (CLI) with a different approach.
  • Continue.dev: A plugin for Visual Studio Code, with some configuration difficulties with llama.cpp.
  • Cline and Kilo Code: Plugins for Visual Studio Code.

For those evaluating on-premise deployments, there are trade-offs to consider carefully. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these aspects.