LLmFit is a command-line utility designed to simplify the selection of large language models (LLMs) based on a system's hardware specifications.

Features

The tool examines system resources, including RAM, CPU, and GPU, to determine which LLMs can be run efficiently. LLmFit evaluates models based on several parameters, including quality, speed, resource fit, and context size. It supports multi-GPU configurations, Mixture of Experts (MoE) architectures, and dynamic quantization selection.

Usage

LLmFit offers a default interactive text-based interface (TUI) and a classic CLI mode. This allows users to choose the approach that best suits their needs. For those evaluating on-premise deployments, there are performance and cost trade-offs that AI-RADAR analyzes in detail in the /llm-onpremise section.