An enthusiast has implemented a solution to replace Anthropic models in Claude-Code with NVIDIA NIM models, leveraging a free API that allows up to 40 requests per minute.

Implementation Details

The implementation acts as middleware between Claude-Code and NVIDIA NIM, offering an alternative for language model inference. The user also replaced the Claude mobile app with Telegram, allowing tasks to be sent and work to be viewed autonomously.

Key Features

Among the distinctive features of this implementation, the preservation of reasoning tokens between tool calls stands out, allowing models like GLM 4.7 and Kimi-K2.5 to fully exploit the context of previous interactions. There is also a fast prefix detection system for bash commands, which avoids sending classification requests to the LLM, speeding up execution. Rate limiting and session concurrency management mechanisms are integrated. The modular architecture of the code facilitates the addition of further providers or messaging applications.

For those evaluating on-premise deployments, there are trade-offs to consider. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these aspects.