OptiML: A Comprehensive Approach to CUDA Kernel Optimization

Generating high-performance CUDA kernels is a complex task, requiring the exploration of a large space of low-level transformations. OptiML addresses this challenge with an end-to-end framework that combines large language models (LLMs) and search techniques to improve CUDA kernel performance.

OptiML operates in two distinct stages. In the first stage, OptiML-G, a generator based on a Mixture-of-Thoughts model, creates an initial executable program from a natural language description. In the second stage, OptiML-X, a search-based optimizer, refines the kernels, whether synthesized or user-provided, using Monte Carlo Tree Search (MCTS) driven by LLMs.

Each candidate transformation is compiled, verified, and profiled with Nsight Compute. Performance is evaluated using a composite objective function that combines runtime with hardware bottleneck proxies and guardrails against regressions. The results demonstrate that OptiML is able to discover verified performance improvements over established LLM baselines and to produce interpretable optimization trajectories based on profiling evidence.

For those evaluating on-premise deployments, there are trade-offs to consider. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these trade-offs.