Found-RL: Reinforcement Learning and Foundation Models for Autonomous Driving

A new study introduces Found-RL, a platform designed to integrate foundation models into Reinforcement Learning (RL) to enhance autonomous driving capabilities. The goal is to overcome the limitations of efficiency and semantic interpretability that plague traditional RL systems in complex scenarios.

Architecture and Key Components

Found-RL is based on an asynchronous batch inference framework, which decouples the reasoning of Vision-Language Models (VLMs) from the simulation loop. This approach solves the latency issues that hinder real-time learning. The platform includes supervision mechanisms such as Value-Margin Regularization (VMR) and Advantage-Weighted Action Guidance (AWAG) to transfer the capabilities of expert VLMs to RL policies. CLIP is also used for dense reward shaping, with a Conditional Contrastive Action Alignment mechanism to overcome CLIP's limitations.

Performance and Availability

The results show that a lightweight RL model, integrated into Found-RL, can achieve performance comparable to that of VLMs with billions of parameters, while maintaining a real-time inference of approximately 500 FPS. The code, data, and models will be made publicly available on GitHub.