Continual Fine-Tuning with Accurate Task Retrieval

A new study published on arXiv presents an innovative method for the continual fine-tuning of pre-trained models. The goal is to adapt a model to new tasks sequentially, while maintaining performance on previous tasks, for which the data are no longer available.

The proposed approach combines the advantages of two existing categories: input-adaptation and parameter-adaptation. Input-adaptation methods rely on retrieving the most relevant prompts at test time, but require continuously learning a retrieval function that is prone to forgetting. Parameter-adaptation methods, instead, use a fixed input embedding function to enable retrieval-free prediction and avoid forgetting, but sacrifice representation adaptability.

The new technique introduces a parameter-adaptation method that enables adaptive use of input embeddings during test time with parameter-free retrieval. Task-retrieval error bounds are derived for clustering-based task retrieval, providing theoretical guarantees that link low retrieval error to structural properties of task-specific representation clusters. This reveals a fresh insight into how a well-organized clustering structure enables reliable retrieval.

The method is designed with two key components: (i) an adaptive module composition strategy that learns informative task-specific updates to preserve and complement prior knowledge, and (ii) a clustering-based retrieval mechanism that captures distinct representation signatures for each task, enabling adaptive representation use at test time. Extensive experiments show that these components work synergistically to improve retrieval and predictive performance under large shifts in task semantics.