Multi-Source Transfer Learning: A New Approach

Transfer learning is a fundamental technique for improving model performance, especially in data-scarce scenarios. A new study introduces a theoretical framework called Unified Optimization of Weights and Quantities (UOWQ) to address the challenges of multi-source transfer learning.

Joint Optimization of Weights and Quantities

UOWQ formulates multi-source transfer learning as a parameter estimation problem, based on an asymptotic analysis of a Kullback-Leibler divergence-based generalization error measure. The framework jointly determines the optimal source weights and optimal transfer quantities for each source task.

Theoretical Results and Practical Algorithms

The analysis demonstrates that using all available source samples is always optimal, once the weights have been adjusted correctly. The study also provides closed-form solutions for the single-source case and develops a convex optimization-based numerical procedure for the multi-source case. Based on the theoretical results, practical algorithms have been developed for multi-source transfer learning and multi-task learning.

Empirical Validation

Experiments on real-world datasets, such as DomainNet and Office-Home, demonstrate that UOWQ consistently outperforms strong baselines, validating both the theoretical predictions and the practical effectiveness of the framework.