Decentralized Federated Learning and Heterogeneity Management

Decentralized Federated Learning (DFL) is a serverless collaborative machine learning paradigm where devices collaborate directly to exchange model information. A significant challenge is the heterogeneity of data and models, arising from varying individual experiences and different levels of device interaction.

DecHW: A Novel Approach to Robust Aggregation

A new study introduces an innovative approach to address this heterogeneity, focusing on parameter-level variations in local models. The proposed method captures these variations and achieves robust aggregation of local updates, generating consensus weights via approximation of second-order information of local models on their respective datasets. These weights are used to scale neighborhood updates before aggregating them into a global neighborhood representation.

Experimental Results

Experimental results in computer vision tasks demonstrate strong generalizability of local models, achieved with reduced communication costs.