Federated Ensemble Learning for Private Recommendation Systems

This technical report presents Federated Ensemble Learning (FEL), a novel framework designed to enhance the capacity of label-private recommendation systems. The document outlines the limitations of traditional federated learning (FL) in the context of recommendation tasks, particularly regarding the large memory requirements of modern neural networks. FEL addresses these challenges by proposing a method that clusters users based on public features and trains multiple smaller models simultaneously on these clusters. The results indicate that FEL can achieve significant improvements in model quality, with experimental data showing enhancements of 0.43% to 2.31% over standard FL approaches. The report details the three stages of FEL, including user clustering, leaf model training with global differential privacy, and the aggregation of these models into a larger server-side model. It also discusses the implications of label-only privacy settings and the potential for extending FEL to incorporate private features, ensuring that user data remains secure while improving recommendation accuracy.