Awesome Federated Computation Systems Papers

A curated list of FL system-related academic papers, articles, tutorials, slides and projects. Star this repository, and then you can keep abreast of the latest developments of this booming research field.

Papers with 🎓 have been peer-reviewed and presented in academic conferences.

Table of Contents

• Awesome Federated Computation Systems Papers
  • Table of Contents
  • FL Systems from big tech companies
    • Paper
    • Framework
    • Vertical FL
  • Open-source FL Framework
  • Edge / Mobile
  • Federated Computation Systems
  • Optimization for FL Systems
  • Security and Privacy
  • Real-world FL Application
  • Real-world device traces
  • Survey
  • General insight for FL
  • Other FL paper list

FL Systems from big tech companies

Paper

Cross-device

• Apple: Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications | PDF, PDF
• Google: Towards Federated Learning at Scale: System Design | MLSys21, Github🎓
• Meta: Papaya: Practical, Private, and Scalable Federated Learning | MLSys22 🎓
• Microsoft: FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations | PDF, Github
• Alibaba-1: FederatedScope: A Flexible Federated Learning Platform for Heterogeneity| PDF
• Alibaba-2: FederatedScope: FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning |KDD22 🎓

Federated Analytics

• LinkedIn: LinkedIn's Audience Engagements API: A Privacy Preserving Data Analytics System at Scale | PDF
• Alibaba-3: Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning | PDF, Github 🎓

Cross-silo

• IBM: IBM Federated Learning: An Enterprise Framework White Paper | PDF, Github
• Nvidia: Federated Learning for Healthcare Using NVIDIA Clara | PDF, Github
• WeBank: Federated Learning White Paper V1.0 | PDF, FATE, KubeFATE, FATE-FLOW, FATE-LLM

Framework

• Cisco: Flame | Github
  • Federated Learning Operations Made Simple with Flame
• OpenMined: PySyft | Github
• Baidu: Paddle | Github
• ByteDance: Fedlearner | Github
• Meta: FLSim | Github
• Ant: SecretFlow | Github
• ZTE: Neursaf FL | Github

Vertical FL

• OpenMined: PyVertical Github, PDF

Open-source FL Framework

• FedScale: Benchmarking Model and System Performance of Federated Learning | ICML 22 🎓
• EasyFL: A Low-code Federated Learning Platform For Dummies
• Flower: A Friendly Federated Learning Research Framework
• Sherpa: Federated Learning and Differential Privacy Framework: Protect user privacy without renouncing the power of Artificial Intelligence
• FedML: A Research Library and Benchmark for Federated Machine Learning
• LEAF: A Benchmark for Federated Settings | NeurIPS 19 🎓
• FedEval: A Benchmark System with a Comprehensive Evaluation Model for Federated Learning
• OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework
• FEDn: A scalable, resilient and model agnostic hierarchical federated learning framework. - Paper
• Rosetta: A Privacy-Preserving Framework Based on TensorFlow
• FedLab: A flexible Federated Learning Framework based on PyTorch, simplifying your Federated Learning research.

Figure 1: Framework Functionality Support

FL x LLM

• FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning

Edge / Mobile

• Google: TFlite | Github, Github
• Alibaba: MNN | Github
• MIT: Tiny Training Engine | Github
• Private Compute Core Architecture
• Enabling conversational interaction on mobile with LLMs | Google Blog

Federated Computation Systems

• Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning | OSDI 22 🎓
• Device-centric Federated Analytics At Ease
• λ-FL : Serverless Aggregation For Federated Learning | AAAI 22 🎓
• Characterizing and Optimizing End-to-End Systems for Private Inference | ASPLOS 23 🎓
• STI: Turbocharge NLP Inference at the Edge via Elastic Pipelining | ASPLOS 23 🎓

Optimization for FL Systems

• Oort: Efficient Federated Learning via Guided Participant Selection | OSDI 21 🎓
• Mistify: Automating DNN Model Porting for On-Device Inference at the Edge | NSDI 21 🎓
• Pisces: Efficient Federated Learning via Guided Asynchronous Training | SoCC 22 🎓
• Resource-Efficient Federated Learning | EuroSys 23 🎓
• Federated Learning with Buffered Asynchronous Aggregation | AISTATS 22 🎓
• Hermes: An Efficient Federated Learning Framework for Heterogeneous Mobile Clients | MobiCom 21 🎓
• PyramidFL: A Fine-grained Client Selection Framework for efficient Federated Learning | MobiCom 22 🎓
• Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning | Google
• Auxo: Heterogeneity-Mitigating Federated Learning via Scalable Client Clustering | SoCC' 23
• Pisces: efficient federated learning via guided asynchronous training| SoCC' 22
• GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learning | MLSys' 23
• Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search | CVPR 20 workshop 🎓
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients | ICLR 21 🎓
• FedorAS: Federated Architecture Search under system heterogeneity | Samsung
• Venn: Resource Management Across Federated Learning Jobs | Umich

Energy-efficiency

• Green Federated Learning | Meta
• A first look into the carbon footprint of federated learning | Flower

Security and Privacy

Security

• SIMC: ML Inference Secure Against Malicious Clients at Semi-Honest Cost PDF
• Secure Federated Learning for Neuroimaging PDF

incoming

Privacy

• The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation PDF
• Differential Privacy reading list | Github

incoming

Real-world FL Application

• Google keyboard query suggestions PDF (2018)
• Google mobile keyboard prediction PDF
• Google Out-Of-Vocabulary Words PDF
• Google Emoji Prediction in a Mobile Keyboard PDF
• Google Training Speech Recognition Models (2021) PDF
• Google Federated Learning of Gboard Language Models with Differential Privacy PDF
• Advancing health research with Google Health Studies (2020) Website
• Federated Evaluation of On-device Personalization PDF

Real-world device traces

• Mobile AI benchmark Website
• Mobile Access Bandwidth in Practice: Measurement, Analysis, and Implications Website
• Real-world data partition FL dataset | FedScale Website
• Mobile availability (client behavior) trace | Characterizing impacts of heterogeneity in federated learning upon large-scale smartphone data. Website

Survey

• UNIFED: A Benchmark for Federated Learning Frameworks
• Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• A Field Guide to Federated Optimization
• Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
• Advances and Open Problems in Federated Learning

General insight for FL

• On Large-Cohort Training for Federated Learning | NeurIPS 2021 🎓
• What Do We Mean by Generalization in Federated Learning? | ICLR 2022 🎓

Other FL paper list

• https://github.com/innovation-cat/Awesome-Federated-Machine-Learning
• https://github.com/chaoyanghe/Awesome-Federated-Learning
• https://github.com/weimingwill/awesome-federated-learning#resource-allocation
• https://github.com/youngfish42/Awesome-Federated-Learning-on-Graph-and-Tabular-Data#federated-learning-framework