Distributed Machine Learning
July 12, 2017 ยท View on GitHub
Distributed Optimization
- 2016 ICDM Efficient Distributed SGD with Variance Reduction
- 2016 KDD Robust Large-Scale Machine Learning in the Cloud
- 2015 KDD Netowrk Lasso: Clustering and Optimization in Large Graphs
- 2012 JMLR Distributed Learning, Communication Complexity and Privacy
- 2012 AISTATS Protocols for Learning Classifiers on Distributed Data
- 2010 NIPS Parallelized Stochastic Gradient Descent | video (One-Short)
- 2010 NAACL Distributed Training Strategies for the Structured Perceptron
- 2009 NIPS Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models
- 2009 NIPS Slow Learners are Fast
Communication Efficiency, Complexity, Delay, Latency
- 2014 ATC Exploiting bounded staleness to speed up Big Data analytics
2014 NIPS Fundamental Limits of Online and Distributed Algorithms for Statistical Learning and Estimation - 2014 ICML Communication-Efficient Distributed Optimization using an Approximate Newton-type Method
- 2013 NIPS Information-theoretic lower bounds for distributed statistical estimation with communication constraints
- 2013 NIPS Optimistic Concurrency Control for Distributed Unsupervised Learning
- 2013 SDM Butterfly Mixing: Accelerating Incremental-Update Algorithms on Clusters
- 2012 NIPS Communication-Efficient Algorithms for Statistical Optimization
- 2011 NIPS Distributed Delayed Stochastic Optimization
Distributed Mini-Batching
- 2014 KDD Efficient Mini-batch Training for Stochastic Optimization
- 2012 JMLR Optimal Distributed Online Prediction Using Mini-Batches
- 2011 ICML Optimal Distributed Online Prediction
- 2011 NIPS Better Mini-Batch Algorithms via Accelerated Gradient Methods
Distributed Consensus
- 2016 ICLRW Revisiting Distributed Synchronous SGD
- 2014 Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers (ADMM)
- 2012 IEEE Trans. on Automatic Control Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
- 2010 NIPS Distributed Dual Averaging in Networks
- 2009 IEEE Trans. on Automatic Control Distributed subgradient methods for multi-agent optimization | slides
- 2008 Convex Optimization in Signal Processing and Communications Cooperative Distributed Multi-Agent Optimization
Distributed ML Systems
- 2014 APSys A Scalable and Topology Configurable Protocol for Distributed Parameter Synchronization
- 2014 ICML Tutorial Emerging System for Large-Scale Machine Learning
- 2013 Distributed Computing When distributed computation is communication expensive
MapReduce / AllReduce
- 2014 JMLR A Reliable Effective Terascale Linear Learning System
- 2010 NIPSW MapReduce/Bigtable for Distributed Optimizationslides
- 2007 NIPS Map-Reduce for Machine Learning on Multicore
Parameter Servers
- 2014 OSDI Project Adam: Building an Efficient and Scalable Deep Learning Training System
- 2014 OSDI Scaling Distributed Machine Learning with the Parameter Server
- 2014 NIPS Communication Efficient Distributed Machine Learning with the Parameter Server
- 2013 NIPSW Parameter Server for Distributed Machine Learning
- 2013 NIPSW Distributed Delayed Proximal Gradient Methods
- 2012 NIPS Large Scale Distributed Deep Networks (DistBelief)
- 2010 VLDB An Architecture for Parallel Topic Models