A major challenge in many large-scale machine learning tasks is to solve an optimization objective involving data that is distributed across multiple machines. In this setting, optimization methods that work well on single machines must be re-designed to leverage parallel computation while reducing communication costs. This requires developing new distributed optimization methods with both competitive practical performance and strong theoretical convergence guarantees. CoCoA is a a novel framework for distributed computation that meets these requirements, while allowing users to re-use arbitrary single machine solvers locally on each node.
National Science Foundation
Expeditions in Computing