Horovod
Horovod makes it easy to take a single-GPU TensorFlow program and successfully train it on many GPUs faster. Horovod also achieved significantly improved GPU resource usage figures. The project uses advanced algorithms and leverages features of high-performance networks to provide data scientists, researchers and AI developers with tooling to scale their deep learning models with ease and high performance. In benchmarking Horovod against standard distributed TensorFlow, Uber has observed large improvements in its ability to scale, with Horovod coming in roughly twice as fast.
Horovod is a graduation-stage project of the LF AI & Data Foundation.
Contributed by: Uber in December 2018 as an incubation-stage project and graduated in September 2020.