LF AI & Data Foundation—the organization building an ecosystem to sustain open source innovation in artificial intelligence (AI) and data open source projects, today is announcing TonY as its latest Incubation Project.
TonY is a framework to natively run deep learning jobs on Apache Hadoop. It currently supports TensorFlow, PyTorch, MXNet and Horovod. TonY enables running either single node or distributed training as a Hadoop application. This native connector, together with other TonY features, aims to run machine learning jobs reliably and flexibly. For a quick overview of TonY and comparisons to other frameworks, please see this presentation.
Dr. Ibrahim Haddad, Executive Director of LF AI & Data, said: “We’re excited to welcome the TonY project in LF AI & Data. The project enables data scientists with a way to manage the challenges associated with scaling up training; fitting well into our portfolio of projects and technologies. We look forward to working with the community to grow the project’s footprint and to create new collaboration opportunities for it with our members and other hosted projects.”
For the past few years, TonY has been empowering all of LinkedIn’s production deep learning jobs to create more relevant content for the company’s 774+ million members. Externally, TonY has also been adopted by companies like iQiyi, and integrated with Google Cloud. Horovod, another LF AI & Data project, was also supported in TonY recently.
Keqiu Hu, technical lead for the TonY project, said: “LinkedIn built TonY to make distributed deep learning training easy on Apache Hadoop for AI model developers. We are thrilled to contribute TonY to the community as the LF AI & Data Foundation’s newest project. The collaboration will further scale the impact of TonY in the open source community, and accelerate the advancement of AI in the industry”.
LF AI & Data supports projects via a wide range of services, and the first step is joining as an Incubation Project. LF AI & Data will support the neutral open governance for the project to help foster its growth. Learn more about TonY on their GitHub and be sure to subscribe to the TonY-Announce and TonY-Technical-Discuss mail lists to join the community and stay connected on the latest updates.
A warm welcome to TonY! We look forward to the project’s continued growth and success as part of the LF AI & Data Foundation. To learn about how to host an open source project with us, visit the LF AI & Data website.
TonY Key Links
LF AI & Data Resources
- Learn about membership opportunities
- Explore the interactive landscape
- Check out our technical projects
- Join us at upcoming events
- Read the latest announcements on the blog
- Subscribe to the mailing lists
- Follow us on Twitter or LinkedIn
- Access other resources on LF AI & Data’s GitHub or Wiki
