Category

Blog

Thanks for Joining us at OSS+ELC EU 2020!

By Blog

Thank you to everyone who joined us last week at Open Source Summit + Embedded Linux Conference Europe 2020! 

The event was held virtually this year, with almost 2,500 registrants from 85 countries

AI/ML/DL Track

The AI/ML/DL Track was well attended, with several AI industry leaders sharing knowledge and insights, and engaging with attendees in several great discussions during and after the sessions.

Miss the sessions? If you registered for OSS+ELC EU, you can always view all of the session recordings on demand on the virtual event platform for 30 days after the event. Or check back on the OSS EU website for a link to all of the sessions recordings on YouTube soon.

LF AI Virtual Booth

The LF AI Foundation virtual booth was also a hit! We had several people visit the booth and interact with our LF AI community members who were staffing the booth. Visitors were also able to download resources to learn more about LF AI.

LF AI Mini Summit

On Thursday, October 29, we hosted our LF AI Mini Summit. Our presenters covered the latest updates from the Foundation, Technical Advisory Council, Trusted AI Committee, and more. We’re looking forward to more engagement opportunities with the community in the future. 

Miss the Summit? If you registered for OSS EU, you can always view the Summit recording on demand on the virtual event platform for 30 days after the event. Or check back on the OSS EU website for a link to all of the sessions recordings on YouTube soon.

LF AI & Data Announcement

On Monday, October 26, during OSS EU, LF AI announced the formation of the LF AI & Data Foundation.

The LF AI & Data Foundation will build and support an open community and a growing ecosystem of open source AI, data and analytics projects, by accelerating development and innovation, enabling collaboration and the creation of new opportunities for all the members of the community.

As one entity under the Linux Foundation, this consolidated and focused effort will enable additional collaboration and integration in the space of AI/ML/DL and Data. With the creation of LF AI & Data, both communities will now support a growing ecosystem of artificial intelligence, machine learning, deep learning and data technologies.

Want to get involved with the LF AI & Data Foundation? Be sure to subscribe to our mailing lists to join the community and stay connected on the latest updates. 

LF AI & Data Resources

New AI & Data Foundation Combines Industry’s Fastest-Growing Open Source Developments in Artificial Intelligence and Open Data

By Blog, Press Release

The Linux Foundation’s AI Foundation & ODPi merge to support growing portfolio of technologies and drive open source collaboration across AI and data

LF AI Foundation (LF AI), the organization building an ecosystem to enable and sustain open source innovation in artificial intelligence (AI), machine learning (ML), and deep learning (DL), and ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced they will come together under the new LF AI & Data Foundation. The LF AI & Data Foundation will build and support an open community and a growing ecosystem of open source AI, data and analytics projects, by accelerating development and innovation, enabling collaboration and the creation of new opportunities for all the members of the community.

As one entity under the Linux Foundation, this consolidated and focused effort will enable additional collaboration and integration in the space of AI/ML/DL and Data. With the creation of LF AI & Data, both communities will now support a growing ecosystem of artificial intelligence, machine learning, deep learning and data technologies. AI and Data are inseparable and codependent on each other. Combining efforts in both spaces will bring developers and projects under a single roof, orchestrated by a single Technical Advisory Council and several committees (Trusted AI, BI & AI), to work together towards building the open source AI & Data ecosystem and accelerating development and innovation. Hosting projects under a single umbrella enables closer collaboration, integration, and interoperability across projects and is a proven recipe for building strong open ecosystems. At the same time, it will provide a unified guidance for end users on tools, interoperability, integration, standards, and the future of AI, Data and Analytics as its use continues to grow in every industry. Furthemore, as member driven organizations, joining forces under LF AI & Data will allow greater efficiency for members across the various services we offer to our hosted projects. 

“LF AI has been growing at the rate of one new project per month, including several data projects. It is a natural move to bring together the open AI and data communities to enable better interoperability and capabilities across all of our hosted projects and to enable closer collaboration, which has been a proven recipe for building a strong open ecosystem. It will also provide our members greater cost efficiency when supporting our projects,” said Dr. Ibrahim Haddad, LF AI Executive Director. “We look forward to supporting innovation in the open source ecosystem focused on AI, Data and Analytics.”

“Over the past 5 years, ODPi has been a part of driving standardization and consolidation in the Big Data and Hadoop ecosystem, as well as becoming a focal point for the data challenges of the enterprises of today such as metadata, governance, and data science,” said John Mertic, Director, ODPi. “Coming together to form LF AI & Data is the next natural step in this mission, enabling greater interoperability to drive both innovation and sustainable growth for key projects.”

ODPi and its projects Egeria and OpenDS4All will become hosted projects under the LF AI & Data Foundation, with BI and AI becoming a committee within the foundation. They will maintain their current open technical governance model, establish collaboration with other hosted projects in LF AI & Data via the efforts of the its Technical Advisory Council, and benefit from a host of services offered to facilitate collaboration and increased adoption. 

Charles “Starlord” Xie, Chairperson, LF AI Governing Board, said: “Today’s announcement of LF AI & Data is very exciting news for the open source AI ecosystem. LF AI & Data brings together companies, projects and communities focusing on AI and Data under a single organization to foster collaboration and integration across projects. We look forward to the months and years to come as we anticipate significant growth in our project portfolio and the development of many collaboration opportunities.”

Craig Rubendall, Chairperson, ODPi, said: “The ODPi Board of Directors and technical community are excited to come together with LF AI to form LF AI & Data. This joint foundation will enable the key open source projects our industry depends on to have a sustainable home, which will drive further innovation and collaboration.”

Jim Spohrer, Director, IBM Center for Open Source Data and AI Technologies and LF AI Technical Advisory Council Chairperson, said: “If you are committed to advancing open source AI and data as an industry-standard infrastructure and democratizing access to AI, there is a great opportunity to contribute within the community. Join our bi-weekly TAC meetings and connect with us on the LFAI slack channel.”

LF AI launched two years ago with nine founding members and one hosted project, and today has 25 members and 20 technical projects. ODPi launched five years ago, and is supported by a strong roster of industry-leading members. Together, under the LF AI & Data Foundation, there are 22 projects supported by more than 60 companies, 20 universities and more than 1,300 active developers contributing to these projects. 

New members of LF AI & Data Foundation include aivancity School for Technology, Business & Society, AlphaBravo, Broadcom, Cloudera, Databricks, Index Analytics, ING Bank, OpenI, Precisely, Peng Cheng Laboratory, SAS Institute, and the Shanghai OpenSource Information Technology Association.

“The future of open source is directly linked to the AI ecosystem and a multitude of data communities. Therefore, it is a natural pairing for LF AI and ODPi to join forces and Cloudera is thrilled to be aiding their efforts in growing the project portfolio to encourage innovation and enable stronger interoperability and collaboration.” – Arun Murthy, CPO, Cloudera.

“Both fresh ingredients and a wonderful recipe are important for a delicious meal. Index Analytics is happy to join this feast as AI (Recipe) and Data (Ingredients) converge!” – Cupid Chan, CTO, Index Analytics

“Precisely is proud to be a member of the Linux Foundation AI & Data to help accelerate the growth and adoption of artificial Intelligence (AI). At Precisely we are committed to delivering the trusted data that is required to enable trusted AI and analytics. We believe that providing data integrity – data with accuracy, consistency, and context – help enable organizations to accelerate their adoption of AI and drive better business decisions.” – Tendü Yoğurtçu, PhD, Chief Technology Officer, Precisely

For more information and to get involved please visit: https://lfai.foundation/about/join/. If you’re interested in hosting a project, please review the Proposal and Hosting Process and check out the Hosting Requirements. For questions, please email info@lfaidata.foundation.

Helpful Resources

About LF AI & Data Foundation

The LF AI & Data Foundation, a Linux Foundation project, accelerates and sustains the growth of Artificial Intelligence (AI), Data Management, Machine Learning (ML), Deep Learning (DL) and Data open source projects. Backed by many of the world’s largest technology leaders, LF AI & Data is a neutral space for harmonization and ecosystem engagement to advance AI and Data innovation. To get involved with the LF AI & Data Foundation, please visit https://lfaidata.foundation.

Thank you ONNX & Amazon for Hosting a Great LF AI Day!

By Blog

A big thank you to Amazon and ONNX for hosting a great virtual meetup! The LF AI Day ONNX Community Virtual Meetup was held on October 14, 2020 and was a great success with over 100 attendees joining live. 

The meetup included ONNX Community updates, partner/end-user stories, and SIG/WG updates. The virtual meetup was an opportunity to connect with and hear from people working with ONNX across a variety of groups. A special thank you to Sheng Zha from Amazon for working closely with the ONNX Technical Steering Committee, SIGs, and ONNX community to curate the content.

Missed the meetup? Check out all of the presentations and recordings here.

This meetup took on a virtual format but we look forward to connecting again at another event in person soon. LF AI Day is a regional, one-day event hosted and organized by local members with support from LF AI, its members, and projects. If you are interested in hosting an LF AI Day please email info@lfai.foundation to discuss.

ONNX, an LF AI Foundation Graduated Project, is an open format to represent deep learning models. With ONNX, AI developers can more easily move models between state-of-the-art tools and choose the combination that is best for them.  Be sure to join the ONNX-Announce mailing list to join the community and stay connected on the latest updates. You can join technical discussions on GitHub and more conversations with the community on LF AI Slack’s ONNX channels.

ONNX Key Links

LF AI Resources

Thank you for Joining the LF AI Foundation at ONES 2020!

By Blog

Thank you to everyone who joined us last week at Open Networking & Edge Summit 2020! 

The event was held virtually this year, with over 1,300 people from 71 countries attending

The virtual LF AI Foundation booth was a hit! We had over 60 people visit our booth in the Gold & Bronze Hall and interact with our LF AI community members who were staffing the booth. Visitors were also able to download resources to learn more about each of LF AI’s projects, and they were able to give us insight into what type of projects they want to see in the future through our attendee survey. We were also able to engage with attendees in several great discussions, both at the booth and in the official ONES 2020 Slack workspace.

The session “The Making of 5G with AI & Open Source, presented by LF AI community member Mazin Gilbert, Vice President of Technology & Innovation at AT&T, was well attended! Attendees were able to hear how the marriage of 5G with edge cloud will enable next generation experiences like holograms for gatherings and meetings, mobile and untethered xR experiences for gaming and remote surgery, and immersive experiences for digital shopping.

Miss the session? If you registered for ONES, you can always view all of the session recordings on demand on the virtual event platform anytime. Or check back on the ONES website for a link to all of the sessions recordings on YouTube soon.

The LF AI Foundation mission is to build and support an open AI community, and drive open source innovation in the AI, ML, and DL domains by enabling collaboration and the creation of new opportunities for all the members of the community.

Want to get involved with the LF AI Foundation? Be sure to subscribe to our mailing lists to join the community and stay connected on the latest updates. 

LF AI Resources

Sparklyr 1.4.0 Release Now Available!

By Blog

Sparklyr, an LF AI Foundation Incubation Project, has released version 1.4.0! Sparklyr is an R Language package that lets you analyze data in Apache Spark, the well-known engine for big data processing, while using familiar tools in R. The R Language is widely used by data scientists and statisticians around the world and is known for its advanced features in statistical computing and graphics. 

In version 1.4.0, sparklyr adds a variety of improvements. Highlights include:

  • This release features efficient and parallelizable weighted sampling methods for Spark data frames. The technique being utilized by those methods is known as “exponential variates” (see https://blogs.rstudio.com/ai/posts/2020-07-29-parallelized-sampling)
  • Tidyr verbs such as `pivot_wider`, `pivot_longer`, `nest`, `unnest`, `separate`, `unite`, and `fill` now have specialized implementations in `sparklyr` for working with Spark data frames
  • Support for newly introduced higher-order functions in Spark 3.0 (e.g., `array_sort`, `map_filter`, `map_zip_with`, and many others)
  • Dplyr-related improvements:
    • All higher-order functions and sampling methods are made directly accessible through `dplyr` verbs
    • Made `grepl` part of the `dplyr` interface for Spark data frames
    • Thanks to a pull request from @wkdavis, `dplyr::inner_join`, `dplyr::left_join`, `dplyr::right_join`, and  `dplyr::full_join` now correctly replace `’.’` with `’_’` in the `suffix` parameter when working with Spark data frames (see https://github.com/sparklyr/sparklyr/issues/2648)
  • Support for newly introduced functionalities in Spark 3.0
    • Thanks to a pull request from @zero323, the `RobustScaler` functionality in Spark 3.0 is now supported in sparklyr through `ft_robust_scaler`
    • RAPIDS GPU acceleration plugin can now be enabled with `spark_connect(…, package = “rapids”)` and configured with `spark_config` options prefixed with “spark.rapids.”

The power of open source projects is the aggregate contributions originating from different community members and organizations that collectively help drive the advancement of the projects and their roadmaps. The sparklyr community is a great example of this process and was instrumental in producing this release. The sparklyr team wanted to give a special THANK YOU to the following community members for their contributions via pull requests (listed in chronological order):

To learn more about the sparklyr 1.4.0 release, check out the full release notes. Want to get involved with sparklyr? Be sure to join the sparklyr-Announce and sparklyr Technical-Discuss mailing lists to join the community and stay connected on the latest updates.

Congratulations to the sparklyr team and we look forward to continued growth and success as part of the LF AI Foundation! To learn about hosting an open source project with us, visit the LF AI Foundation website.

sparklyr Key Links

LF AI Resources

LF AI Day ONNX Community Virtual Meetup – Fall 2020

By Blog

Amazon, ONNX, and the LF AI Foundation are pleased to sponsor the upcoming LF AI Day* – ONNX Community Virtual Meetup – Fall 2020, to be held via Zoom on October 14th.

ONNX, an LF AI Foundation Graduated Project, is an open format to represent deep learning models. With ONNX, AI developers can more easily move models between state-of-the-art tools and choose the combination that is best for them. 

The virtual meetup will cover ONNX Community updates, partner/end-user stories, and SIG/WG updates. Check back on the event website for the agenda. If you are using ONNX in your services and applications, building software or hardware that supports ONNX, or contributing to ONNX, you should attend! This is a great opportunity to connect with and hear from people working with ONNX across many companies. 

Registration is now open and the event is free to attend. Capacity will be 500 attendees. For up to date information on this virtual meetup, please visit the event website

Want to get involved with ONNX? Be sure to join the ONNX-Announce mailing list to join the community and stay connected on the latest updates. You can join technical discussions on GitHub and more conversations with the community on LF AI Slack’s ONNX channels.

Note: In order to ensure the safety of our event participants and staff due to the Novel Coronavirus situation (COVID-19) the ONNX Steering Committee decided to make this a virtual-only event via Zoom.*LF AI Day is a regional, one-day event hosted and organized by local members with support from LF AI and its Projects. Learn more about the LF AI Foundation here.

ONNX Key Links

LF AI Resources

NNStreamer 1.6.0 Release Now Available!

By Blog

NNStreamer, an LF AI Foundation Incubation-Stage Project, has released version 1.6.0. NNStreamer is a set of Gstreamer plugins that support ease and efficiency for Gstreamer developers adopting neural network models and neural network developers managing neural network pipelines and their filters.

In version 1.6.0, NNStreamer adds a variety of improvements; highlights include:

  • New hardware accelerators and neural network frameworks support added: Verisilicon-Vivante, Qualcomm-SNPE, NNFW-ONE-Runtime, and Tensorflow2-lite.
  • Data serialization support with flatbuf and protobuf.
  • Android APIs optimized (i.e., invoke latency in Galaxy S20: 2ms → 0.1ms)
  • Plug-and-play sub-plugins support for tensor-converters
  • Hardware acceleration configuration reworked: multiple candidates may be expressed and options may be altered in run-time.
  • Fixes, semantics updates, and minor features added after commercialization (Galaxy Watch 3 and a few “next-year” products and Tizen 6 releases).

The NNStreamer Project invites you to adopt or upgrade to version 1.6.0 in your application, and welcomes feedback. To learn more about the NNStreamer 1.6.0 release, check out the change log and full release notes. Want to get involved with NNStreamer? Be sure to join the  NNStreamer-Announce and NNStreamer-Technical-Discuss mail lists to join the community and stay connected on the latest updates.

Congratulations to the NNStreamer team! We look forward to continued growth and success as part of the LF AI Foundation. To learn about hosting an open source project with us, visit the LF AI Foundation website.

NNStreamer Key Links

LF AI Resources

3 Trusted AI Toolkits Join LF AI as Newest Incubation Projects

By Blog

LF AI Foundation (LF AI), the organization building an ecosystem to sustain open source innovation in artificial intelligence (AI), machine learning (ML), and deep learning (DL), today is announcing 3 Trusted AI Toolkits as its latest Incubation Projects: AI Fairness 360 Toolkit, Adversarial Robustness Toolbox, and AI Explainability 360 Toolkit. All 3 toolkits were originally released and open sourced by IBM

AI Fairness 360 Toolkit

The AI Fairness 360 (AIF360) Toolkit is an open source toolkit that can help detect and mitigate unwanted bias in machine learning models and datasets. With the toolkit, developers and data scientists can easily check and mitigate for biases at multiple points along their machine learning lifecycle, using the appropriate fairness metrics for their circumstances. It provides metrics to test for biases, and algorithms to mitigate bias in datasets and models. The AI Fairness 360 interactive experience provides a gentle introduction to the concepts and capabilities. Recently, AIF360 also announced compatibility with Scikit Learn, and an interface for R users.

Adversarial Robustness 360 Toolbox

The Adversarial Robustness 360 (ART) Toolbox is a Python library for Machine Learning Security. ART provides tools that enable developers and researchers to evaluate, defend and verify Machine Learning models and applications against the adversarial threats of Evasion, Poisoning, Extraction, and Inference. ART supports all popular machine learning frameworks (TensorFlow, Keras, PyTorch, MXNet, scikit-learn, XGBoost, LightGBM, CatBoost, GPy, etc.), all data types (images, tables, audio, video, etc.) and machine learning tasks (classification, object detection, generation, certification, etc.).

AI Explainability 360 Toolkit

The AI Explainability 360 (AIX360) Toolkit is a comprehensive open source toolkit of diverse algorithms, code, guides, tutorials, and demos that support the interpretability and explainability of machine learning models. The AI Explainability 360 interactive experience provides a gentle introduction to the concepts and capabilities by walking through an example use case for different consumer personas.

See IBM’s full announcement on the donation here. Since IBM’s donation, we are pleased to announce that the Trusted AI projects have been approved by the LF AI TAC and have now been formally moved into the LF AI Foundation, complete with all the legal formalities, new logos, websites and GitHub locations.

Dr. Ibrahim Haddad, Executive Director of LF AI, said: “We are very pleased to welcome these Trusted AI open source projects to LF AI. For the past year, the Trusted AI Committee at LF AI has been actively working on building its community and defining a set of principles that AI software is expected to honor. With the addition of these three tools, our efforts now have a venue to codify these principles and provide an opportunity to collaborate on the code with the global community under a vendor-neutral and open governance. We look forward to supporting these projects and helping them to thrive and grow their community of adopters and contributors.” 

LF AI supports projects via a wide range of benefits; and the first step is joining as an Incubation Project. LF AI will support the neutral open governance for these Trusted AI projects to help foster the growth of the projects. 

“At IBM, at our core, we believe in the fair and equitable use of technology and this is especially true of artificial intelligence. Developers must ensure applications are built with trust and transparency,” said Todd Moore, Vice President, Open Technology, IBM, “Our AI Fairness toolkits and Watson OpenScale are enabling developers and data scientists to address bias, and explain the behavior of our models. By open sourcing IBM’s Adversarial Robustness 360, AI Fairness 360, and AI Explainability 360 toolkits through The LF AI Foundation, we all can advance the technology, in an open governance community and encourage the best and brightest to collaborate on one of the most pressing issues in this technological area.”

Trusted AI Video Series

To learn more about the Trusted AI projects, take a look at the 7-episode video series on YouTube, created by Anessa Petteruti (Computer Science senior at Brown University) in collaboration with LF AI:

EPISODE 1: Introducing Trusting AI: Unlocking the Black Box with Animesh Singh 

Artificial intelligence unlocks countless possibilities for the human race. But is there a darker side to the technology? In the opening episode of Trusting AI: Unlocking the Black Box, Animesh Singh, Chief Architect of the Artificial Intelligence and Machine Learning OSS Platform at IBM, introduces IBM and the Linux Foundation’s Trusted AI efforts.

EPISODE 2: Trusted AI Research with Aleksandra Mojsilović

Learn specifically about research conducted for Trusted AI in this interview with the Foundations of Trusted AI Lead, Aleksandra Mojsilović. Dr. Mojsilović, an IBM Fellow, also co-directs IBM’s Science for Social Good.

EPISODE 3: AI Explainability and Factsheets with Michael Hind

Michael Hind, Distinguished Research Staff Member at IBM Research AI, discusses one of Trusted AI’s toolboxes, AI Explainability 360, as well as AI Factsheets 360 in depth.

EPISODE 4: Ethical AI in Higher Education with Michael Littman

Universities all over the world have taken efforts to incorporate ethical teachings into artificial intelligence and machine learning courses. In Providence, Rhode Island, Professor Michael Littman of Brown University discusses AI ethics in higher education and the computer science department’s Responsible CS program.  

EPISODE 5: Adversarial Robustness with Mathieu Sinn

Artificial intelligence presents numerous benefits as well as security vulnerabilities. Mathieu Sinn, Senior Technical Staff Member and Manager of AI Security and Privacy at IBM, delves into Trusted AI’s Adversarial Robustness Toolbox and the program’s efforts to combat cyber attacks in AI.

EPISODE 6: Open Source and the Linux Foundation with Ibrahim Haddad

Ibrahim Haddad, Vice President of Strategic Programs at the Linux Foundation and Executive Director of LF AI, discusses the importance of open source in software development, specifically artificial intelligence.

EPISODE 7: Trusted AI in Production and MLOps with Tommy Li and Andrew Butler

How can developers use Trusted AI in their own projects? Find out from IBM software engineers Tommy Li and Andrew Butler who guide you through MLOps and using Trusted AI in production.

Get Involved

Check out the Trusted AI GitHub for Getting Started guides to start working with these projects today. Learn more about the Trusted AI toolkits on their websites (AI Fairness 360, AI Explainability 360, Adversarial Robustness Toolbox) and be sure to join the Trusted-AI-360-Announce and Trusted-AI-360-Technical-Discuss mail lists to join the community and stay connected on the latest updates. 

A warm welcome to these Trusted AI projects! We look forward to the projects’ continued growth and success as part of the LF AI Foundation. To learn about how to host an open source project with us, visit the LF AI website.

Trusted AI Key Links

LF AI Resources

Join the LF AI Foundation at OSS+ELC Europe 2020

By Blog

The LF AI Foundation is excited to announce our participation at the upcoming Open Source Summit + Embedded Linux Conference Europe 2020! The event will be held virtually, and registration is only $50 for four days of learning and collaboration. 

Below are all the different ways to interact with the LF AI Foundation at the conference. We hope to see you there!

Attend Sessions in the AI/ML/DL Track 

The LF AI Foundation will be hosting an AI/ML/DL Track at OSS EU. Join these sessions to learn the latest updates from our projects and hear from leaders in the AI industry. Register for OSS EU to attend.

Chat with us in the Virtual Exhibit Hall!

Come chat with us in the virtual exhibit hall at OSS EU. Various LF AI community members will be around all week to answer any questions you have. You’ll also be able to get more information on how to get involved with the LF AI Foundation.

Attend the LF AI Mini Summit!

We invite you to join us for our LF AI Foundation Mini Summit where we will cover the latest updates from the Foundation, Technical Advisory Council, Trusted AI Committee, and more. Also hear the latest updates from our ONNX, Amundsen, Angel and Marquez projects. We look forward to uncovering new collaboration opportunities among our growing community. 

Join us by registering to attend the Open Source Summit EU – Register Now

The LF AI Mini Summit is co-located with the Open Source Summit EU and will be held virtually on Thursday, October 29 at 14:00 – 15:30 Greenwich Mean Time Zone (GMT). You will need to be registered for OSS EU to attend. OSS EU registration costs $50 USD, which includes access to the LF AI Mini Summit.

The LF AI Foundation mission is to build and support an open AI community, and drive open source innovation in the AI, ML, and DL domains by enabling collaboration and the creation of new opportunities for all the members of the community. 

Want to get involved with the LF AI Foundation? Be sure to subscribe to our mailing lists to join the community and stay connected on the latest updates. 

LF AI Resources

LF AI Foundation Announces Graduation of Horovod Project

By Blog

The LF AI Foundation, the organization building an ecosystem to sustain open source innovation in artificial intelligence (AI), machine learning (ML) and deep learning (DL), is announcing today that hosted project Horovod is advancing from an Incubation level project to a Graduate level. This graduation is the result of Horovod demonstrating thriving adoption, an ongoing flow of contributions from multiple organizations, and both documented and structured open governance processes. Horovod has also achieved a Core Infrastructure Initiative Best Practices Badge, and demonstrated a strong commitment to community.

As an Incubation Project, Horovod utilized the LF AI Foundation’s various enablement services to foster its growth and adoption; including program management support, event coordination, legal services, and marketing services ranging from website creation to project promotion. 

Horovod is a distributed training framework for TensorFlow, Keras and PyTorch, which improves speed, scale and resource allocation in machine learning training activities. It was open sourced by Uber, the project founder, and joined LF AI as an Incubation Project in December 2018. 

“The journey of Horovod from Incubation to Graduation has been very impressive,” said Dr. Ibrahim Haddad, Executive Director of the LF AI Foundation. “The speed of development, the growth of its community, and its wide adoption is particularly noteworthy.  Horovod has exceeded all of our graduation criteria and we’re proud to be its host Foundation and to support them across a number of services. As a Graduate project, our support to Horovod will continue to increase as needed. This graduation is our way to present Horovod as an advanced and mature open source technology ready for large scale deployments. Congratulations, Horovod!”

Uber uses Horovod for self-driving vehicles, fraud detection, and trip forecasting. It is also being used by Alibaba, Amazon and NVIDIA. Contributors to the project outside Uber include Amazon, IBM, Intel and NVIDIA. 

“Since joining the LFAI, Horovod has developed into the industry-standard for training deep neural networks at scale, in every framework and on every platform,” said Travis Addair, Technical Lead for the Horovod project. “It’s a continued honor to collaborate with and learn from Horovod’s many exceptional contributors from across the deep learning community. This graduation is a major milestone for the Horovod project, and an acknowledgement of all the hard work and collaboration our contributors have put forward to make this project a success. As a graduated project, I am looking forward to broadening the reach of our community even further, working towards the goal of making deep learning training simple and intuitive to scale.”

Feature Roadmap for 2020

  • Elastic Training / Fault Tolerance
  • Horovod on Ray + Ray Tune Integration
  • Ludwig + Horovod Spark Estimator integration
  • TensorFlow / General Horovod Spark Estimator
  • MXNet Horovod Spark Estimator (Amazon)
  • Horovod Plugin Architecture (NVIDIA)
  • Horovod Spark Dynamic GPU Allocation (NVIDIA)

Curious about how Horovod can make your model training faster and more scalable? Try out the framework now. And be sure to join the Horovod Announce and Horovod Technical-Discuss mailing lists to join the community and stay connected on the latest updates. 

Congratulations to the Horovod team and we look forward to continued growth and success as part of the LF AI Foundation! To learn about hosting an open source project with us, visit the LF AI Foundation website.

Horovod Key Links

LF AI Resources