The LF AI & Data Foundation, the organization building an ecosystem to sustain open source innovation in artificial intelligence (AI) and data open source projects, is announcing today that hosted project Flyte is advancing from an Incubation level project to a Graduate level. This graduation is the result of Flyte demonstrating thriving adoption, an ongoing flow of contributions from multiple organizations, and both documented and structured open governance processes. Flyte has also achieved a Core Infrastructure Initiative Best Practices Badge, and demonstrated a strong commitment to its community of users and contributors.
Flyte is a production-grade, declarative, structured, and highly scalable cloud-native workflow orchestration platform. It allows users to describe their ML/Data pipelines using Python, Java, or in the future, other languages. It manages the data flow, parallelization, scaling, and orchestration of these pipelines. Flyte was released and open sourced by Lyft.
The feature-set and extensibility of Flyte makes it extremely useful for Machine Learning pipelines like Feature Engineering, Adhoc and periodic Model Training, Model evaluation, ML monitoring, ETL, and general data processing. Flyte has extensions for Pandas, Tensorflow, Pytorch, Spark and integrates with Hosted cloud services such as AWS Sagemaker, Athena, etc.
Originally developed and open sourced by Lyft, Flyte joined LF AI & Data as an incubation project in February 2021. As an Incubation project, the project has benefited from the LF AI & Data’s various enablement services to foster its growth and adoption; including program management support, event coordination, legal services, and marketing services ranging from website creation to project promotion.
“Flyte is a great example of a project that grew significantly as an LF AI & Data hosted project and benefited from a range of provided services and collaboration opportunities. The project’s graduation is a sign of maturity, functioning open governance, large-scale adoption, and significant integration efforts with other projects in the ecosystem,” said Dr. Ibrahim Haddad, Executive Director of the LF AI & Data. “The growth and development of Flyte’s contributors community is particularly impressive. Flyte meets our graduation criteria and we’re proud to be its host Foundation. As a Graduate project, we will continue to support it via an extended set of services tailored for Graduated projects. We’re also excited that the project is now eligible for a voting seat on LF AI & Data’s Technical Advisory Council. Congratulations, Flyte!”
“Flyte, even though a relatively young open-source project, is a mature product developed and battle-tested at Lyft, Spotify, Freenome, Blackshark, Wolt (Doordash), GoJek, and many others. Within ten months of being a part of the LF AI & Data family, Flyte has grown more than 10x in usage, garnered many committed contributors, and has got Union.ai to back it up,” explains Ketan Umare, Co-founder and CEO of Union.ai.
“Flyte intends to be the de facto ML-aware orchestration platform that unifies disparate and fragmented MLOps communities, demonstrated by an impressive array of integrations across the Data and ML Stack. The platform is poised to become a central part of the LF AI & Data ecosystem.
Flyte graduation is the first step towards an exciting future for the project with 1.0 around the corner, some outstanding ecosystem projects, and core features in the pipeline.
The LF AI & Data foundation has been a wonderful home to the project. We hope to lead the integrations across several LF projects to create a cohesive landscape,” said Ketan Umare.
Flyte in Numbers
The stats below capture Flyte’s development efforts as of their graduation in December 2021:
- Contributors on GitHub: 360
- GitHub stars: 1.8K
- GitHub forks: 180
- Known community members: 70
Curious about how to get involved with Flyte?
Be sure to join the Flyte Announce and Flyte Technical-Discuss mailing lists to join the community and stay connected on the latest updates. Learn more about Flyte on their website and GitHub.
Congratulations to the Flyte team! We look forward to continued growth and success as part of the LF AI & Data Foundation. To learn about hosting an open source project with us, visit the LF AI & Data Foundation website.
Flyte Key Links
LF AI & Data Resources
- Learn about membership opportunities
- Explore the interactive landscape
- Check out our technical projects
- Join us at upcoming events
- Read the latest announcements on the blog
- Subscribe to the mailing lists
- Follow us on Twitter or LinkedIn
- Access other resources on LF AI & Data’s GitHub or Wiki
