Guest Author(s): LF AI Graduated Projects, Angel and Acumos
The goal of the LF AI Foundation (LF AI) is to accelerate and sustain the growth of Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL) open source projects. Backed by many of the world’s largest technology leaders, LF AI is a neutral space for harmonization and ecosystem engagement to advance AI, ML, and DL innovation. Projects are hosted in either of two stages: graduation and incubation. At the time of publishing this blog post, LF AI hosts three graduation level projects (Acumos, Angel, and ONNX) and eight incubation level projects (Adlik, Elastic Deep Learning, Horovod, Marquez, Milvus, NNStreamer, Pyro and sparklyr).
The incubation stage is designated for new or early-stage projects that are aligned with the LF AI mission and require help to foster adoption and contribution in order to sustain and grow the project. Incubation projects may receive mentorship from the LF AI Technical Advisory Council (TAC) and are expected to actively develop their community of contributors, governance, project documentation, and other variables that factor into broad success and adoption.
Incubation projects are eligible to graduate when they meet a certain number of criteria demonstrating significant growth of contributors and adopters, commitment to open governance, achieving and maintaining a CII best practices badge, and establishing collaboration with other LF AI hosted projects. Getting to this stage requires work, perseverance, and tangible signs of progress.
On the other hand, the graduation stage signaled projects that have achieved significant growth of contributors and adopters, are important to the ecosystem, and also are eligible for foundational financial support.
Angel Project
Angel joined LF AI as an incubation project in August 2018. It is a high-performance distributed machine learning platform based on the philosophy of Parameter Server. It is tuned for high performance and has a wide range of applicability and stability, demonstrating increasing advantage in handling higher dimension models. The Angel Project has been proactively collaborating with the Acumos Project community, resulting in positive outcomes to both communities.
In its effort to move to graduation, the Angel Project community looked at the full range of LF AI hosted projects and chose Acumos for integration.
Why Acumos?
Inside AI open source community, cross-project collaboration is essential. The Angel platform focuses on training of models based on machine learning algorithms while it doesn’t host any public model marketplace. On the other hand, Acumos supports an AI marketplace that empowers data scientists to publish adaptive AI models, while shielding them from the need to custom develop fully integrated solutions.
This makes Angel teaming up with Acumos a perfect match as the two would work like a factory and distributor after integration and therefore create a synergy effect. The Angel team believed that integration with Acumos could encourage and facilitate algorithm sharing by Angel users and therefore benefit the overall community.
In the following sections, we will explore some of the challenges the projects faced during the process and how integration was achieved.
Integration Challenges
—Challenge A: Lack of reference to on-board Java-based model to Acumos marketplace that was dominated by Python models. This challenge was solved with the assistance of Acumos technical gurus from AT&T, Tech Mahindra, and Orange. They provided clear guidance and instructions including jar package access, configuration, as well as Java model preparation.
—Challenge B: Seeking deployed internet accessible environment. It was appreciated that Huawei offered access to Acumos environments set on its public cloud in Hong Kong. However, the uploading process wasn’t all smooth sailing as several attempts failed due to unsuccessful generation of artifacts. The problem was later solved with the help from AT&T and Huawei by restarting Nexus and cleaning the disk to address the insufficient storage issue.
What Was Achieved?
A successful integration of Angel and Acumos demonstrated that Angel’ s Java-based models could be on-boarded to a marketplace dominated by Python projects.
At the same time, connecting Angel and Acumos in both API invoking and production deployment would allow more developers to use the Angel framework to train domain specific algorithms and share their works with people around the world. Acumos also become a stronger platform by adding more frameworks and users.
Cross project collaboration played a key role in Angel’s graduation as it proved that the project was an open system and could be connected with other projects. Only by demonstrating the capability of linking both upstream and downstream components in a productive data pipeline, a project could be deemed as a member of the global machine learning community, rather than an isolated system.
The collaboration between Angel and Acumos sets an example for other incubation level projects hosted by LF AI. The foundation hopes that more projects will follow the footsteps of Angel and Acumos, and with collective efforts, a sustainable development of harmonized community can be achieved soon.
Next Steps
To encourage further collaboration, Angel plans to invite global diversified users to publish their models onto Acumos. In parallel, Angel will also look at opportunities to incorporate their project with other components such as ML-flow framework, Web portal and monitoring system, more formats of model file support, etc.
To learn more about these two LF AI hosted projects, and to view all projects, visit the LF AI Projects page. If you would like to learn more about hosting a project in LF AI and the benefits, click here.
LF AI Resources
- Learn about membership opportunities
- Explore the interactive landscape
- Check out our technical projects
- Join us at upcoming events
- Read the latest announcements on the blog
- Subscribe to the mailing lists
- Follow us on Twitter or LinkedIn
- Access other resources on LF AI’s GitHub or Wiki
