By Mufajjul Ali, Edwin Cheung, and Lindsey Alan
LFAI & Data Foundation is pleased to share that Feathr, an LFAI & Data Foundation Sandbox-Stage Project, has released OSS Feathr 1.0. Feathr is an enterprise scale Feature store, which facilitates the creation, engineering, and usage of ML features in production. It has been used by many organizations for online, offline store, as well as real-time streaming.
This release contains many new features and enhancements since Feathr became open-source 1 year ago. These include features such as the online transformation, rapid sandbox environment, MLOPs V2 accelerator integration and much more to really accelerate the development and deployment of machine learning projects at enterprise scale.
Online Transformation via DSL
In various ML scenarios, features generation is required for both training and inferences. There is a limitation where data source cannot come from online service, as currently transformation only happens before feature data is published to an online store and the transformation is required close to real-time. In such cases, there is a need for a mechanism whereby the user has the ability to run transformation on the inference data dynamically before inferencing via the Model. The new online Transformation via DSL feature addresses these challenges by using a custom transformation engine that can process transformation requests and responses close to real-time on demand.
It allows definition of transformation logic declaratively using DSL syntax which is based on EBNF. It also provides extensibility, where there is a need to define custom complex transformation, by supporting UDF (user defined function) written in Python or Java.
This declarative logic runs in a new high-performance DSL engine. We provide HELM Chart to deploy this service in a container-based technology such as the AKS. Our internal benchmarking shows it can scale up to 100K+/QPS and more.
The transformation engine can also run as a standalone executable which is a HTTP server that can be used to transform data for testing purposes. feathrfeaturestore/feathrpiper:latest.
It also provides the ability to auto-generate the DSL file, if there are already predefined feature transformations, which have been created for the offline-transformation.
For more details, check out the online transformation guide.
Getting Started with Sandbox Environment
This is an exciting feature, especially for data scientists, who may not have the necessary infrastructure background, or know how to deploy the infrastructure in the cloud. The sandbox is a fully featured, quick start Feathr environment that enables organisations to rapidly prototype various capabilities of Feathr without the burden of full-scale infrastructure deployment. It is designed to make it easier for users to get started quickly, validate feature definitions and new ideas and interactive experience.
By default, it comes with a Jupyter notebook environment to interact with the Feathr platform.
Users can also use the UX to visualize the features, lineage and other capabilities.
To get started, check out the quick start guide to local sandbox.
Feathr with MLOps V2 accelerator
MLOps V2 solution accelerator provides a modular end-to-end approach to MLOps in Azure based on pattern architecture. We are pleased to announce an initial integration of Feathr to the classical pattern, that enables Terraform based infrastructure deployment as part of the infrastructure provisioning with AML workspace.
With this integration, enterprise customers can use the templates to customize their CI/CD workflows to run end-to-end MlOps in their organization.
Check out the Feathr integration with MLOps V2 deployment guide.
Feathr GUI enhancement
We have added a number of enhancements to the GUI to improve the usability. These include support for registering features, support for deleting features, support for displaying version, and quick access to lineage via top menu.
Try out our demo UX on our live demo site.
What’s Next
The Feathr journey has just begun, this is the first stop to many great things to come. So, stay tuned for many enterprise enhancements, security, monitoring and compliance features with more enriched MLOps experience. Check out how you can also contribute to this great project, community guidelines, and if you have not already joined, you can join our slack channel here.
Feathr Key Links
LF AI & Data Resources
- Learn about membership opportunities
- Explore the interactive landscape
- Check out our technical projects
- Join us at upcoming events
- Read the latest announcements on the blog
- Subscribe to the mailing lists
- Follow us on Twitter or LinkedIn
- Access other resources on LF AI & Data’s GitHub or Wiki
