The Linux Foundation Projects
Skip to main content
Blog

LF AI DataOps Committee Open Lineage and Egeria session

By February 8, 2022No Comments

Guest Authors: David Radley – Egeria committer employed by IBM, and Saishruthi Swaminathan

DataOps is about the people, processes, and technologies used to operationalize data management. During a meeting on Thursday, February 10th, 2022, at 7:00am PST, the DataOps Committee will demonstrate how using two LF for AI and Data Foundation projects together can help realize your DataOps goals.

ProjectDescription
Egeria 
A picture containing text

Description automatically generated
https://egeria-project.org/
Open metadata and governance for enterprises – automatically capturing, managing and exchanging metadata between tools and platforms, no matter the vendor.
Open Lineage Standard

https://live-lfprojects3.linuxfoundation.org/projects/openlineage/
OpenLineage proposes an open standard and API for lineage collection that data processing engines can implement to publish at run time details of the data sources that it is reading, the types of processing it is performing and the destination of the results.

Mandy Chessell is a committer and the leader of the Egeria project. She is also a committer on the OpenLineage project will be leading this session, explaining how Egeria and OpenLineage complement each other’s capabilities. 

Egeria has many integration connectors that are used to bring 3rd party technology into the Egeria eco-system. This session will provide details on the OpenLineage Cataloguer Integration Connector; what this connector does, how you set it up,  and how that relates to DataOps.

Here is how the connector is structured. 

Diagram

Description automatically generated

DataOps Resources & Meeting Details

LF AI & Data Resources

Author

  • Andrew Bringaze

    Andrew Bringaze is the senior developer for The Linux Foundation. With over 10 years of experience his focus is on open source code, WordPress, React, and site security.