[Feature Request]: Integrate Apache Beam with Open Lineage

[rohitsinha54]

What would you like to happen?

See 33980 for previous background around tracking lineage in Apache Beam direct runner.

Building up open the "Sending this information to some Lineage graph visualization tool."

Beam can report this lineage information in OpenLineage format. This allows Beam users to visualize the lineage with already supported lineage visualization tools of open lineage.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[rohitsinha54]

.set-labels gsoc, gsoc2025

[github-actions]

Label cannot be managed because it does not exist in the repo. Please check your spelling.