Five years of OpenLineage: How we built an industry standard and why agents need it | Datadog

[2026 - DAY 3 - LIGHTNING TALK] Over the past five years, OpenLineage has become the de facto standard for data lineage metadata, adopted across the industry by leading platforms and enterprises. In this talk, we'll trace the journey of building an open standard. You'll learn what changed in the ecosystem that made standardization possible, the critical features that drove adoption (column-level lineage, streaming support, unified facets), and where OpenLineage stands today - five years since its initial release. Most importantly, we'll explore why this matters now: as AI agents increasingly make decisions about data - where to read from, what to trust, how fresh it is - they need a shared understanding of data context. Lineage metadata is the knowledge graph that transforms agents from black boxes into informed decision-makers. The talk covers the standards perspective, the pragmatic integration challenges, and a forward-looking vision for how great metadata enables intelligent data systems. SPEAKER: Harel Shein - Senior Engineering Manager, Datadog 👉 Sign up for our "No BS" Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter ABOUT AI COUNCIL: AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools. FIND US: Website: https://aicouncil.com/ LinkedIn: https://www.linkedin.com/company/aicouncilconf/ X: https://x.com/aicouncilconf