Data Lakehouse + Federation:
Why the Best Enterprise Architecture Uses Both

What Each Architecture Does Best

Data Lakehouse — Consolidate & Optimise: unify structured and unstructured data in a single platform with ACID transactions, schema enforcement, and warehouse-grade query performance on lake-scale storage. Ideal for historical analytics, ML model training, batch reporting, and regulatory archives. Data Federation — Access in Place: query data wherever it lives without moving or duplicating it. Ideal for real-time operational intelligence, cross-source joins, data sovereignty compliance, and live enrichment from transactional systems.

Why "Just Lakehouse" Falls Short

Pipeline latency: data must be extracted, transformed, and loaded before it can be queried — creating a gap for real-time credit decisions. Data gravity: every dataset consolidated is a copy, creating sovereignty risk under DPDPA and GDPR. Lineage fragmentation: when data is extracted from a core banking system and loaded into the lakehouse, its provenance chain is interrupted, making the FREE-AI "Understandable by Design" sutra structurally harder to satisfy.

Why "Just Federation" Falls Short

Performance ceiling: federated queries across distributed systems face network latency and heterogeneous schema reconciliation — inadequate for complex ML model training that iterates over billions of rows. No persistent storage: federation is stateless by design — for regulatory reporting, historical backtesting, and audit archives, enterprises need persistent governed storage. Metadata fragmentation: without a central catalogue, governance across federated sources requires each source to maintain its own metadata independently.

The Convergence Architecture

Three layers, one governance boundary. Lakehouse provides depth: consolidated, optimised, governed historical data for ML training, regulatory archives, and model versioning using open formats (Iceberg, Delta, Hudi). Federation provides breadth: real-time access to core banking, DPI rails (ULI, AA, UPI), third-party APIs, and operational databases. A unified governance control plane spans both — enforcing access policies, tracking lineage, monitoring bias, and producing audit trails regardless of which layer was accessed.

What This Means for Agentic AI

A credit decisioning agent needs a risk model trained on years of historical lending data from the lakehouse, and real-time enrichment of the current application with UPI patterns via Account Aggregator, land records via ULI, and GST filings at source — all through federation. The governance layer ensures this complete decision carries a unified audit trail. The regulator sees one explainable decision with end-to-end traceability, not two separate systems.

The lakehouse provides the storage, metadata, and governance foundation that makes federation practical at enterprise scale. When paired with federation, these features become even more powerful.

— Starburst, "Federated Data: How Does a Data Lakehouse Help?"