A retrieval-augmented generation system is only as good as the data platform beneath it. It needs raw documents at full fidelity, metadata to filter on, classification that reaches the retrieval layer, and an index that stays current as sources change. A lakehouse provides all four on one governed copy; a two-tier lake-and-warehouse estate provides none of them cleanly.
That is the whole argument, and the rest of this article is the engineering detail behind it.
Why does RAG depend on the data platform at all?
Because retrieval is a data problem wearing an AI label. The model is the least differentiated component in the system — the same model is available to everyone. What differs between an assistant that works and one that hallucinates is what it was given to read.
That makes four platform properties decisive.
Fidelity. Retrieval works on the source content: contracts, procedures, tickets, correspondence, transcripts, and full-granularity records. A curated warehouse table has already discarded the detail that grounds an answer.
Filterable metadata. Effective retrieval is rarely pure similarity search. It is similarity search constrained by metadata — this department, this document type, current version only, this customer segment. That metadata has to exist and be attached to the content.
Classification that the retrieval layer can read. What a given user may be shown depends on how the source asset is classified. If classification exists only in a policy document, the retrieval layer cannot enforce anything.
Freshness. A superseded procedure retrieved as current produces a confident wrong answer, which is worse than no answer.
A two-tier estate fails on all four: the raw layer has the fidelity and none of the governance, the warehouse has the governance and none of the fidelity, and neither has classification the retrieval layer can consume. This is the same structural argument that drives enterprises to lakehouse architecture generally, sharpened by an AI workload.
Where should embeddings live?
Three viable patterns, and the choice has real consequences.
A dedicated vector database. Purpose-built for approximate nearest-neighbour search at scale, with mature indexing. The cost is another system to operate, secure and govern, and a second place where derived personal data lives.
Vector search inside the lakehouse. Embeddings stored as a column alongside the content, in the same table format, under the same access control and lineage. Operationally simpler and governance-consistent; historically weaker on pure search performance at very large scale, though the gap has narrowed considerably.
A hybrid. Content and metadata in the lakehouse as the system of record, embeddings replicated into a vector store as a derived index that can be rebuilt at any time.
The hybrid is what Yukon Labs deploys most often, for one specific reason: treating the vector index as derived rather than authoritative makes re-embedding routine. Embedding models improve, chunking strategies change, and a system that cannot re-embed its entire corpus without a migration project is a system frozen at its first design.
Whichever pattern is chosen, one rule is not negotiable: the embedding model and the vector store sit on the same side of the boundary as the source documents. A vector embedding derived from a document containing personal data is itself derived personal data — an argument developed in data residency and personal data law in Azerbaijan. Sending documents to a hosted embedding service is a cross-border transfer, regardless of retention promises.
How does access control reach the retrieval layer?
This is the question that separates a demo from a production system, and it is where most enterprise RAG projects stall.
The naive implementation indexes everything into one vector store and searches it for every user. This works until the first time an assistant surfaces a salary figure, a disciplinary record or a customer's data to someone who should not see it — and because the model paraphrases rather than quoting a file path, the exposure is harder to detect than a file share misconfiguration.
Three mechanisms, used together:
Filter at query time on classification and entitlement. Every chunk carries the classification and access attributes of its source asset. Retrieval filters on the user's entitlements before similarity ranking, not after. Filtering after ranking is a common implementation shortcut that silently returns fewer results to restricted users rather than the right ones.
Partition indexes by sensitivity tier. Highly restricted content in a separate index that most users' queries never touch. This limits blast radius and simplifies the audit story.
Enforce at the data layer as well. Where the assistant queries structured data rather than documents, the row- and column-level policy in the query engine applies. This is the concrete AI-era benefit of federated access control in Starburst: the same policy that governs an analyst's SQL governs an agent's query, without a second policy engine to keep in sync. The mechanics are in Starburst and Trino in Azerbaijan.
The attributes all three depend on come from the catalog. This is why AI projects on ungoverned estates stall at exactly this point: the question "what may this user see" has no recorded answer, and answering it retroactively across thousands of assets is a governance programme, not a sprint.
What does the ingestion pipeline for retrieval look like?
Six stages, and each one has a decision that materially affects answer quality.
Extraction. Documents to text, preserving structure. Headings, tables and lists carry meaning, and an extractor that flattens a table into a run-on sentence destroys the content most likely to be asked about. Scanned documents need OCR, and OCR quality sets a ceiling on everything downstream.
Chunking. Splitting content into retrievable units. Fixed-size chunking is simple and cuts across semantic boundaries; structure-aware chunking that respects sections and clauses performs better on the document types enterprises actually hold — contracts, procedures, regulations. Overlap between chunks reduces boundary losses at the cost of index size.
Metadata attachment. Every chunk carries source document, section, version, effective date, owner, classification and language. This is the stage teams skip, and skipping it makes filtering, currency rules and citation impossible later.
Embedding. Text to vectors. Model choice matters more for non-English content than teams expect — Azerbaijani is far less represented in general-purpose embedding models than English or Russian, which is the practical reason to test retrieval quality per language rather than assuming it transfers.
Indexing. Writing to the vector store with the metadata as filterable attributes.
Registration. Recording in the catalog that this asset is now in retrieval scope. Without this step nobody can answer what the assistant can see, which is the first question in any review.
How do you keep the index current?
The hardest operational requirement, and the one most often solved with a full nightly rebuild that becomes untenable as the corpus grows.
Change detection at the source. For documents, file modification events or a scheduled scan. For structured data, the same CDC mechanism the analytical platform uses — covered in real-time analytics: streaming, CDC and the lakehouse.
Incremental re-embedding. Only changed content is re-embedded. This requires stable chunk identity, which requires deterministic chunking — a design decision that is expensive to retrofit.
Deletion that propagates. When a source document is deleted, its chunks and embeddings must be removed. This is both a quality requirement and a legal one: an erasure request that leaves embeddings behind has not been fulfilled.
Version handling as a first-class concern. Superseded documents should either leave the index or carry an effective-date attribute that retrieval filters on. This single control prevents the most common category of wrong answer in enterprise deployments — confidently citing a policy that was replaced two years ago.
Why does the lakehouse make this tractable?
Five properties, each of which is otherwise an integration project.
One governed copy. Content and structured data under the same access control and classification, so retrieval scope is expressible.
Open table formats give time travel. Reproducing what the system would have retrieved on a given date is possible, which matters when explaining a past output during an audit — a requirement that appears directly in ISO/IEC 42001.
Metadata and content stay together, so chunk-level attributes are queryable rather than trapped in a vector store.
The same engine serves analysts and agents. An agent answering "what was the loan volume last quarter" runs a query, and it runs under the same policy an analyst would face.
Lineage covers the AI path too. Which sources feed which index, and therefore which reports and which assistants are affected when a source changes.
How these fit into the platform as a whole is set out in building a modern data platform.
What does this look like on-premise?
For banks and state institutions here, every component runs inside the perimeter: object storage, table format, query engine, embedding model, vector index and inference.
Two practical notes from deployments.
Embedding is cheaper than inference and easy to underestimate at corpus scale. Embedding a large document estate is a substantial one-off batch job, and re-embedding after a model change repeats it. Size for that rather than for steady-state query load, and schedule it like any other batch workload.
Retrieval quality is the bottleneck, not model size. Teams reach for a larger model when answers disappoint, when the actual defect is almost always chunking, missing metadata or stale content. The GPU sizing and cost reasoning is in deploying LLMs on-premise; the retrieval-quality argument is developed in why context matters in enterprise AI.
Key points
- Retrieval quality is a data platform property. The model is the least differentiated component in the system.
- A RAG system needs fidelity, filterable metadata, readable classification and freshness. A two-tier estate provides none of them cleanly.
- Treat the vector index as derived, not authoritative, so re-embedding is routine rather than a project.
- Keep the embedding model and vector store on the same side of the boundary as the source documents.
- Filter on entitlement before similarity ranking, partition indexes by sensitivity, and enforce policy at the data layer for structured queries.
- Attach metadata at chunking time — version, effective date, owner, classification, language. Retrofitting it is not practical.
- Handle deletion and superseded versions explicitly; stale documents retrieved as current are the most common wrong-answer category.
- Test retrieval quality per language. Azerbaijani is materially less represented in general-purpose embedding models.
Yukon Labs builds the retrieval layer on governed lakehouse foundations, with Starburst providing federated query and consistent access control, OvalEdge providing the classification the retrieval layer reads, and HAVAA as the orchestration layer above them. For the architectural background, see data lakehouse architecture.