Connecting AI to your data · Layer 4
RAG (retrieval-augmented generation)
A method that first retrieves the relevant passages from your documents, then asks the model to answer from them, citing its sources.
A way to picture itIt is an open-book exam: instead of answering from memory, the model looks up the right pages of your documentation before writing, and can show where each statement comes from.
A compliance department provides an assistant that answers staff questions on 400 internal procedures. Each answer cites the relevant procedure and paragraph, with a link to the document. When a procedure is revised, the index is updated the same evening, without retraining any model. Questions left unanswered are passed back to the department, which fills the gaps in the documentation.
An organisation that connects an AI to its own knowledge base is responsible for that processing as soon as the base contains personal data (France · CNIL, 18 July 2024). RAG must apply the source documents’ access rights to every question; otherwise a user can obtain, in one sentence, a summary of a note they are not allowed to open, which is a security failure (EU · GDPR, art. 32). Logging questions, retrieved passages and answers lets you prove what was answered, and on what basis; for high-risk uses, this logging is required (EU · AI Act, arts. 12 and 26).
“How does the tool apply our documents’ access rights to every question, and can you demonstrate it in front of us with two different user profiles?”
To have the AI answer from facts that change (procedures, contracts, catalogues), RAG is generally the right choice: it stays up to date and cites its sources. Fine-tuning is better suited to fixing a style, a format or a repetitive task.
Not always: a small body of documents can be placed in the prompt in full. Beyond that, RAG remains faster, cheaper and easier to control, because it selects and traces the passages used.
Yes, every component can be: documents, vector database, embedding model, generation model and logs. Check it component by component: a single component hosted elsewhere is enough for data to leave.
Educational explainer, not legal advice. Up to date as of 29 October 2026.