What RAG is
RAG stands for retrieval-augmented generation. Before the model answers, the system searches the company’s documents for passages related to the question and passes them along with it. The model then answers from that information rather than only from what it learned during training.
It isn’t training the model on all your files, which is what many people assume. Quality depends mainly on which passages are retrieved and how the answer is validated. That is why I always start with a set of real questions and the documents that should answer them.
Processing the documents
First the content has to be extracted correctly, which takes more work than it seems. A scanned PDF needs optical character recognition, and a table loses its meaning if separated from its headers. I review this stage before judging the answers, because many errors that look like the model’s originate here.
The documents are then split into passages that keep a reference to their source. Search can combine exact term matching with semantic similarity. In a company, with product codes and internal references, exact matching is often decisive, and it has to be tested with the organisation’s own vocabulary.
Permissions and references by design
The assistant should only retrieve what the person asking is entitled to see. Hiding a link after the model has read restricted information is useless, because the content may already be in the answer. Permissions are applied before building the context the model receives.
Every answer should include its references and acknowledge when the documentation doesn’t allow an answer. A citation makes verification easier, but doesn’t by itself guarantee the conclusion is correct.
Architecture and maintenance
There are libraries that handle much of the work, and sometimes it pays to build a component in-house. The decision depends mainly on where the system must be hosted and who will maintain it. A conversational interface suits one-off queries; if the answer must appear inside another application, I expose it as an API.
Before delivery I test the most fragile points: documents that are updated or withdrawn and permissions that change. I also define who reviews errors and what is logged, without keeping unnecessary data.
Further reading
If you are considering an assistant on your company’s knowledge, here is how I work, or book a meeting and we’ll look at your case.