Use cases

Document AI assistant (RAG): query your internal documents

How a document AI assistant finds information in your internal documents and answers with cited sources, and how to tell whether it really works.

6 min read ORIA, Antananarivo

Internal procedures, contracts, circulars, terms and conditions, staff memos: in most organisations, the information already exists. The problem is finding it. Employees dig through shared folders, interrupt the colleague "who knows", and sometimes get different answers to the same question.

A document AI assistant addresses exactly that. Your teams ask their question in plain language and receive a written answer, grounded in your documents, with the sources cited. The technique behind it goes by an unappealing acronym, RAG. Here is what you need to know, without the jargon.

What a document AI assistant is, and what it is not

It is a question-and-answer interface connected to a set of documents you choose, known as the corpus. Every answer relies on specific passages from those documents, which the user can open to check.

It is not:

  • a consumer chatbot answering from general knowledge picked up on the internet;
  • a plain search engine returning a list of files to wade through;
  • an oracle: when the information is not in your documents, a well-designed assistant says so instead of making something up.

How RAG works, in plain terms

RAG stands for Retrieval-Augmented Generation. It works in four stages.

  1. Prepare the documents. Documents are collected, their text is extracted (with character recognition for scans) and split into passages. Each passage is converted into a numerical representation of its meaning, called a vector, and stored in an index along with useful metadata: title, date, version, access rights.
  2. Retrieve the right passages. When someone asks a question, the system looks for the most relevant passages, by meaning rather than keywords alone, among those that user is allowed to see.
  3. Generate the answer. A language model receives the question and the selected passages, with a strict instruction: answer only from these passages, and cite them.
  4. Return it with sources. The user gets the answer plus exact references (document, section, page) to verify in one click.

This design has three practical advantages. The model does not need to be retrained on your data. Updating a document simply means re-indexing it. And every answer is traceable.

Golden rule: no source found, no invented answer.

Use cases by sector and function

Banking and insurance

Internal procedures, policy terms, circulars, product sheets: advisers find the applicable rule without interrupting a colleague, and compliance teams check more quickly that a practice follows the current procedure.

Telecoms

Frequently changing offers and tariffs, technical procedures, support scripts: the assistant helps teams answer with the current version, not last month's.

Human resources

Internal rules, collective agreements, leave, travel and recruitment procedures: employees' recurring questions get a consistent answer, and HR can focus on the cases that need judgement.

Legal and compliance

Finding a clause across a portfolio of contracts, comparing versions, identifying the obligations in a regulation or internal policy. The assistant prepares; the lawyer decides.

Customer support

A knowledge base you can query in plain language reduces dependence on your most experienced agents and makes the answers customers receive more consistent.

What to measure

A document assistant should be judged on facts, not on a demo. Before the project, measure the baseline: average time to find information, correct-answer rate, how much answers vary between employees.

During the pilot, build a reference set with your subject-matter experts: real questions, each with the correct answer and the document that supports it. It lets you track:

  • the rate of correct and complete answers;
  • citation accuracy: does the cited source actually contain the information?
  • the appropriate abstention rate: does the assistant admit it does not know when the answer is not in the corpus?
  • search time, before and after;
  • adoption: active users, questions asked, user feedback.

The reference set is replayed after every change (new model, new documents, new settings) to make sure quality does not slip.

Security: what to check

  • Access rights. A user must never receive an answer drawn from a document they could not open themselves. Filtering happens at retrieval time, not afterwards.
  • Personal and confidential data. Identify it before indexing, exclude what is not needed and comply with Madagascar's Law No. 2014-038 on personal data protection.
  • Where the data lives. Your documents can stay on your infrastructure, with only minimised excerpts sent to the model; or everything can run on your premises with open-source models. If an external model is used, make sure the contract prevents your data from being used to train it.
  • Logging. Who asked what, and which sources were served: this traceability is essential for audits and incident handling.
  • Prompt injection. A document can contain text designed to hijack the model. A curated corpus, clear separation between instructions and content, and dedicated tests reduce the risk.

Our principles and deployment options are set out on our security and compliance page.

Which documents to start with

The first corpus largely determines whether the pilot succeeds. Favour documents that tick most of these boxes:

  • they are consulted often, by many people;
  • they exist in digital form, with an identified reference version;
  • they have an owner who can validate answers and flag updates;
  • they contain little personal data, or data that is easy to exclude;
  • the questions they raise are already known, which makes the reference set easier to build.

A collection of internal procedures or the history of questions sent to your support team often meets these criteria better than years of contract archives.

What a typical pilot looks like

A pilot of an enterprise document assistant usually runs for four to six weeks. It covers a narrow scope: one department, a set of reference documents, a group of pilot users. It follows four steps:

  1. Scoping: choose the scope, the users and the reference set; measure the baseline.
  2. Preparation: clean up the corpus (remove outdated versions), index it, configure access rights.
  3. Trial: real use by the pilot group, feedback collection, adjustments.
  4. Measurement and decision: before/after comparison on the reference set, then a decision to extend, adjust or stop.

On your side, three contributions make the difference: an available business owner, identified reference documents and an IT contact for access.

Limits to be aware of

  • Answer quality depends on document quality: two conflicting procedures will produce hesitant answers.
  • Complex tables, poor scans and handwritten documents need specific treatment.
  • Documents mixing French, Malagasy and English must be tested in each language.
  • A corpus evolves: without regular updates, the assistant ages with it. That is the purpose of an operations, maintenance and training contract.
  • For decisions that commit the company, the assistant prepares and a human approves.

If your need is less about finding information and more about processing incoming documents (invoices, forms, letters), intelligent process automation is the better fit.

Wondering which document set to tackle first? Let's discuss it during a free 45-minute diagnostic call, or start with our AI readiness self-assessment.

Frequently asked questions

What is RAG?
RAG stands for Retrieval-Augmented Generation. The system first retrieves the relevant passages from your documents, then a language model writes an answer based only on those passages and cites them.
Do we need to train an AI model on our documents?
No. Documents are indexed, not used to retrain the model. Updating a document simply means re-indexing it.
Can a document assistant make up an answer?
The risk exists with any language model. It is reduced by requiring answers based only on retrieved passages, with cited sources and abstention when no source is found, and by measuring quality on a reference set of questions.

Read next

Next step

45 minutes to identify your first win.

A structured conversation with an AI architect — no commitment, no jargon.

Book my free assessment