How Much Does a RAG System Cost? Build and Running Costs Explained

Short answer: a retrieval-augmented generation (RAG) system has two costs. The build is set by six things: the number of data sources, the volume and messiness of the content, per-user access control, the accuracy you need, how often the content changes and where it must run. The running cost is embeddings, the vector database, language model usage per question, hosting and monitoring. A focused assistant over one knowledge base is typically a 2–4 week build; multi-source, permission-aware systems take longer.

What a production RAG system is made of

1. Connectors that pull documents from where they live: drives, SharePoint, Confluence, a CRM, a database or a website.
2. Processing: cleaning, splitting documents into passages and handling tables, scans and attachments.
3. Indexing: embeddings in a vector database, plus keyword search for exact terms such as product codes and names.
4. Retrieval: hybrid search and re-ranking to find the best passages for each question.
5. Generation: a language model that answers from those passages and cites them.
6. Access control: each user sees answers only from documents they are allowed to read.
7. Evaluation and monitoring: a test set, logs and feedback so accuracy is measured, not assumed.

The six cost drivers

• Data sources: each connector needs authentication, sync logic and error handling.
• Content volume and quality: clean, current documents are cheap; scanned PDFs, duplicates and outdated versions need extra work. See document AI.
• Access control: mirroring permissions from SharePoint or a CRM adds real engineering.
• Accuracy target: an internal FAQ helper and a compliance assistant need very different testing.
• Freshness: content that changes hourly needs incremental sync; a quarterly manual needs little.
• Deployment: hosted model APIs are fastest; a private model on your own servers costs more to set up but removes per-question API fees and keeps data in-house.

Running costs after launch

• Embeddings for new and changed content (small once the first index is built).
• Vector database hosting.
• Language model usage per question, the largest variable cost.
• Hosting for the API and connectors.
• Monitoring and improvement, optionally as a monthly retainer.

Keep them predictable with caching for repeated questions, a smaller model for simple questions, sensible limits per user and short, well-chosen context.

Typical timelines

• Focused assistant over one knowledge base: about 2–4 weeks.
• Multi-source assistant with per-user permissions: about 4–8 weeks.
• Enterprise rollout across departments with evaluation, monitoring and private deployment: phased over several milestones.

How to test accuracy before you scale

1. Collect 50–100 real questions from the people who will use it, with the correct answers.
2. Measure how often the right passage is retrieved, and how often the answer is correct and cited.
3. Fix the biggest causes first: missing documents, poor splitting, weak search or unclear instructions.
4. Re-run the same test after every change, and keep sampling real questions after launch.

Custom, off-the-shelf or hybrid?

Off-the-shelf "chat with your documents" tools are fine for a small team and public content. Build custom when you need your own permissions, several systems combined, private deployment, or answers inside your own product. A hybrid, using a managed vector database and model API with your own retrieval and access logic, is often the best value.

What to send for an accurate quote

The sources and rough document counts, who will use it and what they may see, ten example questions, where it must run, and how you will judge success. With that, we return a written scope and a fixed quote. More detail: enterprise RAG explained and hybrid RAG for accurate answers.

Work with Nexariza AI

We build enterprise RAG systems with hybrid search, re-ranking, source citations, per-user permissions and evaluation, on hosted models or fully private infrastructure, and you own the code. Get a free scope within 24 hours.

Frequently asked questions

What drives the cost of a RAG system?

Six things: how many data sources it connects, how much and how messy the content is, access control per user, the accuracy target, how often content changes, and where it must run (cloud API or private servers).

What are the running costs of RAG?

Embedding new and changed content, the vector database, language model usage per question, hosting, and monitoring. Caching, smaller models for simple questions and limits per user keep them predictable.

How long does it take to build a RAG system?

A focused assistant over one well-organised knowledge base typically takes 2–4 weeks. Systems that combine several sources with per-user permissions and an evaluation suite take longer.

Can a RAG system run privately?

Yes. The vector database and an open-source language model can run on your own servers or private cloud, so documents and questions never leave your infrastructure.

How do you stop a RAG system from making things up?

Hybrid search with re-ranking finds the right passages, answers cite their sources, and the system is instructed to say it does not know when the answer is not in the documents. A test set of real questions measures accuracy before launch.