RAG Pipeline Architecture Flowchart

How retrieval-augmented generation answers questions from your own documents: documents are split, embedded and indexed, then each question pulls the best passages and the model answers with citations.

5 teams14 steps1 decisionsFree download

Edit this flowarrow_forward
RAG pipeline processContent sources1Ingestion pipeline4Retrieval service3Language model3User3Searched at query timeYesNoDocumentsadded orchanged3Extract andclean the text5Split intochunks6CreateembeddingsStore in thesearch indexUser asksa question4Rewrite thequestion forsearch8Find matchingpassages9Rerank andkeep the best?Enoughrelevantcontent?Say itisn'tcovered11Write theanswer withcitations13Check theanswer isgroundedAnswerwithsources

Free to download. Edit it in Flow Studio to change the teams and steps, ask AI for changes, or export to PowerPoint, Word or draw.io.

Step by step

The same flow as a numbered list, ready for an SOP, a BRD or a ticket.

#Step
1Documents added or changedStartSharePoint, Google Drive, Confluence, Zendesk, PDFs; change events or a nightly syncContent sources · step 3
2User asks a questionStartUser · step 4
3Extract and clean the textParse PDF/HTML/DOCX, OCR scans, strip boilerplate, keep headings and tablesIngestion pipeline · step 5
4Rewrite the question for searchResolve follow-ups using chat history; optional multi-query expansionLanguage model · step 8
5Split into chunks300-800 tokens, 10-20% overlap, split on headings; attach source, date and access listIngestion pipeline · step 6
6Create embeddingsEmbedding model (e.g. text-embedding-3, Cohere, bge); batch, cache by content hashIngestion pipeline · step 7
7Store in the search indexDocumentpgvector, Azure AI Search, Pinecone or Weaviate; plus a keyword index for hybrid searchIngestion pipeline · Searched at query time: step 8
8Find matching passagesHybrid search, filter by the user's permissions, top 20-50 candidatesRetrieval service · step 9
9Rerank and keep the bestCross-encoder reranker to 5-8 passages; drop below a relevance thresholdRetrieval service · step 10
10Enough relevant content?DecisionRetrieval service · Yes: step 11, No: step 12
11Write the answer with citationsAnswer only from the passages; cite passage ids; structured outputLanguage model · step 13
12Say it isn't coveredEndUser · Done
13Check the answer is groundedFaithfulness check: every claim maps to a cited passage; strip or flag unsupported claimsLanguage model · step 14
14Answer with sourcesEndLinks to the source sections; thumbs up/down feeds the eval setUser · Done

When to use it

Use it to explain or build an AI assistant that answers from company knowledge (policies, product docs, contracts, tickets) instead of from the model's general memory.

Who's involved

  • Content sources1
  • Ingestion pipeline4
  • Retrieval service3
  • Language model3
  • User3

Yours works differently?

Describe your own process in a few sentences and Flow Studio draws it, with your teams and your rules.

Describe your processarrow_forward

For the technical team

The build decisions behind this flow. Each step above also carries its technical detail.

  • 01Ingestion: connectors for each source, text extraction (incl. OCR for scans), cleaning, and a content hash so unchanged docs aren't re-embedded.
  • 02Chunk metadata: source URL, title, section, updated date and access-control list, stored with each vector.
  • 03Retrieval: hybrid BM25 + vector search, top 20-50 candidates, then a cross-encoder reranker down to 5-8 passages.
  • 04Generation: system prompt that requires citations by passage id; refuse when retrieval confidence is low.
  • 05Evaluation: track retrieval recall@k and answer faithfulness on a fixed question set on every change.

How to get it right

  1. 1Most bad answers come from bad retrieval, not the model. Check which passages were found before changing the prompt.
  2. 2Keep document permissions on every chunk and filter at query time, so people only get answers from what they're allowed to read.
  3. 3Ask the model to answer only from the passages and to say when they don't cover the question. Show the citations to users.
  4. 4Re-index on a schedule or on change. Stale chunks are a quiet source of wrong answers.

Questions

What is a RAG pipeline?

Retrieval-augmented generation finds the most relevant passages from your own documents for each question and gives them to the language model, so it answers from your content and can cite it.

What chunk size should I use for RAG?

A common starting point is 300 to 800 tokens with 10 to 20% overlap, split on headings and paragraphs rather than fixed lengths. Test on your own questions: dense legal text and short FAQs need different sizes.

Do I need a vector database?

You need somewhere to store embeddings and search them. Postgres with pgvector is enough for most teams; dedicated stores like Pinecone, Weaviate or Azure AI Search help at very large scale. Hybrid search (keywords plus vectors) usually beats vectors alone.

More process flow templates

See all 34 process flow templates