What is retrieval-augmented generation (RAG)?
Retrieval-augmented generation (RAG) is a technique that makes a large language model look things up before it answers. The system searches a trusted source — your documents, a database, or the web — for passages relevant to the question, adds them to the prompt, and the model generates an answer grounded in those passages. The model itself is not retrained.
The term comes from a 2020 research paper by Lewis and colleagues at Facebook AI Research, but the idea is now the default pattern for enterprise chatbots, internal knowledge assistants and AI search. If you are preparing for the Google Generative AI Leader or the AWS AI Practitioner exam, RAG is one of the concepts you are most likely to meet in scenario questions.
Why RAG exists: the problems it solves
A foundation model on its own has three limitations that matter in business settings:
- Knowledge cutoff. It only knows what was in its training data, which stops at a point in time.
- No private knowledge. It has never seen your HR policies, contracts, product specs or support tickets.
- Hallucinations. When it lacks a fact, it can produce a fluent, confident and wrong answer.
Google's Generative AI Leader exam guide lists exactly these limitations — data dependency, the knowledge cutoff, bias, hallucinations — and names grounding and RAG among the recommended practices to address them. A useful way to remember it: fine-tuning changes what the model is; RAG changes what the model can see.
How RAG works, step by step: embeddings, vector search, grounding
1. Chunk and embed your documents (done ahead of time)
Source content — policies, manuals, product pages, tickets — is split into chunks of a few paragraphs. Each chunk is passed through an embedding model, which turns text into a vector: a long list of numbers that represents its meaning. Chunks about similar ideas end up with vectors that are close together, even when they use different words.
2. Store the vectors in a vector index
The vectors, plus a pointer back to the original text, go into a vector database or search index built for fast similarity lookups. This is the "library" the model will consult. When documents change, you re-embed and update the index — no model retraining involved.
3. Retrieve relevant chunks at question time
When a user asks a question, the question is embedded with the same model, and the index returns the chunks whose vectors are nearest to it (vector or semantic search). Many systems combine this with keyword search and a re-ranking step to push the most relevant passages to the top.
4. Ground the prompt
The retrieved chunks are inserted into the prompt alongside the user’s question, usually with an instruction such as "answer only from the sources below and cite them." This is the "augmented" part: the model now has the facts it needs in its context window.
5. Generate the answer
The foundation model writes a response based on the supplied context. Well-built RAG systems return citations or links to the source chunks so users can check the answer — one of the main reasons enterprises prefer RAG for factual Q&A.
A concrete example: an employee asks an HR assistant, “How many days of parental leave do I get?” Retrieval finds the two most relevant paragraphs of the current leave policy, the prompt becomes “Using only these policy excerpts, answer the question and cite the section,” and the model answers with the number and a link to the policy. When HR updates the policy next quarter, only the index changes.
RAG vs fine-tuning vs prompt engineering
These are the three customization levers you will be asked to choose between on both exams. They are complementary, not mutually exclusive, but each has a clear sweet spot.
| Prompt engineering | RAG | Fine-tuning | |
|---|---|---|---|
| What changes | The instructions and examples you send | The facts supplied in the prompt at query time | The model's weights, via further training |
| Adds new knowledge? | Only what fits in the prompt you write | Yes, from an external source you control | Partly, but it is baked in and goes stale |
| Keeping data current | Manual | Update the index; no retraining | Requires another training run |
| Relative cost and effort | Lowest | Moderate (embeddings, index, retrieval) | Higher (labeled data, training compute, evaluation) |
| Citations / traceability | No | Yes, can point to source passages | No |
| Best for | Task framing, format, tone; first thing to try | Q&A over private, large or changing data | Consistent style, specialized tasks or domain language |
Exam shortcut: if a scenario stresses “up-to-date,” “internal documents,” “cite sources” or “without retraining,” think RAG. If it stresses “lowest cost” and “quickest” with no new data, think prompt engineering. If it stresses a consistent specialized style or behavior that prompting cannot achieve, think fine-tuning. Training a model from scratch is almost never the right answer at the foundational level.
Grounding with Google Search and your own data on Google Cloud
Google frames RAG under the broader idea of grounding: connecting model output to verifiable sources. The current GAIL exam guide asks you to differentiate grounding with first-party enterprise data (your own documents and databases), third-party data, and world data (public information), and it names three Google Cloud grounding offerings:
| Offering | Grounds the model in | Typical fit |
|---|---|---|
| Grounding with Google Search | Public web information via Google Search | Questions about recent events or public facts beyond the model's knowledge cutoff |
| Pre-built RAG with Agent Search | Your enterprise content, indexed for you | Fastest route to search and Q&A over company documents with little engineering |
| RAG APIs | Your data, through a pipeline you assemble | Teams that want control over chunking, embeddings and retrieval |
A naming note: in April 2026 Google rebranded Vertex AI as the Gemini Enterprise Agent Platform, and the current exam guide uses the new product names (Agent Platform, Agent Search, Agent Studio). Much existing study material still says Vertex AI and Vertex AI Search, so expect to see both. The exam is about choosing the right kind of grounding for the scenario, not configuration details.
The key decision: public, fast-changing facts point to grounding with Google Search; private company data points to RAG over enterprise data. Getting that distinction right is a recurring theme in the Generative AI Leader exam guide and the GAIL cheat sheet.
Where RAG shows up on GAIL and AIF-C01
| Exam | Where RAG appears in the official guide | What you need to know |
|---|---|---|
| Google Generative AI Leader | Section 2 (Google Cloud's gen AI offerings, ~35%): Google Cloud's RAG offerings. Section 3 (Techniques to improve model output, ~20%): limitations, grounding techniques, how RAG affects output | When to recommend RAG or grounding vs prompting or fine-tuning; enterprise vs world-data grounding; Google's grounding offerings |
| AWS AI Practitioner (AIF-C01) | Domain 3 (Applications of Foundation Models, 28%): define RAG and its business applications, vector database services, cost trade-offs of customization | Amazon Bedrock Knowledge Bases; vector stores such as OpenSearch Service, Aurora, Neptune, RDS for PostgreSQL; RAG vs fine-tuning vs in-context learning costs |
The angle differs. GAIL asks what a leader should recommend and which Google offering fits. AIF-C01 asks which AWS service implements the pattern. If you are deciding between the two certifications, see GAIL vs AWS AI Practitioner.
Sample question: choosing the right grounding
Sample Question (GAIL — Techniques to improve model output)
A travel company's gen AI assistant must answer two kinds of questions: details of the company's own cancellation and refund policies, and current public information such as airport disruptions reported today. Which combination best reduces hallucinations for both?
- Fine-tune the model on the policies and on last year's news archive
- Use RAG over the policy documents, and grounding with Google Search for current public information
- Use grounding with Google Search for both types of question
- Use few-shot prompting with examples of good answers for both types of question
Answer: B. Private policies are first-party enterprise data, best served by RAG over the company's documents. Today's disruptions are fresh world data, best served by grounding with Google Search. A goes stale immediately and cannot know today's events. C cannot see private policies. D improves format and tone but supplies no new facts.
Notice the pattern: identify where the facts live (private vs public, static vs changing) before picking a technique. For more items like this, try the GAIL practice questions.
RAG limitations: what it does not fix
- Garbage in, garbage out. If retrieval returns the wrong or outdated passage, the model will confidently answer from it. Data quality and freshness of the index matter as much as the model.
- It can still hallucinate. The model may misread or over-generalize from retrieved text, or fill gaps when nothing relevant is found. Instructions to answer only from sources, citations and evaluation help.
- Chunking and context limits. Answers that span many documents may not fit in the retrieved chunks or the context window, and poor chunking can split key facts apart.
- Latency and cost. Every query adds an embedding call, a search and a longer prompt. That is usually cheaper than fine-tuning, but not free.
- Access control. Retrieval must respect who is allowed to see which documents, or the assistant can leak restricted content to the wrong user.
- Untrusted content. Retrieved documents or web pages can contain instructions that try to manipulate the model (indirect prompt injection), so sources need vetting.
- It does not change behavior. RAG supplies facts, not a new writing style or skill. If you need consistent specialized behavior, combine it with prompting or fine-tuning.
These limitations are exactly why both exams pair RAG with responsible AI, human-in-the-loop review and ongoing evaluation. To build all of this into a study schedule, see the GAIL study plan.
Frequently asked questions
What is retrieval-augmented generation in simple terms?
Retrieval-augmented generation (RAG) is a technique where an AI system looks up relevant information from a trusted source — such as your company documents or a search engine — and gives it to a large language model along with the question, so the answer is based on those facts instead of only on what the model memorized during training.
Is RAG better than fine-tuning?
Neither is better in general; they solve different problems. RAG is the usual choice when the model needs access to specific, changing or private facts, and when you want citations. Fine-tuning is the usual choice when you need the model to adopt a consistent style, format or specialized behavior that prompting cannot achieve. Many production systems use both. On certification exams, scenarios that mention frequently updated data and avoiding retraining almost always point to RAG.
Does RAG eliminate hallucinations?
No. RAG reduces hallucinations by giving the model relevant facts, but a model can still misread the retrieved text, blend sources incorrectly, or answer confidently when retrieval returned nothing useful. Mitigations include instructing the model to answer only from sources, returning citations, evaluating retrieval quality, and keeping a human in the loop for high-stakes outputs.
What is the difference between RAG and grounding?
Grounding is the broader goal: connecting a model’s output to verifiable sources of information. RAG is one common way to achieve it, by retrieving documents and adding them to the prompt. Google’s Generative AI Leader exam guide treats grounding as the umbrella and distinguishes grounding with first-party enterprise data, third-party data, and world data such as Google Search.
Do I need to know how to build a RAG pipeline for GAIL or AIF-C01?
No. Neither exam asks you to write code. GAIL expects you to know what RAG and grounding are, when to recommend them, and which Google Cloud offerings provide them. AIF-C01 expects you to define RAG, recognize Amazon Bedrock Knowledge Bases, identify AWS services that can store embeddings in vector databases, and understand the cost trade-offs between RAG, fine-tuning and other customization approaches.
What is a vector database?
A vector database stores embeddings — numeric representations of text, images or other data — and finds the stored items most similar to a query vector. In a RAG system it is the index that retrieval searches. Examples include managed services from cloud providers and open-source options; on AWS, the AIF-C01 guide names Amazon OpenSearch Service, Amazon Aurora, Amazon Neptune and Amazon RDS for PostgreSQL.
Sources & official references
- Google — Generative AI Leader exam guide (PDF)
- Gemini Enterprise Agent Platform release notes (Vertex AI rename)
- AWS — AIF-C01 exam guide
Exam facts checked against the official pages above in October 2026. Google does not publish a GAIL passing score; ~70% is a community estimate.
Test Your RAG Reasoning
500 Generative AI Leader practice questions on grounding, RAG, prompting and fine-tuning trade-offs. Start with 30 free questions per session with a free account — no card needed.