AI Engineering: Context, RAG & Citations

What you will learn

How to provide the right information to an AI, the basics of Retrieval-Augmented Generation (RAG), and how to reduce hallucinations with structured context.

Why context structure matters

An AI is only as good as the context you give it. If the answer to your question is in a 200-page document but you only send 20 pages, the AI may guess (and guess wrong).

Retrieval-Augmented Generation (RAG)

RAG is the standard pattern for giving an AI relevant knowledge without retraining:

User Question
       ↓
[Vector Database]  →  Retrieve top-5 relevant chunks
       ↓
[Prompt Assembly]  →  "Use ONLY this context to answer:" + chunks + question
       ↓
      [AI] → Answer with citations

Key components:

1. Chunking — split documents into pieces (~500–1000 tokens each)
2. Embeddings — convert chunks to vectors (numerical representations)
3. Vector search — find chunks similar to the question
4. Prompt assembly — stuff the chunks into a prompt with instructions

Citation best practices

Always ask the model to cite its sources from the provided context:

"Answer using only the provided documents. For each claim,
cite the document name and paragraph number in brackets.
If the documents don't answer the question, say so."

Guardrails against hallucination

Add to every knowledge-based prompt:

• "Only use information from the context below."
• "If the context doesn't contain the answer, say 'I cannot find this in the provided documents.'"
• "Do not make up quotes, statistics, or sources."

Common mistakes

• Mixing relevant and irrelevant documents — the model may latch on to the wrong information.
• Chunking too small (loses context) or too large (exceeds window, dilutes relevance).
• Trusting vector search blindly — always set a similarity threshold to discard low-confidence chunks.
• Failing to instruct the model to ignore irrelevant chunks — it will try to use everything.

Quick check below!