Building a RAG from scratch

What we built#

A private web page where you type a question and an AI answers using only the content you gave it: first an FAQ we wrote together, later any PDF, Word file or web page. Under every answer it shows which passages it used and how closely each one matched.

This technique has a name: RAG, short for Retrieval-Augmented Generation.

• Retrieval: find the pieces of your content that relate to the question.
• Augmented: add those pieces to the instructions we send the AI.
• Generation: the AI writes the answer from them.

The big idea: meaning as numbers#

Computers can't compare meaning directly, so we convert text into a list of numbers called an embedding. We used OpenAI's text-embedding-3-small, which turns any text into 1,536 numbers.

The magic property: texts with similar meaning get similar lists of numbers, even when they use different words.

Measuring "closeness": cosine similarity#

Think of each embedding as an arrow pointing in a direction in a 1,536-dimension space. Two texts that mean similar things point in nearly the same direction. We measure the angle between arrows with cosine similarity:

• 1.0 = same direction (same meaning)
• 0 = unrelated
• In practice, good matches for us scored about 0.5–0.8.

The calculation is just multiply the two lists number by number and add it all up (the dot product). OpenAI's embeddings are already "unit length", so the dot product is the cosine similarity.

$score = 0.0;
foreach ($passageVector as $i => $v) {
    $score += $v * $questionVector[$i];   // 1,536 multiplications
}

The two stages#

BUILD (once, when content changes)        ASK (every question)
──────────────────────────────────         ─────────────────────────────
FAQ / PDF / web page                       visitor's question
   │ split into passages                      │ embed the question
   ▼                                          ▼
passages ──► embed each ──► save        compare with every saved passage
                         (index file)        │ keep the top 5 (score ≥ 0.20)
                                             ▼
                                   AI model answers from those 5 only
                                             │
                                             ▼
                                answer + "used these passages"

Stage 1: building the index#

1. Write the source. We drafted 24 questions and answers about the business, using only facts we could verify. Unknowns (prices, contact details) were deliberately left out and listed for later: an AI can't invent what isn't there.
2. Split into passages ("chunking"). One FAQ entry fits in one passage. For long documents we later split into ~800-character pieces (lesson 07).
3. Embed every passage with one API call (you can send many texts at once).
4. Save the passages and their numbers in a file: the index.

Stage 2: answering a question#

1. Embed the question (one small API call).
2. Score every passage against it with cosine similarity.
3. Keep the top 5 that score at least 0.20 (below that it's noise).
4. Send the AI model a message like this:

You are the assistant for "SgAiSolution FAQ".
Answer using ONLY the passages below. Keep answers short.
If the passages don't contain the answer, say the knowledge base
doesn't cover that yet. Never invent prices, dates or contact details.

Passages:
[FAQ] Q: Do I need coding experience? A: No...
...

5. Return the answer plus the passages and their scores, so the reader can see where it came from.

Where each piece runs (and why)#

Keeping it private#

• Password: the server asks for a username and password before showing the page (HTTP Basic Auth, configured in a file called .htaccess, see lesson 08).
• Not in Google: a noindex tag and header tell search engines not to list it.
• Secrets locked away: the key and index live in a folder the web server refuses to serve (we tested: it returns 403 Forbidden).

What went wrong (and how we found it)#

Key takeaways#

• RAG = find the relevant facts, then ask the model to answer from them.
• Embeddings turn meaning into numbers; cosine similarity measures how close two meanings are.
• The prompt must forbid guessing and allow "I don't know".
• Keys and data stay on the server, never in the browser.
• Show your sources: it builds trust and makes debugging easy.

Quick quiz#

Try it yourself#