How RAG Works: A Developer's Guide with C# Examples

A language model needs context

Retrieval-augmented generation combines a search system with a generation system. It retrieves relevant passages from an authorized corpus, then supplies them alongside a question. It does not guarantee correctness, and it does not train the model on those passages.

Load documents with a contract

Validate file types, size, encoding, and source ownership. Extract readable text and retain a stable source identifier. Record the document version so you can delete stale chunks when it changes.

Chunk deliberately

Very small chunks lose context. Very large chunks waste the context budget and can dilute retrieval. Start with paragraph-aware splitting and a measured overlap. Evaluate with questions representative of your users rather than choosing a size because a tutorial used it.

Embed consistently

Use one embedding-model version for both indexed text and questions. Store the model identifier with the vectors. Cosine similarity compares directions; it is not a probability of factual agreement.

public record Passage(string SourceId, string Text, double[] Vector);
public record Match(Passage Passage, double Score);

// Cosine is the tested helper from the embeddings lesson.
var matches = authorizedPassages
    .Select(p => new Match(p, Cosine(queryVector, p.Vector)))
    .OrderByDescending(m => m.Score)
    .Take(4)
    .ToArray();

This exact in-memory scan is useful for a small teaching corpus. For a larger corpus, use an index appropriate to the scale and measure the recall/latency tradeoff.

Generate from bounded evidence

Build a context block with explicit source IDs. Set a maximum context budget. Ask for citations and allow the answer to say that evidence is insufficient. Treat source text as untrusted data and defend against prompt injection.

Authorization belongs before retrieval

Do not retrieve private text into a shared context and ask a model to keep it secret. Apply tenant and document permissions in the retrieval query. Verify every citation against the retrieved set before displaying it.

Evaluate the failure modes

Test questions with known sources, ambiguous questions, missing answers, conflicting documents, and unauthorized sources. Separate retrieval quality from answer faithfulness. A single fluent demonstration proves very little.

Continue with the RAG Engineering Lab to implement deterministic chunking and vector comparison before connecting an external model.