Build a RAG Engine from Scratch in C#

01 / Understand the problem

Build a small document question-answering pipeline. Given approved text files and a question, find relevant excerpts and construct a source-grounded prompt. The deterministic indexing/retrieval parts can be tested without calling a model.

02 / Think before coding

Write the contract for each boundary: document loader, chunker, embedder, vector store, retriever, and generator. Do not allow retrieved text to decide which documents a user may access.

03 / Design

Indexing: document → validated text → chunks + source IDs → embeddings → authorized vector index.

Query: authorized question → query embedding → top-k eligible chunks → bounded context + source IDs → answer + checked citations.

04 / Implement a deterministic chunker

public static IEnumerable<string> Chunk(
    string text, int size = 800, int overlap = 100)
{
    if (size <= 0 || overlap < 0 || overlap >= size)
        throw new ArgumentOutOfRangeException(nameof(overlap));
    for (int start = 0; start < text.Length; start += size - overlap)
    {
        int length = Math.Min(size, text.Length - start);
        yield return text.Substring(start, length);
        if (start + length == text.Length) yield break;
    }
}

public static double Cosine(double[] a, double[] b)
{
    if (a.Length == 0 || a.Length != b.Length)
        throw new ArgumentException("Equal non-empty dimensions required.");
    double dot = 0, aa = 0, bb = 0;
    for (int i = 0; i < a.Length; i++)
    {
        dot += a[i] * b[i];
        aa += a[i] * a[i];
        bb += b[i] * b[i];
    }
    if (aa == 0 || bb == 0) return 0;
    return dot / (Math.Sqrt(aa) * Math.Sqrt(bb));
}

Build the prompt boundary

Place source text in clearly delimited data blocks. Ask for answers supported by those blocks and explicit abstention when evidence is missing. Keep credentials and system instructions out of the corpus. Never treat a document instruction as a privileged command.

05 / Test

• Empty documents produce no chunks.
• Overlap is smaller than chunk size.
• Wrong embedding dimensions are rejected.
• A user cannot retrieve another tenant's document.
• An unanswerable question is allowed to have no answer.
• Every returned citation refers to an actual retrieved source.

06 / Improve

Measure retrieval recall on a small labeled question set. Add deduplication, reranking, ingestion limits, and model-version tracking before increasing sophistication. Evaluate answer faithfulness separately from retrieval.

07 / Compare with AI

Review an AI-generated pipeline for missing authorization, unchecked source instructions, stale vectors, and fabricated citations. A convincing demo does not establish a secure system.