RAG (Retrieval-Augmented Generation) Integration

What the knowledge base is for

Exceptions turn on text: the clause in a supplier contract that allows or forbids a charge, the AP (Accounts Payable) policy that sets a threshold, the pricing schedule that fixes a unit price. A retrieval-augmented system finds the right passages, and the answer is only as good as those passages and the checks on the answer. See ADR-004.

The pipeline

1. Parse by clause

   Layout-aware parsing keeps headings, clause numbers and tables. Each chunk is one clause or table row group, prefixed with its heading path.

2. Attach metadata

   vendor_id, document type, effective and expiry dates, version, and the access groups allowed to read it.

3. Index hybrid

   Dense embeddings for meaning plus BM25 (Best Matching 25 ranking) for exact terms such as clause numbers and PO (Purchase Order) references.

4. Filter at query time

   Tenant, vendor and access group are applied as hard filters before ranking. The model never sees a passage the user cannot read.

5. Retrieve and rerank

   Top 40 candidates by hybrid score, reranked to the top 6 passages.

6. Answer with citations

   The model answers only from those passages and must cite a clause for each claim.

7. Check grounding

   A second check confirms every claim maps to a cited passage. Failures route to a human.

The corpus

Only the clauses a user is entitled to see are ever retrieved. Restricted contract pricing stays with procurement.

A question and its grounded answer

The exception agent asks whether the supplier may bill for a quantity the buyer did not receive. The excerpt below is fictional contract text, used for the demo.

Question: Can Example Supplies Ltd invoice for quantity above the goods receipt?

Retrieved (rank 1, score 0.91)
  doc:      contract/V-10442-2025-msa.pdf   version 3   effective 2025-01-01
  clause:   7.3 Receipt and invoicing
  text:     "Supplier shall invoice only for goods or services accepted by Buyer at
             goods receipt. Quantities invoiced above accepted quantities are not
             payable and shall be credited."

Answer (grounded)
  The supplier may invoice only for quantities accepted at goods receipt. Quantities
  above accepted quantities are not payable and must be credited (clause 7.3).

  citations: [contract/V-10442-2025-msa.pdf#7.3]
  grounding: 1/1 claims cited · confidence 0.94
  action:    dispute 4 units ($720.00); approve accepted quantity

Evaluation

The knowledge base is judged on evidence, not impressions. A golden set of questions with known source clauses is run on every change to the chunker, the embedding model, the rankers or the prompt.

When it says no

• No passage above the relevance floor: the system returns no answer and routes the question to a reviewer.
• Conflicting clauses from different versions: the newer effective version is cited and the conflict is noted.
• Ungrounded claims: the answer is withheld and the draft shows only the cited parts.
• Scanned contracts with poor OCR (Optical Character Recognition): the chunk is flagged low-quality and the question is routed to review.