1
Diagnostic step 1
Run an A/B retrieval test: compare exact cosine similarity (brute-force) vs ANN results for a representative query set to quantify recall loss; do this on a copy of the DB (non-production) to avoid impacting latency.
FULL AI ANALYSIS
RAG assistant returns high-similarity but irrelevant passages from vector DB on production queries
Generated Oct 12, 2026, 10:57:58 PM
Analysis summary
The discrepancy between approximate index results and exact search indicates the vector index configuration (IVF nlist + nprobe) or stale/incorrect index may be returning suboptimal approximate nearest neighbors. Contributing factors: low-variance query embeddings (embedding model mismatch or normalization issues), incomplete chunk metadata causing filtering to exclude good results, and retrieval pipeline ordering (approximate search then metadata filter) causing relevant items to be dropped. Each of these is independently reversible and should be tested systematically.
Possible causes
Aggressive approximate index params (low nprobe) causing false negatives in ANN search — relevant vectors not retrieved in top-K.
90%Chunking pipeline dropped or mis-populated metadata, so post-retrieval filters exclude otherwise relevant chunks.
80%Embedding drift or normalization problem producing low-variance query embeddings that don't separate queries well from corpus.
70%Prompt engineering forces reliance on metadata labels that are missing or inconsistent, causing LLM to ignore retrieved content even when relevant.
65%Suggested fixes
2
Diagnostic step 2
Temporarily increase nprobe or re-create the ANN index with a larger nlist on a staging index to measure relevance improvements; monitor query latency and resource use before promoting to production.
3
Diagnostic step 3
Validate embedding pipeline: compute per-dimension and norm statistics for a sample of query and document embeddings to detect model drift or missing normalization (e.g., ensure consistent use of L2 normalization if expected).
4
Diagnostic step 4
Audit chunking metadata for a random sample; fix the pipeline to populate source/document IDs and timestamps consistently. Avoid filtering by metadata in the first retrieval pass — apply metadata filters before or combine filtering with re-ranking to prevent dropping good candidates.
5
Diagnostic step 5
Improve prompting to include explicit re-ranking step: send top-N candidates to the LLM with citation instructions and have a deterministic re-ranker (e.g., lexical overlap + embedding score) as a guardrail. Roll changes to a canary portion of traffic and monitor relevance metrics; escalate to ML engineers if embedding model retraining or index redesign is required.