Artificial Intelligence
RAG assistant cites documents that do not contain the answer
troublesync_editorial
2w ago
Environment
Environment
Support assistant using OpenAI embeddings, PostgreSQL vector search and five retrieved chunks.
What happens
What happens
Answers sound plausible but references point to unrelated policy sections.
What I have tried
What I have tried
Increased the number of retrieved chunks and strengthened the system prompt.
I would appreciate advice on how to identify the root cause without risking existing data or configuration.
COMMUNITY RESPONSES
Verified community troubleshooting
0 replies
No answers yet..
Showing 0 of 0 community responses
AI DIAGNOSTICS
Incident analysis
Most likely causes
Retrieval ranking optimizes semantic similarity without enough metadata filtering or reranking. 86%
A recent software, firmware or configuration change is exposing an existing compatibility issue. 38%
Summary
The reported behaviour is consistent with retrieval ranking optimizes semantic similarity without enough metadata filtering or reranking. The safest next step is to verify that hypothesis with a reversible test before making permanent configuration changes.
Suggested Fixes
Verify the likely cause
Measure retrieval separately, filter by product and document version, add a reranker and require the model to abstain when evidence is weak.
Validate and monitor
Reproduce the issue once after the change, review relevant logs or metrics, and keep a rollback path until the system has remained stable.
Continue troubleshooting
Related troubles
Matched from the title, description and tags
- Artificial Intelligence26% match
RAG assistant returns high-similarity but irrelevant passages from vector DB on production queries
Open0 answers - Artificial Intelligence24% match
Incremental vector index job leaves stale vectors and duplicate document IDs — RAG answers cite deleted passages
Open0 answers - Artificial Intelligence23% match
Vector search returns semantically unrelated high-similarity passages after incremental index update
Open0 answers - Artificial Intelligence22% match
Dense vector index returns stale document IDs and wrong passages after incremental update job — RAG answers become semantically irrelevant
Open0 answers