Fix retrieval that returns irrelevant AI source passages

A search assistant answers a pricing question using a page about installation. The generation step may be following its instructions correctly while working with the wrong evidence.

Why this happens

Poor chunk boundaries, missing metadata, incompatible embedding versions or a weak query can damage retrieval. Returning more chunks can simply add more irrelevant text.

Step-by-step solution

1. Create a small set of questions with known relevant passages. Inspect the retrieved IDs and scores before evaluating the generated answer.

2. Verify that documents and queries use compatible embedding configuration. Rebuild the index when changing embedding models rather than mixing vectors without a migration plan.

3. Improve section boundaries and metadata filters, then compare retrieval quality on the fixed question set. Consider hybrid search for exact identifiers and product codes.

Worked example

A question about plan PRO-2026 should retrieve the current plan section, not an unrelated paragraph that merely repeats the word professional. Add that exact case to the retrieval evaluation set.

Verify the fix

Measure whether the relevant passage appears in the selected results. Then verify answer quality separately, so a generation change does not hide a retrieval regression.

Prevent a repeat

Version the embedding model, index and source snapshot together. Keep deleted or superseded documents out of active retrieval.

Reference

Official documentation or standards guidance

An original Gistmax troubleshooting guide. Scenarios and sample values are illustrative; configuration depends on your software version and environment. Cover diagram: Gistmax Tutorials.