# AI Update Log

## Scope
- File: `search/main.py`
- Purpose: fixed and extended retrieval/rerank pipeline according to TODO items.

## Done Changes
- Switched base query selection to `question.search_text` with fallback to `question.text`.
- Added support for `question.variants` as additional query variants in retrieval.
- Added support for `question.hyde` as additional dense-only queries.
- Added support for `question.keywords` as the primary source for sparse query text with fallback to current query variant.
- Stopped losing retrieval candidates after rerank: rerank is applied to head (`RERANK_LIMIT`), tail candidates are preserved.
- Added deduplication of retrieval points by Qdrant point id before rerank.
- Implemented score aggregation by `message_id` (sum of chunk scores mapped to same message).
- Limited final response to `top-50` message ids via `FINAL_TOP_K = 50`.
- Final output message ids are now selected from aggregated scores (sorted by score desc, tie-break by message_id).

## Notes
- Earlier step introduced direct `message_ids` deduplication before response.
- Current logic supersedes this by ranking and selecting unique `message_id` values from aggregated scores.

## Verification
- Syntax check passed after each main change: `python -m py_compile search/main.py`.

## How To Use This Log
- Treat this file as the source of truth for already completed `search/main.py` tasks.
- On next tasks, read this file first to avoid duplicate edits.