zovos/vk_hackathon
1
0
Fork 2
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Filter is_system and is_hidden messages in chunking (align with Lotus reference)

Browse source 
Lotus explicitly filters both flags before building chunks. Our _clean_all
was only filtering by is_empty, so system/hidden messages with content
(e.g. member_event) were included in chunks and polluted the index.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
q 2026-04-18 17:02:11 +03:00
parent 5d50a219bf
commit fec71a98b9
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
index/chunking.py
View file

@ -11,7 +11,7 @@ OVERLAP_MESSAGES = 2
def _clean_all(messages: list[Message]) -> list[CleanedMessage]: def _clean_all(messages: list[Message]) -> list[CleanedMessage]:
cleaned = [clean_message(m) for m in messages] cleaned = [clean_message(m) for m in messages if not m.is_system and not m.is_hidden]
return [c for c in cleaned if not c.is_empty] return [c for c in cleaned if not c.is_empty]