Skip to content

Fix malformed embedding recovery - #8

Merged
AliZakaee merged 1 commit into
mainfrom
automation/maintenance-20260815-noon-skip-invalid-embeddings
Aug 15, 2026
Merged

AliZakaee merged 1 commit into
mainfrom
automation/maintenance-20260815-noon-skip-invalid-embeddings

Conversation

@AliZakaee

Copy link
Copy Markdown
Owner

Summary

  • validate each stored knowledge-base embedding before adding it to the NumPy matrix
  • skip non-numeric, nested, non-finite, and float32-overflowing vectors
  • strengthen regression coverage for mixed and entirely unusable stored vectors

Why this is meaningful

The loader already promises to ignore malformed embedding rows, but a correctly sized JSON list containing a non-numeric value raised during the final NumPy conversion. That exception escaped the retrieval layer and could abort both the incoming customer-response pipeline and the owner's knowledge-base search. Validating per row preserves healthy vectors and lets an all-invalid set use the existing keyword fallback.

Validation

  • python -m pytest tests/unit/services/test_embedding_service.py -q — 13 passed
  • python -m pytest — 283 passed
  • python -m pytest --cov=. --cov-report=term-missing --cov-report=xml --cov-report=html --junitxml=junit.xml — 283 passed, 61% total coverage
  • python -m compileall -q config.py database.py handlers services security.py telegram_api.py utils.py — passed
  • git diff --check — passed

Risk and rollback

Risk is low: valid finite embeddings retain the same float32 representation, while only unusable rows are filtered earlier. Revert commit 5ac50d2 to roll back.

Prepared with automated assistance by EQ for AliZakaee.

@AliZakaee
AliZakaee merged commit c24604b into main Aug 15, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant