Fix #1926: feat: improve MemOS performance#1944
Closed
Memtensor-AI wants to merge 1 commit into
Closed
Conversation
Three targeted hot-path improvements to GeneralTextMemory: - Add a per-instance bounded LRU cache (size 256) on `_embed_one_sentence` so identical queries within a process — common during retry / rerank loops, repeated chat lookups, and update() calls that reuse a recent search() text — skip the embedder round-trip. Cached values are deep- copied on the way in and out so downstream mutation cannot poison the cache. - Drop the redundant `sorted(..., reverse=True)` in `search`. Qdrant's `query_points` already returns hits in descending-by-score order; the in-process sort was a no-op. A regression test now guards the contract. - Use `TextualMemoryItem.model_validate(payload)` instead of `TextualMemoryItem(**payload)` on the read paths (`search`, `get`, `get_by_ids`, `get_all`) — equivalent semantics on a dict input and the recommended Pydantic v2 idiom. Adds 4 unit tests under tests/memories/textual/test_general.py covering cache hit, LRU eviction, search ordering preservation, and search↔update cache sharing. All 62 textual-memory tests pass; ruff format + check clean. No public API change, no schema change, no new dependency. Refs #1926
Memtensor-AI
requested review from
CarltonXiang,
MatthewZhuang,
World-controller and
syzsunshine219
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Improves the textual-memory hot path in
GeneralTextMemory(src/memos/memories/textual/general.py) with three small, isolated, backwards-compatible optimizations endorsed by the Phase-1 timeout default scope.Specifically: (1) a per-instance bounded LRU cache (size 256) on
_embed_one_sentenceso identical queries within a process — common in retry / rerank loops, repeated chat lookups, andupdate()calls that reuse a recentsearch()text — skip the embedder round-trip; cached embeddings are deep-copied on the way in and on the way out so downstream mutation cannot poison cache state. (2) Drops a redundant in-processsorted(..., reverse=True)insearch(): Qdrant'squery_pointsalready returns hits in descending score order; a regression test now guards that contract. (3) SwitchesTextualMemoryItem(**payload)toTextualMemoryItem.model_validate(payload)on the read paths (search,get,get_by_ids,get_all) — equivalent semantics on a dict input and the recommended Pydantic v2 idiom.Tests: adds 4 new unit cases in
tests/memories/textual/test_general.pycovering cache hit, LRU eviction, search-ordering preservation, and search↔update cache sharing. All 62 textual-memory tests pass. Broader smoke (tests/memories/ tests/mem_os/ tests/mem_cube/) shows 101 passed; 2test_kvfailures are pre-existing transformersDynamicCacheAPI drift reproduced on the unchanged base and unrelated to this PR. Ruff lint + format both clean. No public API change, no schema change, no new dependency, no API contract change. Branchfeature/autodev-1926pushed to origin at commitecdd118a; specs archived to memos-autodev-specs main.Related Issue (Required): Fixes #1926
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Not run; documentation-only change.
Checklist
@MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller please review this PR.
Reviewer Checklist