Fix #1926: feat: improve MemOS performance#1942
Closed
Memtensor-AI wants to merge 1 commit into
Closed
Conversation
Three targeted hot-path improvements to GeneralTextMemory: - Add a per-instance bounded LRU cache (size 256) on `_embed_one_sentence` so identical queries within a process — common during retry / rerank loops, repeated chat lookups, and update() calls that reuse a recent search() text — skip the embedder round-trip. Cached values are deep- copied on the way in and out so downstream mutation cannot poison the cache. - Drop the redundant `sorted(..., reverse=True)` in `search`. Qdrant's `query_points` already returns hits in descending-by-score order; the in-process sort was a no-op. A regression test now guards the contract. - Use `TextualMemoryItem.model_validate(payload)` instead of `TextualMemoryItem(**payload)` on the read paths (`search`, `get`, `get_by_ids`, `get_all`) — equivalent semantics on a dict input and the recommended Pydantic v2 idiom. Adds 4 unit tests under tests/memories/textual/test_general.py covering cache hit, LRU eviction, search ordering preservation, and search↔update cache sharing. All 62 textual-memory tests pass; ruff format + check clean. No public API change, no schema change, no new dependency. Refs #1926
Memtensor-AI
requested review from
CarltonXiang,
MatthewZhuang,
World-controller and
syzsunshine219
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Issue #1926 asks for "better overall MemOS performance" without a specific scenario. A Phase-1 clarification round timed out, so per opsp rules the locked-in default scope applies: targeted, low-risk hot-path wins inside
src/memos/memories/textual/general.pyonly — no public API change, no schema change, no new dependency.This PR makes three concrete improvements to
GeneralTextMemory. First,_embed_one_sentencenow consults a per-instance bounded LRU cache (size cap 256) keyed on raw sentence text, so identical queries within a process — common in retry / rerank loops, repeated chat lookups, andupdate()calls whose memory text matches a recentsearch()— skip the embedder round-trip; cached values are deep-copied on the way in and out so downstream mutation cannot poison the cache. Second, the redundantsorted(..., reverse=True)insearchis removed becauseqdrant_client.query_pointsalready returns hits in descending-by-score order; a regression test guards the ordering contract. Third, the read paths (search,get,get_by_ids,get_all) constructTextualMemoryItemviamodel_validate(payload)instead of**payload— equivalent semantics on a dict input and the idiomatic Pydantic v2 path.Verification: 4 new unit tests in
tests/memories/textual/test_general.pycover cache hit, LRU eviction, search ordering preservation, and search↔update cache sharing. Full textual suite passes 62/62. Broader smoke (tests/memories/,tests/mem_os/,tests/mem_cube/) passes 101 tests; 2 pre-existing failures intests/memories/activation/test_kv.pyare an unrelated transformersDynamicCache.key_cacheAPI drift, confirmed reproducible on the unmodified base branch viagit stash. Ruff format and ruff check are clean on the touched files.Related Issue (Required): Fixes #1926
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Not run; documentation-only change.
Checklist
@MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller please review this PR.
Reviewer Checklist