Open
Conversation
Smart memory retrieval system with cross-lingual search capabilities. New endpoint: POST /memory/retrieve - Frequency-based keyword extraction from user text - Translation cascade: Wiktionary dictionary (91k bidirectional keys) → ru↔en transliteration → Google Translate auto-supplement - Cross-lingual clustering with 65% prefix stemming - IDF-weighted voting search (rare terms = heavier votes) - Hebbian co-occurrence tracking (COOCCURS edges between Concept nodes) - Write-behind buffer: RAM accumulator → LadybugDB flush every 15s - 2-level cache: expand cache + stem query cache (TTL 60s) Additional endpoints: - POST /memory/decay — apply decay to Hebbian weights, prune weak - POST /memory/flush — force write-behind flush - GET /memory/stats — tracker statistics Files: - memory.js: retrieval pipeline, Hebbian tracker, voting search - dict_ru_en.json: Wiktionary FreeDict ru↔en translation dictionary - index.js: mount /memory router (1 line added)
Bekka review fixes for PR LadybugDB#5: 1. Remove Google Translate fallback entirely — eliminates network latency spikes on dict misses. Replace with three-layer local dict lookup: exact O(1) → Snowball stem O(1) → trigram fuzzy ~2ms. 2. Replace naive 65% prefix stemming with Snowball stemmers (Russian + English). Proper morphological normalization instead of crude truncation that broke short words (шел→ше, шла→шл). 3. Cap IDF weight at 0.5 (min denominator 2) to prevent single-hit typo/rare-term dominance in voting search results. Trigram index (Dice coefficient) handles typos in both Cyrillic and Latin: кортошка→картошка, яблоука→яблоко, abliteraton→abliteration. Performance: big prompt 17sec → 1.2sec (cold), 100ms → 5ms (cached). Dependencies: +snowball-stemmers (pure JS, 24 languages, 0 deps).
Two bugs caused infinite retry loop and 7.6GB memory leak over 10h: 1. Undirected DELETE pattern -[r:COOCCURS]- not supported by LadybugDB. Changed to directed -[r:COOCCURS]->. 2. Batch try/catch re-added ALL dirty items on any single failure. Now each item has its own try/catch — failed items retry only if they still exist in RAM (prevents ghost retry of evicted/pruned items).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hebbian Memory Retrieval Module for LadybugDB api-server
Adds associative memory retrieval with Hebbian learning, cross-lingual search, and self-learning dictionary.
Features
Endpoints
POST /memory/retrieve— keyword extraction + cross-lingual expansion + voting search + Hebbian boostPOST /memory/decay— decay weights, prune weak concepts/edgesPOST /memory/flush— force write-behind flush to DBGET /memory/stats— tracker statisticsSchema (additive, zero risk to existing data)
Conceptnode table (id, weight, access_count, created, last_accessed)COOCCURSrel table (Concept→Concept, weight, count)Performance (after Bekka review fixes)
Dependencies
snowball-stemmers— pure JS Snowball stemmer (Russian + English), 0 depsdict_ru_en.json— 91k bidirectional Wiktionary translation pairs (4.3MB)Review fixes (628f076)