Fix the blank page reported in #1 and #2, and add tests - #4
Conversation
Both open issues report the same symptom from two different causes. Issue #1: query() embedded the user's text before opening the stream, and get_embedding raised on any non-200. A missing model therefore produced an HTTP 500 instead of an event stream, and the front end only had console.error in EventSource.onerror -- so the page stayed blank and silent. Reproduced against the original code: HTTP 500, zero SSE events. Issue #2: two retry branches used `continue` without incrementing step_count, so a model that kept answering over 700 characters, or kept trying to finish before the minimum step count, held the loop forever without yielding anything. Reproduced against the original code: still running after 15 seconds having emitted zero events, printing the exact "content exceeded 700 characters. Retrying..." line quoted in the issue. Both are now bounded, and every path out of the reasoning loop ends in a final or an error event that the page renders visibly. Also fixed along the way: - find_similar hardcoded 4096 dimensions while build_annoy_index took it as a parameter, so changing embedding model broke search silently. The dimension is now discovered from the backend and stored per row. - Annoy's angular distance was reported as though it were cosine similarity, which could go negative. Replaced with an exact numpy scan; an approximate index only pays off around a hundred thousand vectors and a query here produces about ten. - The strongest-path search accumulated negated weights, which invalidates Dijkstra's greedy choice, so it returned the first path it reached rather than the strongest. It now minimises -log(similarity), which is non-negative and therefore exactly optimal, and reports the geometric mean. - Edge `length` was computed and then dropped during serialisation, so the similarity-proportional spring length never reached vis.js. - max_tokens and temperature were sent as top-level keys, where Ollama ignores them. They belong under options as num_predict and temperature. - The response stream stripped every apostrophe from the model's text. - The final answer was drawn as a second node holding the same text as the last step, producing a spurious 1.00 edge between them. - Model-generated markdown went into innerHTML unsanitised. - debug=True on 0.0.0.0 exposed the Werkzeug console to the network. - Enter did not submit, the button gave no busy state, and a second submit left the first EventSource running. New: /health explains what is missing and names the ollama pull command. Backends are injectable, so 56 tests run with no model and no network, including twelve that drive a real browser and read the graph back out of vis.js. vis-network and marked are vendored so those tests do not depend on a CDN. scripts/check_numbers.py verifies the README's measured claims against data/claims/ and runs in the same gate as lint and tests. Download PNG now exports the canvas the user is actually looking at rather than re-plotting the graph server-side with a different layout engine. Co-authored-by: blueprintparadise <rhiray03@gmail.com>
The store was wiped at the start of every request, which made "Related
Questions and Answers" structurally unable to show anything except the
current run's own steps. It now persists across questions, so the panel does
what its heading says. LKG_RESET_DB=1 restores the old behaviour.
That raises a fair question -- if the store grows into the hundreds or
beyond, does the search need an approximate index? Measured rather than
argued, in scripts/bench_search.py, at 768 dimensions:
vectors exact scan annoy query annoy build (per insert)
100 0.007 ms 0.031 ms 3.5 ms
1 000 0.017 ms 0.032 ms 36 ms
10 000 0.30 ms 0.031 ms 366 ms
100 000 3.3 ms 0.032 ms 4 020 ms
An exact scan over a thousand vectors costs 0.017 ms against an LLM call
that takes seconds, and an Annoy index is immutable once built -- this app
appends after every reasoning step, so the whole index would be rebuilt each
time. That last column is worse than the exact scan at every size measured.
Separately, and decisively: on a current numpy the index returns wrong
answers. With annoy 1.17.3 and numpy 2.5.2 on Python 3.12,
get_nns_by_item(7, 5) returns [1] -- one result instead of five, and not the
vector itself, which must always be its own nearest neighbour at distance
zero. Reproduced in a clean environment built from the old requirements.txt,
so the shipped "Related Questions" panel was returning a single arbitrary
row. Pinned as a falsified-prediction guard: if a future build starts
behaving, the check fails and the decision gets revisited.
Search now keeps the matrix in memory instead of decoding every blob out of
SQLite per query, and uses argpartition rather than a full sort. The
decoding, not the arithmetic, was what would have made a growing store slow.
tests/test_store.py asserts exactness directly rather than assuming it: a
vector is its own nearest neighbour, and the ranking matches a full
brute-force sort. Plus cache invalidation on write, dimension and model
isolation, and growth across successive questions.
|
Follow-up commit: the store now accumulates across questions instead of being wiped at the start of every request. That wipe made "Related Questions and Answers" structurally unable to show anything except the current run's own steps, so the panel never did what its heading says. That makes it fair to ask whether the search needs an approximate index once the store grows. Measured rather than argued (
An Annoy index is immutable once built, and this app inserts after every reasoning step, so the whole index gets rebuilt each time — that last column is worse than the exact scan at every size measured. Separately, and decisively: on a current numpy the index returns wrong answers. With One result instead of five, and not item 7 — a vector is always its own nearest neighbour at distance zero. Reproduced in a clean venv built from the Search now also keeps the matrix in memory rather than decoding every blob out of SQLite per query, and uses
|
An embedding API hands back one pooled vector from the top of the stack. layers.py taps a chosen point inside a local model instead, which also makes a model with no embedding endpoint usable, since a forward pass is all that is required: LKG_EMBED_BACKEND=hf LKG_HF_MODEL=HuggingFaceTB/SmolLM2-135M \ LKG_HF_LAYER=blocks.-1 python app.py Verified end to end: the graph is built from layer -1 inside SmolLM2-135M (576 dimensions) with llama3.2:3b doing the reasoning. Layers are addressed structurally rather than by a per-architecture path. blocks.0, blocks.12, blocks.-1, blocks.-1.mlp, or any explicit dotted path. The block stack is found by taking the longest nn.ModuleList whose children share one class, which covers Llama, Qwen, Mistral, Gemma, Phi, GPT-2, GPT-NeoX, Falcon, BERT, ViT and CLIP with no lookup table. Three things here are load-bearing and are the usual sources of a silently wrong layer comparison: - Hooks that pool inside the hook, not output_hidden_states=True. The flag materialises (n_layers+1, batch, tokens, hidden) before pooling, which is several gigabytes on an 8B model at a realistic batch and context. - Intermediate blocks emit the raw residual stream while the model's own last hidden state has already been through the final norm. That norm is applied to every layer, otherwise two layers are compared across two different spaces and the result looks plausible rather than broken. - Last-token pooling has to skip padding, on either side. Pooling the pad instead of the last real token is the classic version of this bug. make sweep reports how well each layer separates one topic from another, over the same four-topic corpus used for the embedding-model table. On SmolLM2-135M separation grows roughly seventyfold with depth, from 0.002 at blocks.0 to 0.148 at blocks.29. The first block is the control: it sees each token before any context is mixed in, so it cannot tell the topics apart, and its near-zero score is what says the deeper numbers are real rather than an artefact of the metric. All four claims are pinned in check_numbers. 24 new tests. The addressing and pooling logic runs against synthetic modules so it needs no download; the tests that need real weights are marked hf. One of them asserts that two different layers do not return the same answer, because if they did the hook would not be tapping where it claims.
pip install mpe-lkg mpe-lkg Published: https://pypi.org/project/mpe-lkg/0.2.0/ The modules move to src/mpe_lkg/ so there is a real package to install, with templates and the vendored JS inside the wheel. Root app.py stays as a shim: this has been a clone-and-run-app.py project since 2024 and the README said so for two years, so `python app.py` keeps working from a clone whether or not the package is installed. New entry points: mpe-lkg start the app mpe-lkg doctor say what is missing and the exact ollama pull command, exiting non-zero so it works in a script python -m mpe_lkg from mpe_lkg import create_app, health Pure Python, so the artefact is a single py3-none-any wheel: one file for Linux, macOS and Windows on every supported interpreter, nothing compiled at install time. On the question of writing this in Rust for the Python binding: it would not be easier here. PyO3 turns one universal wheel into a matrix of per-OS, per-architecture, per-ABI wheels needing cibuildwheel, and the codebase is Flask, numpy and networkx -- a rewrite means reimplementing an HTTP server, SSE streaming and a graph library to get identical behaviour. The WASM argument is real but it is a different project: running the embedding model in the browser needs the model itself compiled to WASM, which is not a binding question. Green on three platforms does not require it, as this commit shows. CI is three workflows, one per platform, so each carries its own badge: - Linux, the primary gate: Python 3.10 to 3.13, browser tests, the documented-numbers check, and a fresh-venv install of the built wheel. - macOS and Windows: one Python version each. The platform axis is what these are for; the version axis is already covered on Linux. All three cache pip via setup-python, and cache the Playwright browsers under a key that includes the Playwright version -- roughly 100 MB per job that would otherwise be downloaded every run. Windows is the one most likely to find a real bug here, and the comment in that workflow says why: an open SQLite handle cannot be deleted, path separators differ, and pytest's temporary directories are cleaned up differently. Verified before spending Actions minutes: the suite passes locally on Python 3.10, 3.11, 3.12 and 3.13, the wheel installs into a clean venv and serves its templates, and `pip install mpe-lkg` from PyPI works end to end. pypi.yml publishes on a v* tag, after checking that the tag matches the version in pyproject.
|
Now packaged and tested on all three platforms.
|
| Result | |
|---|---|
| Linux, Python 3.10 / 3.11 / 3.12 / 3.13 | 69 passed, 12 deselected |
| macOS | 69 passed |
| Windows | 69 passed |
Three workflows so each carries its own badge. Browser render tests run on all three — that is the point of the platform axis. pip is cached via setup-python, and the Playwright browsers are cached under a key containing the Playwright version, which is ~100 MB per job that would otherwise be re-downloaded every run.
Verified locally on 3.10, 3.11, 3.12 and 3.13 before spending any Actions minutes, and all three platforms passed on the first run.
On doing this in Rust
Asked and answered honestly, since the question was conditional ("if it is easier"): it is not easier here. Pure Python gives one py3-none-any wheel covering Linux, macOS and Windows on every supported interpreter, with nothing compiled at install time. PyO3 turns that into a per-OS × per-architecture × per-ABI matrix needing cibuildwheel. And the codebase is Flask, numpy and networkx — a rewrite means reimplementing an HTTP server, SSE streaming and a graph library to reach identical behaviour.
The WASM argument is real but it is a different project: running the embedding model in the browser needs the model itself compiled to WASM, which is not a binding question. Green on three platforms does not require Rust, as this PR shows.
Three workflows each had a job called "test", so they produced three check contexts called "test". Branch protection identifies a required check by its context name, and cannot distinguish two identically named ones -- so requiring "macOS passed" was not expressible. The checks are now Linux 3.10 .. 3.13, macOS, and Windows.
People screenshot a project's GitHub page and share it, so the file listing is part of the first impression. The root is now five directories and four files: .github/ docs/ scripts/ src/ tests/ .gitignore Makefile README.md pyproject.toml - .DS_Store was tracked. Removed from the index and from disk. - example.png moves to docs/, alongside docs/claims/ (was data/claims/). - requirements.txt and requirements-dev.txt are gone. pyproject.toml already declares the same dependencies and the [dev] extra, and two files listing the same thing is one file that will drift.
1 participant
Both open issues report the same symptom — the page stays blank — from two different causes. This fixes both, and adds the tests that would have caught them.
Issue #1: HTTP 500 instead of a stream
query()embedded the user's text before opening the SSE stream, andget_embeddingraised on any non-200 response. A missing model therefore produced an HTTP 500, and the front end only hadconsole.errorinEventSource.onerror— so the browser showed nothing at all.Reproduced against the code on
main:Issue #2: the loop never terminated
Two retry branches used
continuewithout incrementingstep_count. A model that kept answering over 700 characters, or kept trying to finish before the minimum step count, held the loop forever without yielding anything.Reproduced against the code on
main:That "content exceeded 700 characters. Retrying..." line is the one quoted in #2.
Both branches are now bounded, and every path out of the reasoning loop ends in a
finalor anerrorevent that the page renders visibly. There is also a/healthroute that names the exactollama pullcommand needed.Other bugs found while in there
find_similarbuild_annoy_indextook it as a parameter, so changing embedding model broke search silentlyfind_similarsqrt(2(1-cos)), range [0,2]) was reported as cosine similarity, and could go negativecalculate_strongest_pathcost - weight, i.e. negative weights, which invalidates Dijkstra's greedy choice — it returned the first path it reached, not the strongest, and called it a "weighted average"calculate_strongest_pathstart_node = 'Step1'; a graph without it raisedNetworkXError, which theexcept NetworkXNoPathdid not catchserialize_graph_datalengthand then dropped it, so the similarity-proportional spring length never reached vis.jsstream_api_callmax_tokensandtemperaturewere sent as top-level keys, where Ollama ignores them; they belong underoptionsasnum_predict/temperaturestream_api_call.replace("'", '')stripped every apostrophe from the model's textgenerate_responseindex.htmlinnerHTMLunsanitisedapp.pydebug=Trueon0.0.0.0exposed the Werkzeug console to the networkindex.htmlEventSourcerunningStructure and tests
app.pyis split intobackends.py(chat/embedding backends, discovery, health),reasoning.py(the loop),graph.py(similarity and paths) andstore.py(SQLite + search). The backends are injectable, so 56 tests run with no model and no network, including twelve that drive a real browser and read the graph back out of vis.js.vis-networkandmarkedare vendored locally so those tests do not depend on a CDN.annoyandscikit-learnare gone. An approximate index only starts paying for itself around a hundred thousand vectors and a query here produces about ten, so similarity search is now an exact numpy scan — which also removed the angular-distance bug and the one dependency that needs a C++ toolchain.Measured, not asserted
make measurerecords how much the drawn edge weights actually vary, over four unrelated six-step reasoning chains:all-minilmnomic-embed-textThe bigger model draws the less informative graph: under
nomic-embed-textnearly every pair of steps scores about 0.67, so the edge labels stop distinguishing anything. Ordinary distance concentration, but worth knowing before picking a default.scripts/check_numbers.pyties every measured claim in the README to a value indata/claims/and runs in the same gate as lint and tests, so a claim that stops holding fails the build rather than quietly becoming wrong. Some of its checks are ground truths computed from arithmetic rather than from a previous run.Download PNG
Includes the feature from #3, with the export moved to the client:
vis.jsalready draws to a canvas, socanvas.toBlob()gives the layout the user is actually looking at. The server-side version re-plotted the graph withspring_layout, so the downloaded image did not match the screen, and it kept the graph in a module-level global shared between concurrent users. Credited viaCo-authored-by.Verification
Checked end to end against
llama3.2:3bwith bothall-minilm(384d) andnomic-embed-text(768d).example.pngis regenerated from a real run.