Skip to content

feat: sync with cortext.cpp engine v5 (reference time, LI asset, text with media) - #2

Merged
gabewillen merged 12 commits into
mainfrom
feat/engine-v5-sync
Oct 20, 2026
Merged

gabewillen merged 12 commits into
mainfrom
feat/engine-v5-sync

Conversation

@gabewillen

Copy link
Copy Markdown
Contributor

Summary

Brings cortext.go up to date with augmem/cortext.cpp main (f0b681e9: engine v5 #28, per-blob content timestamps #30, reference-time retrieval #31). The branch starts from the local engine-prune work (evidence budget, standing_memory, typed and mixed content, ProceduralEnabled removal), merges origin/main (already contained, so no changes) and merges PR #1 (feat/text-media-payload).

Requires the next engine release. AssetsVersion / ModuleVersion stay at 1.2.4, and the CHANGELOG "Unreleased" section says so. With the pinned native, the v5 tests skip.

Per API item

  • Evidence budget: ProcessOptions.EvidenceBudgetBytes → evidence_budget_bytes, on every Process* call (from engine-prune).
  • Reference time (new):
    • ProcessOptions.ReferenceTime time.Time and AsOf bool map to has_reference_ms / as_of / reference_ms. The native struct is 48 bytes, and a test pins its layout.
    • The zero ReferenceTime means unset.
    • AsOf without ReferenceTime is an error, and so is a time before the epoch.
  • Standing memory: returned as ctx["standing_memory"] and documented. A new test checks that it holds the directive and not the fact.
  • Mixed modality / per-content MIME type: modality: "mixed", per-entry mimetype, and the memory mimetype taken from its first signal. Documented, with an existing test.
  • Per-content timestamps (new): documented content[].timestamp (ms, 0 = unknown). A test checks it falls within the store window.
  • Text with media (from feat: add text processing with media payloads #1):
    • Merged as is, then the docs were corrected: media replaces the stored payload, empty media stores the text, and text/* payloads count as text.
    • It takes ProcessOptions, so the budget and reference time apply.
    • The optional-symbol guard for natives without the entry point is kept.
  • LI model asset (new):
    • MaterializeModel checksums AIST-87M-LI.gguf and places it beside AIST whenever the manifest declares it. Older manifests that don't declare it are skipped.
    • It can install from an already-assembled file as well as from chunks.
    • New validates CORTEXT_LI_MODEL_PATH and forwards it through the CRT on Windows.
  • Older natives:
    • Options that a loaded native cannot read now return an error instead of being ignored.
    • Process options are offered in each known layout, newest first. Before this, released natives clipped the 32- or 48-byte struct to the include_embedding prefix and silently dropped retention.
    • New rejects SequentialEdgesEnabled / SignalFilter* on a pre-v5 native, because there they would land one field off.

Breaking

  • Config.ProceduralEnabled is removed (a change to the C cortext_config layout).
  • Soft Anchors are gone from the packet.
  • There is no longer a 3 KiB per-memory trim: memories come back whole unless a budget is set.
  • v5 engines require the LI reranker whenever AIST resolves.
  • Options or Config fields that the loaded native cannot honour now return errors.

Test plan

All runs used CGO_ENABLED=0 go test ./... -count=1 on macOS arm64 with Go 1.26.0.

  • Against the local engine (CORTEXT_LIBRARY_PATH=/Volumes/Build/cortext/bindings/shared/libcortext.dylib built from f0b681e9, CORTEXT_ASSETS_DIR=/Volumes/Build/cortext/bindings/models, CORTEXT_LI_MODEL_PATH=…/AIST-87M-LI.gguf): 26 passed, 0 skipped, 0 failed (ok … 375s).
  • Release native only (no env overrides, pinned 1.2.4 downloaded, as in CI): 16 passed, 10 skipped (v5-only), 0 failed (ok … 466s). TestNewProcessEmbed now also asserts that the 1.2.4 native keeps the retention layout and rejects shifted Config fields.
  • -short: ok.
  • The v5 native tests (reference time, timestamps, text with media, standing memory) were run 5 times (-count=5): ok.
  • Red then green:
    • With the API surface stubbed (fields present but not wired), the reference-time options test, AsOf hiding, the older-native error, LI materialization and LI env validation all failed. They pass after the change.
    • Against the previous cortext.go, the legacy-layout test showed StructSize:12 … Retention:3 (retention dropped), and the Config guard accepted shifted fields. Both pass now.
    • The packet-shape tests (timestamps, text with media, standing memory) pin engine behaviour through existing binding calls. They were green on first run because they need no binding change.

Not verified: Windows and Linux (CI only), and 32-bit (the layout test skips there).

Supersedes #1.

The engine removed the no-op procedural_enabled flag from the C ABI
cortext_config struct; drop it from Config and nativeConfig so the purego
struct layout matches. go test -short passes; a local New() smoke test
with a Config passes against a libcortext.dylib built from the engine
change.
A turn's text and photo under one source become one "mixed" memory whose
content entries each carry their own mimetype (C API memory JSON
content[].mimetype). Contexts are passed through as decoded JSON, so the
binding exposes both fields; the README documents the memory content shape.
ProcessOptions.EvidenceBudgetBytes (zero = recalled memories whole) fills
the native cortext_process_json_options struct's new evidence_budget_bytes
field. The README documents the budget (text by its UTF-8 bytes, other
payloads 480 bytes) and that query contexts lead with standing_memory, ahead
of retrieved_memory. go test passes against a libcortext.dylib built from
the engine change (CORTEXT_LIBRARY_PATH).
# Conflicts:
#	CHANGELOG.md
#	README.md
… asset

ProcessOptions gains ReferenceTime and AsOf, mirrored into the
has_reference_ms/as_of/reference_ms tail of cortext_process_json_options.
Options the loaded native cannot read now return an error.

MaterializeModel checksums and places AIST-87M-LI.gguf beside AIST when the
manifest declares it, and New validates CORTEXT_LI_MODEL_PATH.

Tests cover reference time with and without as_of, per-content timestamps,
text with media, standing memory and LI asset resolution against the engine.
The 48-byte options struct made released natives, which keep only their
exact struct size, clip it to the include_embedding prefix and drop
retention. Offer each known layout newest first instead.

Pre-v5 cortext_config still has procedural_enabled at the same size, so New
rejects the shifted Config fields there rather than misrouting them, and the
v5-only tests skip on such natives.
AssetsVersion and ModuleVersion are 2.0.0. The module path becomes
github.com/augmem/cortext.go/v2 for the v2 major. CI and the release hook
set CORTEXT_REQUIRE_ENGINE_V5=1 so the engine v5 tests fail instead of
skipping, and the LI tests fall back to the downloaded release assets.
@gabewillen
gabewillen merged commit 4fd2448 into main Oct 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant