Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
# Changelog

## 0.11.1 - 2026-10-04

- Fixed the Hermes live-compression adapter's missing `_coerce_threshold_tokens_cap`
method and lazy context-window/threshold cache invalidation.
- Reused installed Hermes route metadata and per-model threshold handling; current
`select_context` budgets refresh the window without resetting retry/usage state.
- Preserved independent cloned-engine configuration, exact recovery, internal-call
isolation and the existing strict tokenizer acceptance gate.
- Added nine real-Hermes regression cases and corrected their Ruff formatting.
- Updated release/install references and corrected the README's stale context-engine
FAQ. Rust companion version aligned; its runtime behavior is unchanged.
- Unknown-model tokenizers and the host's plugin-discovery race remain separate
limitations; publication does not install or restart a running Hermes profile.

## 0.11.0 - 2026-09-29

**Context, learning & observability.** Includes the unshipped 0.10.0 milestone;
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "token-terminator"
version = "0.11.0"
version = "0.11.1"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the bundled Rust documentation for 0.11.1

Publishing this crate as 0.11.1 still uses rust/README.md as its package README, but that file identifies the crate as v0.11.0 and tells users to run cargo add token-terminator@0.11.0; docs/RUST_CRATE.md likewise contains only 0.11.0 commands and links. Consequently, the crates.io/docs guidance for the new release directs users back to the previous version instead of installing 0.11.1, so these bundled docs should be version-aligned with this bump.

Useful? React with 👍 / 👎.

edition = "2021"
rust-version = "1.78"
description = "Rust interoperability helpers for Token Terminator artifact identities and reduction invariants"
Expand Down
24 changes: 24 additions & 0 deletions MIGRATION.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,27 @@
# Migration to Token Terminator 0.11.1

## 0.11.0 to 0.11.1

This is the PR #20 Hermes live-compression compatibility repair. No new key,
database migration, tokenizer override or engine-selection change is required.
Upgrade the environment actually used by the managed Hermes launcher, not an
unrelated Python environment or stale venv copy:

```bash
python -m pip install --upgrade 'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.1'
python -m rtk_hermes_plus.cli install-context-engine
```

Confirm the imported `rtk_hermes_plus` path in that environment. When the relevant
backend is idle, restart it once through its normal supervisor; do not start a
competing gateway or delete histories/vaults. Verify the loaded version and
current-model request metrics. A healthy restart or a lower UI percentage alone
does not prove reduction. Unknown target tokenizers still bypass the engine.
See [the compatibility guide](docs/LIVE_COMPRESSION_COMPAT.md). These instructions
do not imply publication performs a live installation or restart.

---

# Migration to Token Terminator 0.11.0

## Optional learned policy
Expand Down
35 changes: 24 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@

<p align="center">
<a href="https://github.com/AronAxe/Token-Terminator/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/AronAxe/Token-Terminator/actions/workflows/ci.yml/badge.svg"></a>
<a href="https://github.com/AronAxe/Token-Terminator/releases/tag/v0.11.0"><img alt="Release v0.11.0" src="https://img.shields.io/badge/release-v0.11.0-ef2b25"></a>
<a href="https://github.com/AronAxe/Token-Terminator/releases/tag/v0.11.1"><img alt="Release v0.11.1" src="https://img.shields.io/badge/release-v0.11.1-ef2b25"></a>
<a href="https://github.com/AronAxe/Token-Terminator/wiki/Jev-Semantic-Context-Gate" title="Optional JEV integration via OpenRouter or direct TypeSafe; off by default"><img alt="JEV-powered" src="https://img.shields.io/badge/JEV-powered-8b5cf6"></a>
<a href="https://github.com/AronAxe/Token-Terminator/wiki"><img alt="GitHub Wiki" src="https://img.shields.io/badge/docs-GitHub%20Wiki-181717?logo=github"></a>
<a href="https://crates.io/crates/token-terminator"><img alt="crates.io" src="https://img.shields.io/badge/crates.io-v0.11.0-orange?logo=rust"></a>
<a href="https://crates.io/crates/token-terminator"><img alt="crates.io" src="https://img.shields.io/badge/crates.io-v0.11.1-orange?logo=rust"></a>
<a href="https://docs.rs/token-terminator"><img alt="docs.rs" src="https://img.shields.io/docsrs/token-terminator?logo=docs.rs"></a>
<img alt="Python 3.10–3.13" src="https://img.shields.io/badge/Python-3.10%E2%80%933.13-3776AB?logo=python&logoColor=white">
<a href="LICENSE"><img alt="MIT License" src="https://img.shields.io/badge/License-MIT-22c55e.svg"></a>
Expand All @@ -27,6 +27,19 @@ Token Terminator is an agent-runtime optimization layer. It removes token bloat

Documentation: see the [GitHub Wiki](https://github.com/AronAxe/Token-Terminator/wiki) for quick start, architecture, configuration, recovery, security, troubleshooting, migration, and release notes.

## Patch v0.11.1: Hermes live-compression compatibility

Repairs the missing `_coerce_threshold_tokens_cap` adapter method and live
window/threshold invalidation. The selected TT engine now follows Hermes'
current route budget and isolates cloned engine settings. No new settings,
generative summarizer or relaxed tokenizer gate are introduced.

For managed/multiplex installs, upgrade the environment that actually imports
TT, then restart the appropriate idle backend through its normal supervisor.
This patch does not add unknown-model tokenizer mappings or fix Hermes' separate
concurrent plugin-discovery race. [Patch notes](docs/releases/v0.11.1.md) ·
[Compatibility details](docs/LIVE_COMPRESSION_COMPAT.md).

## New in v0.11.0

**Context, learning & observability.** This release includes the unshipped 0.10.0
Expand Down Expand Up @@ -146,12 +159,12 @@ See [JEV roles and implemented scope](docs/JEV_ROLES.md) and

## Select Token Terminator as the Hermes ContextEngine

Install **v0.11.0** in the Python environment that runs Hermes, then install its
Install **v0.11.1** in the Python environment that runs Hermes, then install its
managed adapter. Replace `python` below with that environment's interpreter:

```bash
python -m pip install --upgrade \
'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.0'
'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.1'
python -m rtk_hermes_plus.cli install-context-engine
```

Expand Down Expand Up @@ -309,9 +322,9 @@ The optional working-state block defaults to zero characters, even in `balanced`

## Install: Hermes Agent (turnkey)

Token Terminator 0.11.0 builds on v0.9.0 and replaces the earlier `rtk-hermes-plus` distribution. `token-terminator` and `rtk-hermes-plus` must not coexist because both own the `rtk_hermes_plus` Python import package.
Token Terminator 0.11.1 builds on v0.9.0 and replaces the earlier `rtk-hermes-plus` distribution. `token-terminator` and `rtk-hermes-plus` must not coexist because both own the `rtk_hermes_plus` Python import package.

This is the supported zero-glue installation: the repository already contains the Hermes hooks, slash command, recovery tool, and lifecycle accounting. The commands below pin the `v0.11.0` release tag.
This is the supported zero-glue installation: the repository already contains the Hermes hooks, slash command, recovery tool, and lifecycle accounting. The commands below pin the `v0.11.1` release tag.

### 1. Install RTK when using terminal rewriting

Expand All @@ -333,7 +346,7 @@ HERMES_PY="$HOME/.hermes/hermes-agent/venv/bin/python"
hermes plugins disable rtk-plus
"$HERMES_PY" -m pip uninstall -y rtk-hermes-plus token-terminator
"$HERMES_PY" -m pip install \
'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.0'
'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.1'
```

Windows example:
Expand All @@ -342,7 +355,7 @@ Windows example:
$HermesPy = "$env:LOCALAPPDATA\hermes\hermes-agent\venv\Scripts\python.exe"
hermes plugins disable rtk-plus
& $HermesPy -m pip uninstall -y rtk-hermes-plus token-terminator
& $HermesPy -m pip install "git+https://github.com/AronAxe/Token-Terminator.git@v0.11.0"
& $HermesPy -m pip install "git+https://github.com/AronAxe/Token-Terminator.git@v0.11.1"
```

`tiktoken` now ships with Token Terminator and is used automatically for supported OpenAI-family models, including common provider-qualified model IDs. Hugging Face `tokenizers` remains optional when pointing Token Terminator at a local `tokenizer.json`:
Expand Down Expand Up @@ -374,7 +387,7 @@ Installation and enablement are separate operations. Disabling affects subsequen
Rust agent hosts can use the supported `token-terminator` companion crate for the stable cross-language pieces of the Token Terminator contract:

```bash
cargo add token-terminator@0.11.0
cargo add token-terminator@0.11.1
```

```rust
Expand All @@ -396,7 +409,7 @@ Install the same distribution in the environment that owns your agent loop:

```bash
python -m pip install \
'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.0'
'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.1'
```

Then connect your runtime's tool-result and final-request hooks to `Runtime`. The adapter must map equivalent tools to Token Terminator's canonical names (`search_files`, `process`, and optionally `read_file`) and expose `Runtime.tool` to the model for exact recovery.
Expand Down Expand Up @@ -649,7 +662,7 @@ No. Provider-visible content may be compacted, and `artifact_peek` is deliberate

### Does it replace the host's memory or context engine?

No. Token Terminator operates after or alongside normal context assembly. It does not own the transcript, alter persisted conversation history, or require a particular memory engine.
It can replace the selected **Hermes context engine**, including LCM, when explicitly configured with `context.engine: token-terminator`. Middleware-only mode keeps the host engine upstream. Neither mode replaces the memory system or transcript store, or destructively rewrites persisted conversation history.

### Does enabling Jev turn off the normal Token Terminator pipeline?

Expand Down
36 changes: 36 additions & 0 deletions docs/LIVE_COMPRESSION_COMPAT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Live-compression adapter repair

The v0.11.0 adapter omitted the compatibility surface used by current Hermes
`_apply_live_compression_config`. That host function updates the selected
`context_compressor` even when it is a plugin ContextEngine, including calling
`_coerce_threshold_tokens_cap` and clearing lazy window/threshold caches.

This patch supplies positive-int/None cap normalization, lazy threshold and window
invalidation, current-route metadata resolution when an explicit pin is removed,
per-model threshold handling and isolated clone state. The current host request's
`budget_tokens` refreshes the engine's window without resetting response counters.
No hardcoded 272k or assumed replacement window is introduced. Host-side metadata
is reused; the underlying request tokenizer and reduction acceptance gate are not
relaxed. Internal service calls remain excluded. No generative compressor is built.

Nine offline regression tests passed against the installed Hermes checkout
`188a1a5efd098f067638032fea6e24c42d06178f` on Python 3.14. The tests call the real
live-config function with mocked model metadata and disposable stores. They do not
claim paid-provider validation or a reduction of an existing live session.

Release scope: v0.11.1 publishes PR #20's adapter and tests. Earlier incident
deployment notes are historical; a separately deployed adapter snapshot is not
proof that this exact release has loaded in a running process. Installation,
engine discovery and actual current-model request reduction remain separate
checks. This release performs no live installation or backend restart.

The observed host directory loader can expose a partially initialized cached module
to simultaneous discovery. This adapter patch does not modify that upstream loader
or claim to resolve its race. Deployment must use the host's managed plugin update
path and a controlled backend restart, then check loaded engine identity and actual
provider-request metrics. Do not kill unrelated relay/messaging/dashboard processes,
delete conversation/vault data, or claim a reset UI percentage proves reduction.

Unknown target tokenizers still cause ContextEngine/IR to pass through. This
patch does not supply new tokenizer mappings, guess a replacement model window,
or guarantee a reduction in a live session.
57 changes: 57 additions & 0 deletions docs/releases/v0.11.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Token Terminator v0.11.1 — Hermes live-compression compatibility

A focused patch release of PR #20. The v0.11.0 ContextEngine, learned JEV policy,
Context IR, exact-evidence vault and Observatory remain in place.

## Fixed

- Supply `_coerce_threshold_tokens_cap` for Hermes' live-compression configuration.
- Honor invalidated context-window and threshold caches, explicit pin removal,
current-route metadata and per-model threshold handling.
- Refresh the engine window from the host's current `select_context` budget
without resetting retry bindings or usage counters.
- Keep cloned engines' settings independent. No generative compressor, destructive
history truncation, new key or weakened tokenizer gate is introduced.
- Include nine real-Hermes regression cases and the formatting-only CI correction.

## Upgrade

Use the Python environment actually loaded by your managed Hermes launcher:

```bash
python -m pip install --upgrade 'git+https://github.com/AronAxe/Token-Terminator.git@v0.11.1'
python -m rtk_hermes_plus.cli install-context-engine
```

Keep your existing plugin/engine selection and JEV settings. Once idle, restart
only the relevant backend through its normal supervisor. Do not delete the
conversation history or evidence vault. Verify the imported package path,
loaded version and current-model request metrics, not only a UI percentage.

## Validation and scope

The formatted PR candidate passed Python 3.10–3.13, Windows, packaging, Rust,
learned-policy, ContextEngine, dashboard and Context IR workflows before release
preparation. The learning artifact reported 611 passed and three skips; one
skip is the whole optional real-Hermes test module. Its nine cases were previously
run locally against installed Hermes `188a1a5` on Python 3.14, not counted as
executed in that learning job. Final release-head CI status is recorded on PR #20.
No paid inference or new performance/answer-quality claims accompany this patch.

**Not fixed by this patch:** unknown-model tokenizer support and the separate
host plugin-discovery race. Without an exact target tokenizer, ContextEngine/IR
still pass through. This is not a guarantee that an affected live session has
shrunk, and it does not merge the separate deployed-adapter branch.

README changes are limited to a compatibility note, current install/version
references and correcting the stale FAQ: explicitly selected TT can replace LCM
as Hermes' ContextEngine. Artwork and schematics are unchanged.

The Rust interoperability companion is version-aligned to 0.11.1 without a Rust
behavior change; it is not the Python runtime. GitHub assets include the wheel,
source distribution and SHA256SUMS. Existing releases remain unchanged.
Publication does not install, configure or restart your running agents.

[PR #20](https://github.com/AronAxe/Token-Terminator/pull/20) ·
[Compatibility guide](https://github.com/AronAxe/Token-Terminator/blob/v0.11.1/docs/LIVE_COMPRESSION_COMPAT.md) ·
[Migration](https://github.com/AronAxe/Token-Terminator/blob/v0.11.1/MIGRATION.md)
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "token-terminator"
version = "0.11.0"
version = "0.11.1"
description = "Fail-open token reduction, exact recovery, and request compilation for Hermes Agent"
readme = "README.md"
requires-python = ">=3.10"
Expand Down
2 changes: 1 addition & 1 deletion src/rtk_hermes_plus/_version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.11.0"
__version__ = "0.11.1"
Loading
Loading