Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.d/topic-lineage-dense-status-admission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
## Fixed

- Offline topic-lineage CLI now bounds simultaneously retained dense topic,
document-coordinate and prevalence-coefficient fitting payload before
allocation, in addition to sparse-work and dense-design limits.
- Reference lineage/objective admission now uses only observed transition
edges. Inferred proposals remain provenance and cannot satisfy an
observed-transition requirement or become emitted lineage.
- Added dimension-only resource boundaries and CLI/native-reference
observed/inferred positive and negative regressions. These repairs preserve
PRD v0.4 and ADR 0012's posterior-aware target; they are not scientific,
GPU, protected-main, or production-release acceptance claims.
29 changes: 29 additions & 0 deletions CHANGELOG.d/topic-lineage-offline-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
### Added

- Add the standalone offline `tepp-topic-lineage` binary to `analysis_engine`.
It reads one bounded `tepp.topic_lineage_cli_input.v1` JSON file and prints
one `tepp.topic_lineage_cli_output.v1` JSON line that holds the digest-bound
`tepp.trsl_topic_lineage.v1` artifact, its request-bound terminal result,
the SHA-256 digest of the exact input bytes, and exclusion counts.
- Documents, memberships, and relations whose `available_time` is later than
the request `knowledge_cutoff` are excluded and counted before estimation.
Memberships and relations that touch an excluded document are also
excluded. Relation kinds `causes` and `intervenes_on` are refused because
the association estimator does not identify causation.
- Estimation is the unchanged ADR 0012 CPU `f64` reference path
(`execute_topic_lineage_run`) with a caller-declared topic count. The binary
does not select `K`, use a GPU, persist a run, or access the network.
- The accepted receipt is minted locally from the input `run_id`; it is not a
durable, service-registered analysis run.

### Fixed

- Reject nonregular input paths before reading with redacted `io`/exit `5`.
Unix nonblocking opens and descriptor type revalidation also prevent a
pathname-to-FIFO replacement from waiting for a writer. Regular-file
symlinks remain supported; bounded regular-file reads are unchanged.
- Correct the reference-model and lineage-edge documentation: proportions
are MAP-fitted point estimates, and association strength is their minimum
for a shared dominant topic, not a posterior mean or uncertainty probability.
Numeric estimates, field names, schema versions, and posterior-aware targets
are unchanged.
16 changes: 16 additions & 0 deletions CHANGELOG.d/topic-shared-dense-admission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Shared reference topic dense-allocation admission

- Enforce checked 16 MiB dense design admission before feature expansion and
design allocation in `ReferenceTopicInput`, including actual sparse
covariate high indices and active membership contexts.
- Enforce the existing conservative 128 MiB simultaneous numeric payload
envelope at shared fit entry before any seed, returning typed
`ResourceLimitExceeded` on overbudget or overflow. Reuse the shared planners
from CLI admission without changing its stricter operator count/work limits.
- Exercise dimension-only boundaries/overflow, direct sparse inputs/fits,
executor refusal/no artifact, and actual candidate-K all-refused/mixed
selection, retaining small positives and observed-only graph semantics.
- Supersede CLI-only resource authority in ADR 0012 and API/source-trace docs.
Payload caps are not RSS/OOM guarantees; optional joint precision remains
independently capped. No estimand, latent meaning, PRD target, or wire version
changes; this remains active-PR offline CPU evidence, not release acceptance.
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin
| Span-grounded mention numeric provenance | [`docs/doctoring/span-grounded-mentions.md`](docs/doctoring/span-grounded-mentions.md) |
| Event-intelligence status-gate doctoring | [`docs/research/event-intelligence-status-gates.md`](docs/research/event-intelligence-status-gates.md) |
| Stopword-deletion doctoring | [`docs/research/stopword-deletion.md`](docs/research/stopword-deletion.md) |
| Offline topic-lineage allocation/status doctoring | [`docs/research/topic-lineage-cli-admission.md`](docs/research/topic-lineage-cli-admission.md) |
| Topic log-ratio coordinate doctoring | [`docs/research/topic-logratio-coordinates.md`](docs/research/topic-logratio-coordinates.md) |
| Hourly NIM OpenCode doctoring | [`docs/doctoring/hourly-nim-opencode-development.md`](docs/doctoring/hourly-nim-opencode-development.md) |
| Analysis engine v1 doctoring | [`docs/doctoring/analysis-engine-v1.md`](docs/doctoring/analysis-engine-v1.md) |
Expand Down
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,27 @@ an unstable compiler capability. A zero denominator is reported explicitly for
any crate whose lane still ships no executable behavior; it must never conceal
uncovered production behavior.

## Offline topic-lineage CLI

```bash
cargo run -p analysis_engine --bin tepp-topic-lineage -- examples/topic_lineage_cli_input_v1.json
```

The binary reads one bounded `tepp.topic_lineage_cli_input.v1` file, excludes
documents, memberships, and relations that were not available at the request
knowledge cutoff, refuses `causes`/`intervenes_on` relations, runs the CPU
`f64` reference topic-lineage estimator with the supplied topic count and
seeds, and prints one `tepp.topic_lineage_cli_output.v1` JSON line. The output
is fitted topic association, not causation. Its association strength is a
MAP-fitted proportion, not a posterior mean or uncertainty probability. It is
not a durable analysis run, calibrated recovery evidence, or GPU parity evidence.
Input must resolve to a regular file (regular-file symlinks are allowed).
Directories, FIFOs, devices, and sockets return redacted `io` diagnostics with
exit `5`; Unix opens are nonblocking and recheck the opened file type to reject
FIFO replacement races without waiting for a writer. These checks are not a
general filesystem I/O deadline. See
`docs/API_CONTRACT.md` for the field and exit-status contract.

## Centralized CI contract source

`.github/workflows/ci.yml` dispatches the reviewed central workflow
Expand Down
15 changes: 11 additions & 4 deletions crates/analysis_engine/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,17 @@ keywords.workspace = true
categories.workspace = true
publish = false

[[bin]]
name = "tepp-topic-lineage"
path = "src/bin/tepp_topic_lineage.rs"
test = false
bench = false

[dependencies]
corpus_split = { path = "../corpus_split", version = "0.2.0" }
event_core = { path = "../event_core", version = "0.2.0" }
membership_core = { path = "../membership_core", version = "0.2.0" }
relation_graph = { path = "../relation_graph", version = "0.2.0" }
serde = { workspace = true }
serde_json = { workspace = true }
sha2 = { workspace = true }
Expand All @@ -23,10 +32,8 @@ temporal_core = { path = "../temporal_core", version = "0.2.0" }
topic_measurement = { path = "../topic_measurement", version = "0.2.0" }
uuid.workspace = true

[dev-dependencies]
corpus_split = { path = "../corpus_split", version = "0.2.0" }
membership_core = { path = "../membership_core", version = "0.2.0" }
relation_graph = { path = "../relation_graph", version = "0.2.0" }
[target.'cfg(unix)'.dependencies]
libc = "0.2.189"

[lints]
workspace = true
16 changes: 16 additions & 0 deletions crates/analysis_engine/src/bin/tepp_topic_lineage.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
//! Offline command that runs the CPU `f64` topic-lineage estimator on one
//! bounded JSON input file and prints one digest-bound JSON result line.
//!
//! Usage: `tepp-topic-lineage <input.json>`. Exit status `0` means success,
//! `2` usage, `3` invalid input, `4` estimator non-convergence, and `5` I/O.

use std::io;
use std::process::ExitCode;

fn main() -> ExitCode {
ExitCode::from(analysis_engine::run_topic_lineage_cli(
std::env::args_os().skip(1),
&mut io::stdout().lock(),
&mut io::stderr().lock(),
))
}
10 changes: 10 additions & 0 deletions crates/analysis_engine/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ mod case_deletion_refit;
mod lineage_criterion;
mod topic_context_posterior;
mod topic_lineage_artifact;
mod topic_lineage_cli;

use serde::Serialize;
use sha2::{Digest, Sha256};
Expand Down Expand Up @@ -58,6 +59,15 @@ pub use topic_lineage_artifact::{
TOPIC_LINEAGE_MODEL_CONTRACT_VERSION, TOPIC_LINEAGE_OUTPUT_PROFILE, TopicLineageArtifact,
TopicLineageArtifactEdge, TopicLineageExecution, execute_topic_lineage_run,
};
/// Offline, file-driven topic-lineage execution contracts and bounds.
pub use topic_lineage_cli::{
TOPIC_LINEAGE_CLI_DOCUMENT_LIMIT, TOPIC_LINEAGE_CLI_EDGE_LIMIT,
TOPIC_LINEAGE_CLI_INPUT_BYTE_LIMIT, TOPIC_LINEAGE_CLI_INPUT_SCHEMA_VERSION,
TOPIC_LINEAGE_CLI_ITERATION_LIMIT, TOPIC_LINEAGE_CLI_NONZERO_LIMIT,
TOPIC_LINEAGE_CLI_OUTPUT_SCHEMA_VERSION, TOPIC_LINEAGE_CLI_SEED_LIMIT,
TOPIC_LINEAGE_CLI_VOCABULARY_LIMIT, TOPIC_LINEAGE_CLI_WORK_LIMIT, TopicLineageCliError,
TopicLineageCliOutput, run_topic_lineage, run_topic_lineage_cli,
};

/// Versioned artifact schema emitted by this engine.
pub const ANALYSIS_ARTIFACT_SCHEMA_VERSION: &str = "tepp.temporal_evidence_readiness.v1";
Expand Down
6 changes: 4 additions & 2 deletions crates/analysis_engine/src/topic_lineage_artifact.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ pub struct TopicLineageArtifactEdge {
pub successor_document_id: String,
/// Artifact-local global topic index.
pub topic_index: u64,
/// Minimum dominant-topic posterior mean across the linked documents.
/// Minimum dominant-topic MAP-fitted proportion across the linked documents.
/// Not a posterior mean, uncertainty probability, or causal effect.
pub association_strength: f64,
}

Expand Down Expand Up @@ -177,7 +178,8 @@ pub struct TopicLineageExecution {
///
/// # Errors
///
/// Returns a request/receipt/snapshot/cutoff/profile error, estimator failure,
/// Returns a request/receipt/snapshot/cutoff/profile error, estimator failure
/// (including shared typed resource refusal before dense seed allocation),
/// arithmetic error, or invalid/oversized artifact error.
pub fn execute_topic_lineage_run(
request: &AnalysisRunRequest,
Expand Down
Loading
Loading