diff --git a/AGENTS.md b/AGENTS.md
index cc46bf8..62fe053 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -129,13 +129,10 @@ so the version a reader picks in the navbar is always one they can install.
A snapshot is changed only where it was wrong for its own release, a patch release that changes
behaviour included. A new feature goes into `docs/`, never back into a snapshot.
-**Where this stands.** No snapshot exists yet, so the published site describes `main`, and a
-reader on 0.2.0 can meet calls their version lacks; the quick start note warns them. The steps
-above are all that is left to run, and `editCurrentVersion` is already set so a snapshot's "Edit
-this page" will point at `docs/`. The first cut waits on the first tagged release; the SDK has
-tagged only `v0.2.0`, and bumped `main` past it without releasing, so no number in between gets a
-snapshot. Nothing checks these rules automatically yet. The doc tests being added under
-`doctests/` are meant to, by building against pinned SDK and platform commits.
+**Where this stands.** The `1.0` snapshot was cut when Python, Rust and Java all released 1.0.0.
+It is served at `/` and `docs/` is served at `/next/`. The next cut waits on the next tagged `vX.Y.0`.
+Nothing checks these rules automatically yet. The doc tests being added under `doctests/` are meant
+to, by building against pinned SDK and platform commits.
## Every example has to be runnable
diff --git a/docs/quickstart.mdx b/docs/quickstart.mdx
index b9bf7e7..88f2306 100644
--- a/docs/quickstart.mdx
+++ b/docs/quickstart.mdx
@@ -11,15 +11,15 @@ From zero to a stored datapoint in about five minutes.
## 1. Install
-:::note Which version these pages describe
-These pages follow the SDKs as they are being developed, which can be ahead of the latest
-release. The Python and Rust SDKs are pre-1.0, so a release can still rename or remove a call.
-If an example calls something your installed SDK does not have, it may be newer than your release.
+:::note Which version this page describes
+This is the Next (unreleased) version of the documentation. It follows `main` of the SDKs, so it can
+use calls that the latest release does not have. Pick the release you have installed in the version
+menu in the navbar.
| SDK | Latest release |
| --- | --- |
-| Python | 0.2.0, on PyPI |
-| Rust | 0.2.0, on crates.io |
+| Python | 1.0.0, on PyPI |
+| Rust | 1.0.0, on crates.io |
| Java | 1.0.0, on Maven Central |
:::
@@ -58,7 +58,7 @@ pip install intellistream-datahub-sdk
```toml
# Cargo.toml
[dependencies]
-intellistream-datahub-sdk = "0.2"
+intellistream-datahub-sdk = "1.0"
```
@@ -75,7 +75,7 @@ import intellistream_datahub_sdk as dh
```toml
# Cargo.toml — a crate-wide rename
-dh = { package = "intellistream-datahub-sdk", version = "0.2" }
+dh = { package = "intellistream-datahub-sdk", version = "1.0" }
```
:::
diff --git a/docs/tutorial.mdx b/docs/tutorial.mdx
index c2a9180..8e83af1 100644
--- a/docs/tutorial.mdx
+++ b/docs/tutorial.mdx
@@ -98,7 +98,7 @@ portably:
```toml
# Cargo.toml
[dependencies]
-intellistream-datahub-sdk = "0.2"
+intellistream-datahub-sdk = "1.0"
tokio = { version = "1", features = ["full"] }
chrono = "0.4"
sysinfo = "0.37"
@@ -113,7 +113,7 @@ The code below lives in `src/main.rs`, with `#[tokio::main]` driving the async c
```toml
# Cargo.toml
[dependencies]
-intellistream-datahub-sdk = { version = "0.2", features = ["blocking"] }
+intellistream-datahub-sdk = { version = "1.0", features = ["blocking"] }
chrono = "0.4"
sysinfo = "0.37"
hostname = "0.4"
diff --git a/docusaurus.config.js b/docusaurus.config.js
index 0a38d58..c65ebdb 100644
--- a/docusaurus.config.js
+++ b/docusaurus.config.js
@@ -28,11 +28,11 @@ const config = {
docs: {
sidebarPath: './sidebars.js',
routeBasePath: '/', // docs at site root, GitBook-style
- // No `versions` block and no snapshot yet: `docs/` is the only version
- // and is served at '/'. AGENTS.md has the three lines to add when the
- // first release is tagged. Leave `lastVersion` out then too — it
- // defaults to the newest name in versions.json, so cutting a snapshot
- // moves readers onto it with nothing here to keep in step.
+ // `docs/` is the development version, served at /next/. The newest
+ // snapshot in versions.json is the release and is served at '/'.
+ // Leave `lastVersion` out: it defaults to the newest name in
+ // versions.json, so cutting a snapshot moves readers onto it.
+ versions: { current: { label: 'Next (unreleased)' } },
// Adds "Edit this page" to every doc. Docusaurus appends the file's
// path relative to this site directory. Note the branch here is
@@ -41,7 +41,7 @@ const config = {
// Send every "Edit this page" to docs/, including from a snapshot once
// one exists: a snapshot takes corrections for its own release, but a
// new feature belongs in docs/, and the stock link would invite the
- // wrong edit. No effect until the first snapshot.
+ // wrong edit.
editCurrentVersion: true,
},
blog: false, // SDK docs site — no blog
@@ -88,11 +88,7 @@ const config = {
// which pushState's the URL and then renders this site's own 404 —
// the href looks right in the HTML but the click never leaves the SPA.
{ type: 'html', position: 'right', value: 'Platform documentation' },
- // The site's own version, not an SDK version. Swap it for
- // { type: 'docsVersionDropdown', position: 'right' } once a snapshot
- // exists — with none, the dropdown renders as a lone link labelled
- // after the development version, which reads as a release.
- { type: 'html', position: 'right', value: 'v1.0' },
+ { type: 'docsVersionDropdown', position: 'right' },
// This site's own repo, so "GitHub" is unambiguous. The old link went
// to the SDK code on Gitea; if a code link is wanted too it needs its
// own item, since the SDK spans three language repos.
diff --git a/versioned_docs/version-1.0/advanced/_category_.json b/versioned_docs/version-1.0/advanced/_category_.json
new file mode 100644
index 0000000..01090b6
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/_category_.json
@@ -0,0 +1,9 @@
+{
+ "label": "Advanced scenarios (AI)",
+ "position": 7,
+ "link": {
+ "type": "generated-index",
+ "title": "Advanced scenarios (AI)",
+ "description": "Longer, end-to-end builds that go past CRUD into prediction and classification. No data-science background needed — start with 'Machine learning, gently' for the ideas in plain language, then 'Generate sample data' to get a sandbox you can run everything against. The SDK gets data in and out; the learning step in between is shown in Python. Each page carries an effort estimate."
+ }
+}
diff --git a/versioned_docs/version-1.0/advanced/asset-health-score.mdx b/versioned_docs/version-1.0/advanced/asset-health-score.mdx
new file mode 100644
index 0000000..4cef18a
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/asset-health-score.mdx
@@ -0,0 +1,174 @@
+---
+sidebar_position: 4
+title: Asset health scoring
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Asset health scoring
+
+:::info At a glance
+**Effort:** ~20–30 minutes · **You'll build:** a composite 0–100 health index from
+several signals, with a healthy/watch/critical classification · **Stack:** the SDK plus
+a little arithmetic, no model training.
+:::
+
+A machine rarely fails on one signal. Vibration is creeping up, the bearing runs a
+little hot, oil pressure sags at load, each is fine alone, but together they tell a
+story. A **health score** rolls those signals into one comparable number so an operator
+can rank a whole fleet at a glance and a dashboard can show green/amber/red. It's the
+lightweight cousin of [predictive maintenance](/advanced/predictive-maintenance): no
+model to train, just a transparent, tunable index.
+
+:::info Needs a sandbox
+This reads several pump signals. [Generate a sandbox](/advanced/generate-sample-data)
+first, section A ingests `pump_07_bearing_temp_c`, `pump_07_oil_pressure_kpa` and a
+stand-in `pump_07_vibration_anomaly` (or run [predictive maintenance](/advanced/predictive-maintenance)
+to produce the real one).
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Pull the latest value of each signal
+
+Take the most recent reading of each contributing series for the asset. The seed writes
+these hourly, so ask for the latest datapoint rather than a short window that may be empty.
+
+
+
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def latest(external_id):
+ pts = client.timeseries.retrieve_latest_datapoints([external_id])[0].get_datapoints()
+ return float(pts[-1].value)
+
+signals = {
+ "vibration": latest("pump_07_vibration_anomaly"), # 0..~1, from the anomaly model
+ "bearing_temp": latest("pump_07_bearing_temp_c"),
+ "oil_pressure": latest("pump_07_oil_pressure_kpa"),
+}
+```
+
+
+
+
+```java
+// Java has no latest-datapoint call: read the last day and take the newest point.
+// The scoring arithmetic below is shown in Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("pump_07_bearing_temp_c");
+filter.setStart(ZonedDateTime.now().minusHours(24));
+filter.setEnd(ZonedDateTime.now());
+filter.setLimit(100);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var pts = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+double bearingTemp = Double.parseDouble(pts.get(pts.size() - 1).getValue());
+```
+
+
+
+
+```rust
+// The latest datapoint, per signal; the scoring arithmetic below is shown in Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+
+let page = api.time_series
+ .retrieve_latest_datapoint(&DataWrapper::from(vec![
+ IdAndExtId::from_external_id("pump_07_bearing_temp_c")])).await?;
+let latest = &page.get_items()[0];
+let bearing_temp = latest.datapoints.last().and_then(|p| p.value).unwrap_or(f64::NAN);
+```
+
+
+
+
+## 2. Normalise each signal to a "badness" in 0–1
+
+Every signal lives on its own scale, so map each to a common 0 (fine) → 1 (alarm) range
+against its healthy and limit values. A reading at or below healthy scores 0; at or
+above the limit scores 1; in between, it ramps linearly.
+
+```python
+def badness(value, healthy, limit):
+ if limit == healthy:
+ return 0.0
+ return max(0.0, min(1.0, (value - healthy) / (limit - healthy)))
+
+LIMITS = { # (healthy, limit) per signal
+ "vibration": (0.05, 0.30),
+ "bearing_temp": (60.0, 90.0),
+ "oil_pressure": (350.0, 250.0), # inverted: lower is worse
+}
+parts = {k: badness(v, *LIMITS[k]) for k, v in signals.items()}
+```
+
+## 3. Weight, combine, and classify
+
+Weight the signals by how much each matters for this asset class, combine into a 0–100
+score (100 = perfect health), and bucket it.
+
+```python
+WEIGHTS = {"vibration": 0.5, "bearing_temp": 0.3, "oil_pressure": 0.2}
+
+badness_total = sum(parts[k] * WEIGHTS[k] for k in parts)
+score = round(100 * (1 - badness_total), 1)
+
+band = "healthy" if score >= 80 else "watch" if score >= 60 else "critical"
+```
+
+## 4. Publish the score and flag the bad ones
+
+Write the score back as its own series, now you can chart, rank and subscribe to asset
+health like any other signal, and raise an event when an asset drops to `critical`.
+
+```python
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="pump_07_health_score", name="Pump 07 health score", unit="score", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=[pd.Timestamp.now(tz="UTC")], values=[score], ts="pump_07_health_score")
+
+if band == "critical":
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"health_critical_pump_07_{int(pd.Timestamp.now().timestamp())}",
+ type="health_critical", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"asset": "pump_07", "score": str(score),
+ "worst_signal": max(parts, key=parts.get)})])
+
+# read the score back
+stored = client.timeseries.retrieve_latest_datapoints(["pump_07_health_score"])[0].get_datapoints()
+print(f"health {stored[-1].value:.1f} ({band})")
+assert abs(stored[-1].value - score) < 0.01, "the score was not written"
+```
+
+Run it across the fleet on a schedule and you have a single ranked health view,
+`worst_signal` in the metadata tells maintenance *why* each asset is red.
+
+## Where to take it further
+
+- **Feed it the model.** Swap the hand-set vibration limits for the
+ [anomaly score](/advanced/predictive-maintenance) as a direct input.
+- **Trend the score.** A falling health score over days is itself a predictor, forecast
+ it like any [other series](/advanced/demand-forecasting).
+- **Roll up the graph.** Average child scores up a [site graph](/guides/model-assets-graph)
+ for a line- or plant-level health number.
+
+## Further reading
+
+- **Normalising signals to a common scale**: [Feature scaling](https://en.wikipedia.org/wiki/Feature_scaling)
+- **The ideas in plain language**: [Machine learning, gently](/advanced/machine-learning-gently)
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate): pulling the contributing signals.
+- [Predictive maintenance](/advanced/predictive-maintenance): a learned input to the score.
+- [Turn readings into events](/guides/detect-events): flagging critical assets.
diff --git a/versioned_docs/version-1.0/advanced/data-cleaning-lineage.mdx b/versioned_docs/version-1.0/advanced/data-cleaning-lineage.mdx
new file mode 100644
index 0000000..b26f6a4
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/data-cleaning-lineage.mdx
@@ -0,0 +1,209 @@
+---
+sidebar_position: 0.7
+title: Data pipeline & lineage
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Build a data pipeline with full lineage
+
+:::info At a glance
+**Effort:** ~1–1.5 hours · **What you'll build:** a multi-step pipeline (clean →
+transform → extract features → score) where every derived series is written back **and**
+linked in the graph, so the whole `raw → … → result` chain is traceable both directions ·
+**Stack:** the SDK for data and graph, plus `pandas`, `numpy`.
+:::
+
+:::info Needs a sandbox
+This reads two raw sensors that already exist. [Generate a sandbox](/advanced/generate-sample-data)
+first, section K ingests `engine_temperature_raw` and `engine_vibration_raw`, defects and all.
+:::
+
+:::tip New to machine learning?
+No data-science background needed. Skim the [gentle primer](/advanced/machine-learning-gently)
+for any unfamiliar term.
+:::
+
+**The idea in one paragraph.** Real data never goes straight from sensor to model. It's
+cleaned, resampled, turned into features, combined, scored, often a dozen steps, each
+producing a new series. After a few of these, nobody remembers what came from what. Two
+questions then become impossible to answer: *"this number looks wrong, where did it come
+from?"* and *"this sensor was faulty, what did it poison?"* **Data lineage** fixes both:
+record every step as nodes in the graph, `raw → cleaning function → cleaned → feature
+function → feature → …`, and the answers become a single graph walk.
+
+## 1. Run the transformations, writing each result back
+
+Each step reads its input series and writes a **new** derived series (never over the
+input). A tiny helper keeps it readable:
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def load(ext_id):
+ rf = intellistream_datahub_sdk.RetrieveFilter(ts=ext_id,
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=7), end=pd.Timestamp.now(tz="UTC"), limit=100_000)
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pd.Series([float(p.value) for p in pts],
+ index=pd.to_datetime([p.timestamp for p in pts], utc=True)).sort_index()
+
+def store(ext_id, series, name=None, unit="value"):
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=ext_id, name=name or ext_id, unit=unit,
+ value_type="float",
+ description="Derived series — not a raw measurement.")])
+ s = series.dropna()
+ client.timeseries.insert_from_lists(timestamps=s.index, values=s.to_numpy(), ts=ext_id)
+```
+
+The pipeline, two raw sensors, cleaned, then several features extracted from each, then
+assembled and scored:
+
+```python
+# --- clean (range + rolling-median outliers + short-gap fill) ---
+def clean(s):
+ s = s.where((s > -1e6) & (s < 1e6))
+ med = s.rolling(7, center=True).median()
+ mad = (s - med).abs().rolling(7, center=True).median()
+ return s.mask((s - med).abs() > 5 * mad).interpolate(limit=5)
+
+temp_clean = clean(load("engine_temperature_raw")); store("engine_temperature_clean", temp_clean)
+vib_clean = clean(load("engine_vibration_raw")); store("engine_vibration_clean", vib_clean)
+
+# --- features off the cleaned temperature ---
+store("engine_temp_roll_mean", temp_clean.rolling("1h").mean()) # smoothed trend
+store("engine_temp_roc", temp_clean.diff()) # rate of change
+
+# --- features off the cleaned vibration ---
+store("engine_vib_rms", vib_clean.pow(2).rolling("1min").mean().pow(0.5)) # energy
+store("engine_vib_band", vib_clean.rolling("1min").std()) # proxy for band energy
+
+# --- assemble the features and score (your model goes here) ---
+feature_vector = pd.concat(
+ [load(f) for f in ["engine_temp_roll_mean", "engine_temp_roc",
+ "engine_vib_rms", "engine_vib_band"]], axis=1).mean(axis=1)
+store("engine_feature_vector", feature_vector)
+store("engine_health_score", (100 - feature_vector.abs()).clip(0, 100))
+```
+
+Every box in the pipeline is now a stored series, but stored series alone don't tell you
+how they *relate*. That's step 2.
+
+## 2. Record the lineage graph
+
+Model each transformation as a **function node** and link it up: each input series is
+`processed_by` the function, which `produces` an output series. Done across the pipeline,
+this builds a branching lineage DAG of 18 nodes (8 functions and 10 series) in one call.
+(`resources.create` is the same call in Java and Rust, see
+[model assets as a graph](/guides/model-assets-graph).)
+
+```python
+def fn(ext_id, name):
+ return intellistream_datahub_sdk.Resource(external_id=ext_id, name=name, labels=["PipelineStep"])
+
+functions = [
+ fn("clean_temp_fn", "Clean temperature"), fn("clean_vib_fn", "Clean vibration"),
+ fn("roll_mean_fn", "Rolling mean"), fn("roc_fn", "Rate of change"),
+ fn("rms_fn", "RMS energy"), fn("band_fn", "Band energy"),
+ fn("assemble_fn", "Assemble features"), fn("score_fn", "Health score"),
+]
+
+def proc(src, f): return intellistream_datahub_sdk.RelForm.by_external_ids(src, f, "processed_by")
+def prod(f, out): return intellistream_datahub_sdk.RelForm.by_external_ids(f, out, "produces")
+
+edges = [
+ proc("engine_temperature_raw", "clean_temp_fn"), prod("clean_temp_fn", "engine_temperature_clean"),
+ proc("engine_vibration_raw", "clean_vib_fn"), prod("clean_vib_fn", "engine_vibration_clean"),
+ # cleaned temperature branches into two features
+ proc("engine_temperature_clean", "roll_mean_fn"), prod("roll_mean_fn", "engine_temp_roll_mean"),
+ proc("engine_temperature_clean", "roc_fn"), prod("roc_fn", "engine_temp_roc"),
+ # cleaned vibration branches into two features
+ proc("engine_vibration_clean", "rms_fn"), prod("rms_fn", "engine_vib_rms"),
+ proc("engine_vibration_clean", "band_fn"), prod("band_fn", "engine_vib_band"),
+ # the four features converge into the feature vector, then the score
+ proc("engine_temp_roll_mean", "assemble_fn"), proc("engine_temp_roc", "assemble_fn"),
+ proc("engine_vib_rms", "assemble_fn"), proc("engine_vib_band", "assemble_fn"),
+ prod("assemble_fn", "engine_feature_vector"),
+ proc("engine_feature_vector", "score_fn"), prod("score_fn", "engine_health_score"),
+]
+
+client.resources.create(functions, edges)
+```
+
+That graph branches (each cleaned signal feeds two features), converges (four features
+into one vector), and runs nine nodes deep from raw sensor to health score:
+`engine_temperature_raw` → `clean_temp_fn` → `engine_temperature_clean` → `roll_mean_fn` →
+`engine_temp_roll_mean` → `assemble_fn` → `engine_feature_vector` → `score_fn` →
+`engine_health_score`.
+
+## 3. The payoff, trace it backward
+
+Someone questions the health score. Walk the graph back from it and the entire recipe
+appears: every function and every series between the score and the **raw sensors** it
+ultimately came from. No guessing, no stale wiki page.
+
+```python
+back = client.resources.fetch_related(external_id="engine_health_score", depth=12)
+raw_sources = {n.external_id for n in back.nodes if n.external_id.endswith("_raw")}
+print("this score derives from:", raw_sources) # engine_temperature_raw, engine_vibration_raw
+assert raw_sources == {"engine_temperature_raw", "engine_vibration_raw"}, raw_sources
+```
+
+## 4. The payoff, trace it forward (impact analysis)
+
+Now the opposite, and the bigger win. The temperature sensor is found faulty. **What did
+it poison?** Walk *forward* from the raw sensor and every downstream series it touched
+lights up, while the vibration-only features stay clean. That's the precise list to
+recompute or quarantine.
+
+`fetch_related` walks edges in both directions, so from the raw sensor it reaches the
+whole connected pipeline, the vibration branch included. Each returned edge keeps its
+direction, though, `start` → `end`, so follow those from the sensor to keep only what lies
+downstream of it.
+
+```python
+net = client.resources.fetch_related(external_id="engine_temperature_raw", depth=12)
+ext_id = {n.id: n.external_id for n in net.nodes}
+start = next(i for i, x in ext_id.items() if x == "engine_temperature_raw")
+
+downstream, frontier = set(), [start]
+while frontier:
+ node = frontier.pop()
+ for e in net.edges:
+ if e.start == node and e.end not in downstream:
+ downstream.add(e.end)
+ frontier.append(e.end)
+
+affected = sorted(ext_id[i] for i in downstream if ext_id[i].startswith("engine_"))
+print("affected by the bad temperature sensor:", affected)
+# engine_temperature_clean, engine_temp_roll_mean, engine_temp_roc,
+# engine_feature_vector, engine_health_score — but NOT the vibration features
+assert "engine_health_score" in affected and "engine_vib_rms" not in affected, affected
+```
+
+This is the same [blast-radius traversal](/guides/correlate-alarms) used for asset
+failures, here applied to data: one bad input, and lineage tells you exactly how far the
+damage spreads.
+
+## Where to take it further
+
+- **Version the functions.** Bump `clean_temp_fn` to `clean_temp_fn_v2` and the old
+ outputs still point at the version that made them, reproducibility for free.
+- **Quality events.** Emit a `data_quality` [event](/guides/detect-events) per run with
+ what each step changed, for an audit trail beside the lineage.
+- **Feed the models.** Point the [forecasters](/advanced/lstm-forecasting) and
+ [detectors](/advanced/predictive-maintenance) at the cleaned/feature series, and their
+ outputs become new nodes on the same lineage graph.
+
+## Further reading
+
+- **Data lineage**: [Wikipedia](https://en.wikipedia.org/wiki/Data_lineage)
+- **Directed acyclic graph (DAG)**: [Wikipedia](https://en.wikipedia.org/wiki/Directed_acyclic_graph)
+- **Outlier removal (Hampel / MAD)**: [Median absolute deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation)
+
+## See also
+
+- [Model assets as a graph](/guides/model-assets-graph): the nodes-and-relations basics.
+- [Correlate alarms with the graph](/guides/correlate-alarms): the traversal behind both payoffs.
+- [Generate sample data](/advanced/generate-sample-data): raw signals to run the pipeline on.
diff --git a/versioned_docs/version-1.0/advanced/demand-forecasting.mdx b/versioned_docs/version-1.0/advanced/demand-forecasting.mdx
new file mode 100644
index 0000000..c52d7a2
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/demand-forecasting.mdx
@@ -0,0 +1,201 @@
+---
+sidebar_position: 2
+title: Demand forecasting
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Demand forecasting
+
+:::info At a glance
+**Effort:** ~1 hour · **You'll build:** a lag + calendar feature set and a gradient-
+boosted forecaster · **Stack:** the SDK for data in/out, plus `pandas`, `numpy`,
+`scikit-learn`.
+:::
+
+Forecasting turns history into a plan: how much power a feeder will draw tomorrow, how
+many units a store will sell next week, how much load a network will carry at the busy
+hour. The pattern is the same regardless of domain, past values plus calendar effects
+predict the next ones, and the forecast becomes a **new series** you can chart against
+actuals and alert on.
+
+:::info Needs a sandbox
+This reads a load history that already exists. [Generate a sandbox](/advanced/generate-sample-data)
+first, section G ingests the `feeder_f12_load_mw` curve this build forecasts.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Load the history
+
+Pull a long, regularly-sampled history of the quantity you want to forecast.
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="feeder_f12_load_mw",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=120),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h", limit=100_000) # hourly history
+
+points = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+demand = pd.Series([float(p.average) for p in points],
+ index=pd.to_datetime([p.timestamp for p in points], utc=True)).sort_index()
+demand = demand.asfreq("1h").interpolate() # regular hourly grid
+```
+
+
+
+
+```java
+// Java pulls the same hourly history; the modelling below is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("feeder_f12_load_mw");
+filter.setStart(ZonedDateTime.now().minusDays(120));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg"));
+filter.setGranularity("1h");
+filter.setLimit(100000);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var history = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust pulls the same hourly history; the modelling below is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("feeder_f12_load_mw".into()),
+ start: Some(Utc::now() - chrono::Duration::days(120)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into()]),
+ granularity: Some("1h".into()),
+ limit: Some(100_000),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let history = &page.get_items()[0];
+```
+
+
+
+
+## 2. Build lag + calendar features
+
+Demand depends on its recent past (an hour ago, a day ago, a week ago) and on the
+calendar (hour of day, day of week, season). Encode both into a supervised frame whose
+target is the value to predict.
+
+```python
+def build_frame(series):
+ df = pd.DataFrame({"y": series})
+ for lag in (1, 24, 168): # 1h, 1 day, 1 week
+ df[f"lag_{lag}"] = series.shift(lag)
+ df["roll_24"] = series.shift(1).rolling(24).mean()
+ df["hour"] = df.index.hour
+ df["dow"] = df.index.dayofweek
+ df["month"] = df.index.month
+ return df.dropna()
+
+frame = build_frame(demand)
+FEATURES = [c for c in frame.columns if c != "y"]
+```
+
+## 3. Train the forecaster
+
+A gradient-boosted regressor handles the non-linear calendar interactions well and
+needs little tuning. Hold out the most recent slice to check it honestly.
+
+```python
+from sklearn.ensemble import HistGradientBoostingRegressor
+from sklearn.metrics import mean_absolute_error
+
+split = int(len(frame) * 0.85)
+train, test = frame.iloc[:split], frame.iloc[split:]
+
+model = HistGradientBoostingRegressor(max_iter=400, learning_rate=0.05)
+model.fit(train[FEATURES], train["y"])
+
+mae = mean_absolute_error(test["y"], model.predict(test[FEATURES]))
+print(f"holdout MAE: {mae:.2f} MW")
+```
+
+## 4. Forecast the horizon and write it back
+
+Future lags aren't known, so forecast **recursively**, predict one hour, feed it back
+in as the next hour's lag, and step forward. Then publish the forecast as its own
+series alongside the actuals.
+
+```python
+HORIZON = 48 # hours ahead
+history = demand.copy()
+preds = {}
+
+for step in range(HORIZON):
+ t = history.index[-1] + pd.Timedelta(hours=1)
+ row = {
+ "lag_1": history.iloc[-1],
+ "lag_24": history.iloc[-24],
+ "lag_168": history.iloc[-168],
+ "roll_24": history.iloc[-24:].mean(),
+ "hour": t.hour, "dow": t.dayofweek, "month": t.month,
+ }
+ yhat = model.predict(pd.DataFrame([row])[FEATURES])[0]
+ preds[t] = yhat
+ history.loc[t] = yhat # feed the prediction forward
+
+forecast = pd.Series(preds)
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="feeder_f12_load_mw_forecast", name="Feeder F12 load — 48h forecast", unit="mw", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=forecast.index, values=forecast.to_numpy(), ts="feeder_f12_load_mw_forecast")
+
+# read the horizon back: one point per forecast hour
+stored = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts="feeder_f12_load_mw_forecast",
+ start=forecast.index[0], end=forecast.index[-1] + pd.Timedelta(minutes=1),
+ limit=1000))[0].get_datapoints()
+print(f"{len(stored)} forecast hours stored")
+assert len(stored) == HORIZON, f"expected {HORIZON}, got {len(stored)}"
+```
+
+With the forecast stored as a series, a [threshold rule](/guides/detect-events) on it
+becomes a *predictive* alert, raise a `capacity_risk` event when the forecast, not the
+actual, is set to exceed a limit, hours before it happens.
+
+## Where to take it further
+
+- **Exogenous drivers.** Add weather, price, or a promotions flag as features, usually
+ the biggest accuracy win.
+- **Prediction intervals.** Train quantile models (`loss="quantile"`) for a P10/P90 band
+ instead of a single line.
+- **Backtest rolling.** Re-fit on a sliding window and score each step to track drift
+ over time.
+
+## Further reading
+
+- **Gradient boosting**: [Wikipedia](https://en.wikipedia.org/wiki/Gradient_boosting) · [scikit-learn](https://scikit-learn.org/stable/modules/ensemble.html#histogram-based-gradient-boosting)
+- **Time series forecasting**: [Wikipedia](https://en.wikipedia.org/wiki/Time_series)
+- **The ideas in plain language**: [Machine learning, gently](/advanced/machine-learning-gently)
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate): building the hourly history.
+- [Turn readings into events](/guides/detect-events): turning the forecast into a predictive alert.
+- [Renewable energy](/industries/energy-utilities/wind) · [Retail](/industries/transport-logistics/retail), domains this fits.
diff --git a/versioned_docs/version-1.0/advanced/fraud-classification.mdx b/versioned_docs/version-1.0/advanced/fraud-classification.mdx
new file mode 100644
index 0000000..644914e
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/fraud-classification.mdx
@@ -0,0 +1,244 @@
+---
+sidebar_position: 5
+title: Fraud classification (graph + ML)
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Fraud classification, graph features + machine learning
+
+:::info At a glance
+**Effort:** ~1–2 hours · **You'll build:** graph-derived features from network traversal,
+combined with behavioural features, feeding a supervised classifier · **Stack:** the SDK
+for data and traversal, plus `networkx`, `pandas`, `scikit-learn`.
+:::
+
+Money laundering doesn't look suspicious one
+payment at a time; it looks suspicious in the *shape* of the network, funds fanning out
+through mules and looping back, combined with behaviour like rapid pass-through. A rules
+engine flags thousands of alerts a day; a classifier that scores each one by its network
+shape *and* its behaviour lets investigators work the riskiest first.
+
+The technique generalises to any "score an entity by its connections plus its behaviour"
+problem, [wafer-lot risk](/industries/manufacturing-process/semiconductor), [insurance rings](/industries/financial-services/insurance-fraud),
+[telecom abuse](/industries/technology-operations/network).
+
+:::info Needs a sandbox
+This walks a transfer graph that already exists. [Generate a sandbox](/advanced/generate-sample-data)
+first, section F creates the flagged ring around `account_77310`.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Pull the account's network
+
+For a given account, [walk the money-flow graph](/guides/correlate-alarms) to get its
+cluster, the sub-graph of accounts and the transfers between them.
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd, networkx as nx
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def network_of(account_external_id):
+ return client.resources.fetch_related(external_id=account_external_id, depth=4)
+```
+
+
+
+
+```java
+// Java extracts the same sub-graph; the feature engineering below is Python.
+RelatedResourcesForm form = new RelatedResourcesForm();
+form.setExternalId("account_77310");
+form.setDepth(4);
+form.setRelationshipTypes(List.of("sent_to"));
+ResourceNetwork net = client.resources().fetchRelated(form);
+// net.nodes() / net.edges() — feed your feature pipeline
+```
+
+
+
+
+```rust
+// Rust extracts the same sub-graph; the feature engineering below is Python.
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let net = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("account_77310")
+ .with_depth(4)
+ .with_relationship_types(vec!["sent_to".into()])).await?;
+// net.nodes() / net.edges() — feed your feature pipeline
+```
+
+
+
+
+## 2. Turn the network shape into features
+
+This is the heart of it. Load the returned nodes and edges into a directed graph and
+compute the structural signals that distinguish a laundering cluster from a normal
+account's neighbourhood, ring size, pass-through "mules", and whether the money loops
+back to where it started.
+
+```python
+def graph_features(account_external_id):
+ net = network_of(account_external_id)
+
+ g = nx.DiGraph()
+ for e in net.edges: # edges carry numeric start/end ids
+ g.add_edge(e.start, e.end)
+
+ accounts = [n for n in net.nodes if n.external_id.startswith("account_")]
+ focal = next(n.id for n in net.nodes if n.external_id == account_external_id)
+
+ # mules: pure pass-through nodes — money in, same money straight out
+ mules = sum(1 for n in g.nodes if g.in_degree(n) == 1 and g.out_degree(n) == 1)
+ # loops: is the focal account inside a cycle? (funds returning is a strong signal)
+ cycles = [c for c in nx.strongly_connected_components(g) if len(c) > 1]
+ in_loop = any(focal in c for c in cycles)
+
+ return {
+ "ring_size": len(accounts),
+ "edge_count": g.number_of_edges(),
+ "out_degree": g.out_degree(focal) if focal in g else 0,
+ "in_degree": g.in_degree(focal) if focal in g else 0,
+ "mule_count": mules,
+ "in_loop": int(in_loop),
+ "reach": len(nx.descendants(g, focal)) if focal in g else 0,
+ }
+```
+
+## 3. Add behavioural features from the transaction series
+
+Network shape alone produces false positives, a busy merchant looks connected too.
+Combine it with how the account *behaves*: how much flows through, and whether what comes
+in goes straight back out (the pass-through ratio that defines a mule).
+
+```python
+def behaviour_features(account_external_id, hours=24):
+ def agg(metric, *aggregates):
+ rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts=f"{account_external_id}_{metric}",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=hours),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=list(aggregates), granularity=f"{hours}h")
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pts[-1] if pts else None
+
+ out = agg("outflow", "sum", "avg")
+ inn = agg("inflow", "sum")
+ out_sum = float(out.sum) if out else 0.0
+ in_sum = float(inn.sum) if inn else 0.0
+ return {
+ "outflow_24h": out_sum,
+ "avg_txn": float(out.average) if out else 0.0,
+ "pass_through": out_sum / max(in_sum, 1.0), # ~1.0 = money in → straight out
+ }
+
+def features(account_external_id):
+ return {**graph_features(account_external_id), **behaviour_features(account_external_id)}
+```
+
+## 4. Train on confirmed outcomes
+
+Past investigations are your labels: accounts that became Suspicious Activity Reports
+(SARs) are positives, the rest negatives. Build the feature matrix and train. SARs are
+rare, so judge the model on **average precision** (ranking quality), not accuracy.
+
+```python
+from sklearn.ensemble import HistGradientBoostingClassifier
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import average_precision_score, classification_report
+
+# labelled history: [(account_external_id, was_sar), ...]
+# In production this comes from your case-management system. For the sandbox, the flagged
+# ring around account_77310 are the known SARs and the other connected accounts are cleared.
+def load_investigation_outcomes():
+ ring = ["account_77310", "account_44120", "account_61885", "account_22907"]
+ cleared = [f"account_{i}" for i in range(1, 31)]
+ return [(a, True) for a in ring] + [(a, False) for a in cleared]
+
+labelled = load_investigation_outcomes()
+X = pd.DataFrame([features(acct) for acct, _ in labelled])
+y = np.array([int(was_sar) for _, was_sar in labelled])
+
+Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)
+clf = HistGradientBoostingClassifier(max_iter=300, learning_rate=0.05,
+ min_samples_leaf=5, # modest labelled set — let leaves specialise
+ class_weight="balanced").fit(Xtr, ytr)
+
+proba = clf.predict_proba(Xte)[:, 1]
+print(f"average precision: {average_precision_score(yte, proba):.3f}")
+print(classification_report(yte, (proba > 0.5).astype(int)))
+```
+
+## 5. Score live alerts and explain the call
+
+Score a new alert, **write the risk back as a series** so it's trendable, and raise a
+`sar_candidate` event for the high-risk ones. Crucially, include the features that drove
+the score, an investigator needs to know *why* it's flagged, not just that it is.
+
+```python
+alert = "account_77310"
+feat = features(alert)
+risk = clf.predict_proba(pd.DataFrame([feat]))[0, 1]
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id=f"{alert}_aml_risk", name="AML risk score", unit="score", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=[pd.Timestamp.now(tz="UTC")], values=[risk], ts=f"{alert}_aml_risk")
+
+if risk > 0.7:
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"sar_candidate_{alert}_{int(pd.Timestamp.now().timestamp())}",
+ type="sar_candidate", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={
+ "account": alert,
+ "risk": f"{risk:.2f}",
+ "ring_size": str(feat["ring_size"]),
+ "mule_count": str(feat["mule_count"]),
+ "in_loop": str(feat["in_loop"]),
+ "pass_through": f"{feat['pass_through']:.2f}",
+ })])
+
+# read the score back
+pts = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts=f"{alert}_aml_risk",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(minutes=5),
+ end=pd.Timestamp.now(tz="UTC")))[0].get_datapoints()
+print(f"{len(pts)} risk score(s) stored, latest {pts[-1].value:.2f}")
+assert len(pts) >= 1, "the risk score was not written"
+```
+
+The risk score is now a live series the investigations dashboard ranks on, and each
+`sar_candidate` event arrives with its network-shape evidence attached, the alert and
+the reason for it, together.
+
+## Where to take it further
+
+- **Real explainability.** Swap the hand-picked metadata for SHAP values so every score
+ comes with its true top contributors.
+- **Richer graph features.** Add betweenness, community detection, or counterparty
+ diversity, the structural signal is deep.
+- **Stream it.** Drive scoring from the [flagged-payments subscription](/industries/financial-services/aml)
+ so alerts are scored the moment they're raised.
+
+## Further reading
+
+- **Gradient boosting**: [Wikipedia](https://en.wikipedia.org/wiki/Gradient_boosting)
+- **Graph centrality** (the network features), [Wikipedia](https://en.wikipedia.org/wiki/Centrality)
+- **Shapley values / SHAP** (explainability), [Wikipedia](https://en.wikipedia.org/wiki/Shapley_value)
+
+## See also
+
+- [Correlate alarms with the graph](/guides/correlate-alarms): the traversal these features are built on.
+- [Banking, AML](/industries/financial-services/aml) · [Insurance, fraud rings](/industries/financial-services/insurance-fraud), the quick versions.
+- [Predictive maintenance](/advanced/predictive-maintenance): the other end-to-end model build.
diff --git a/versioned_docs/version-1.0/advanced/generate-sample-data.mdx b/versioned_docs/version-1.0/advanced/generate-sample-data.mdx
new file mode 100644
index 0000000..864e7a0
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/generate-sample-data.mdx
@@ -0,0 +1,355 @@
+---
+sidebar_position: 0.5
+title: Generate sample data
+---
+
+# Generate sample data
+
+:::info Start here
+The advanced scenarios **read** series, events and graphs that already exist. To run any
+of them end-to-end, first populate a sandbox with realistic synthetic data, that's what
+this page does, using the same SDK ingestion calls you'd use for real data. Point each
+scenario's `external_id`s at the series you create here.
+:::
+
+Everything below is Python (`numpy` + `pandas` + the SDK). Each generator builds a signal
+with the *shape* a model needs to learn, then ingests it.
+
+## A reusable ingest helper
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def ingest(external_id, index, values, unit=None, name=None):
+ try:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id=external_id, name=name or external_id, unit=unit or "value", value_type="float")])
+ except intellistream_datahub_sdk.DataHubException as e:
+ if e.status_code != 409: # a second run finds the series already there
+ raise
+ client.timeseries.insert_from_lists(timestamps=index, values=np.asarray(values), ts=external_id)
+ print(f"ingested {len(values):,} points → {external_id}")
+```
+
+## A. A degrading sensor signal
+
+For [predictive maintenance](/advanced/predictive-maintenance),
+[LSTM anomaly detection](/advanced/lstm-anomaly-detection) and
+[health scoring](/advanced/asset-health-score): a vibration signal that runs healthy,
+then develops a fault, rising amplitude and impulsiveness over the final stretch. It is
+written at **one reading a second**, which is the rate the predictive-maintenance page's
+`SAMPLE_HZ` and window length assume.
+
+```python
+def degrading_vibration(hours=24, fs_per_hour=3600):
+ n = hours * fs_per_hour # one reading a second
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=n, freq="1s")
+ t = np.linspace(0, 1, n)
+ healthy = 0.6 * np.sin(2 * np.pi * 50 * np.arange(n) / fs_per_hour) # base tone
+ fault = np.where(t > 0.8, (t - 0.8) * 4, 0.0) # ramps in late
+ impulses = fault * np.random.standard_normal(n) * 3 # impulsiveness
+ return idx, healthy + fault + impulses + np.random.normal(0, 0.1, n)
+
+idx, vib = degrading_vibration()
+ingest("pump_07_vibration_mm_s", idx, vib, unit="mm_s")
+```
+
+The same pump's slower signals, bearing temperature, oil pressure, and a stand-in
+anomaly score, feed the [asset health score](/advanced/asset-health-score):
+
+```python
+hrs = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=24*60, freq="1h")
+n = len(hrs)
+ingest("pump_07_bearing_temp_c", hrs, 55 + np.linspace(0, 18, n) + np.random.normal(0, 1, n), unit="celsius")
+ingest("pump_07_oil_pressure_kpa", hrs, 320 - np.linspace(0, 60, n) + np.random.normal(0, 5, n), unit="kpa")
+# stand-in for the predictive-maintenance output, if you haven't run that scenario yet
+ingest("pump_07_vibration_anomaly", hrs, np.clip(np.linspace(0, 0.9, n) + np.random.normal(0, 0.03, n), 0, 1))
+```
+
+## B. A production decline curve
+
+For [LSTM forecasting](/advanced/lstm-forecasting): a hyperbolic decline with noise and a
+weekly wobble, the shape of a producing well's rate.
+
+```python
+def decline_curve(days=540, qi=1200, b=0.8, d=0.006):
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=days, freq="1d")
+ t = np.arange(days)
+ rate = qi / (1 + b * d * t) ** (1 / b) # hyperbolic decline (still visibly falling at the tail)
+ weekly = 1 + 0.03 * np.sin(2 * np.pi * t / 7)
+ return idx, rate * weekly + np.random.normal(0, 5, days)
+
+idx, rate = decline_curve()
+ingest("well_a12_oil_rate_bpd", idx, rate, unit="bpd")
+```
+
+## C. Correlated multivariate process data, with a fault
+
+For [PCA process monitoring](/advanced/pca-process-monitoring): several sensors that move
+together normally, with a fault window where one breaks the correlation. (Section J below
+reuses this exact shape with drilling names for the multivariate
+[LSTM autoencoder](/advanced/lstm-anomaly-detection).)
+
+```python
+def correlated_process(minutes=60*24*14):
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=minutes, freq="1min")
+ feed = 100 + 10 * np.sin(2 * np.pi * np.arange(minutes) / (60*24)) + np.random.normal(0, 1, minutes)
+ top_temp = 150 + 0.4 * feed + np.random.normal(0, 0.5, minutes) # correlated with feed
+ pressure = 200 + 0.8 * feed + np.random.normal(0, 1, minutes)
+ reflux = 2.0 + 0.005 * feed + np.random.normal(0, 0.02, minutes)
+ bottom_temp = 340 + 0.6 * feed + np.random.normal(0, 0.6, minutes) # bottoms also track feed
+ reboiler = 30 + 0.15 * feed + np.random.normal(0, 0.3, minutes) # reboiler duty follows feed
+ top_temp[-600:] += np.linspace(0, 12, 600) # fault: temp drifts off the relationship
+ return idx, {"cdu_1_feed_bpd": feed, "cdu_1_top_temp_c": top_temp,
+ "cdu_1_pressure_kpa": pressure, "cdu_1_reflux_ratio": reflux,
+ "cdu_1_bottom_temp_c": bottom_temp, "cdu_1_reboiler_duty_gj": reboiler}
+
+idx, channels = correlated_process()
+for tag, values in channels.items():
+ ingest(tag, idx, values)
+```
+
+## D. Sparse lab samples vs. dense sensors
+
+For the [soft sensor](/advanced/random-forest-soft-sensor): cheap online sensors every
+minute, plus an expensive lab measurement only once an hour that depends on them.
+
+```python
+idx, channels = correlated_process() # reuse the dense sensors above
+# lab value is a (hidden) function of the sensors, sampled hourly with measurement noise
+lab_idx = idx[::60]
+lab = (5.0 + 0.05 * (channels["cdu_1_top_temp_c"][::60] - 190) # strong, learnable dependence
+ - 2.0 * (channels["cdu_1_reflux_ratio"][::60] - 2.5) # on the online sensors...
+ + np.random.normal(0, 0.05, len(lab_idx))) # ...with small measurement noise
+ingest("cdu_1_product_sulfur_ppm_lab", lab_idx, lab, unit="ppm")
+```
+
+## E. A labelled failure history (events) and the sensors leading up to it
+
+For [XGBoost failure prediction](/advanced/xgboost-failure-prediction): historical
+failures as events, so each `(asset, day)` row can be labelled.
+
+```python
+for day_offset in (40, 95, 160, 240): # a few past failures
+ when = pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=day_offset)
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"esp_failure_a12_{int(when.timestamp())}",
+ type="esp_failure", status="closed", event_time=when,
+ metadata={"pump": "pump_esp_a12"})])
+```
+
+The model reads three sensors on `pump_esp_a12`. Give each a slow degradation that ramps
+up in the fortnight before every seeded failure (and resets after the implied repair), so
+the 7-day-ahead label has real signal to learn from:
+
+```python
+days = 300
+esp_idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=days*24, freq="1h")
+n = len(esp_idx)
+stress = np.zeros(n) # 0 = healthy, 1 = about to fail
+for day_offset in (40, 95, 160, 240): # the same failures seeded above
+ fail = n - 1 - day_offset*24
+ lo = max(fail - 14*24 + 1, 0) # degrade over the fortnight before
+ stress[lo:fail+1] = np.maximum(stress[lo:fail+1], np.linspace(0, 1, fail + 1 - lo))
+ingest("pump_esp_a12_intake_pressure_bar", esp_idx, 90 - 25*stress + np.random.normal(0, 1.5, n), unit="bar")
+ingest("pump_esp_a12_motor_temp_c", esp_idx, 70 + 45*stress + np.random.normal(0, 1.0, n), unit="celsius")
+ingest("pump_esp_a12_vibration_mm_s", esp_idx, 2.0 + 6*stress + np.abs(np.random.normal(0, 0.2, n)), unit="mm_s")
+```
+
+## F. A small graph
+
+For the graph-feature builds ([fraud classification](/advanced/fraud-classification),
+[K-Means communities](/advanced/kmeans-clustering)): 30 ordinary accounts
+`account_1…30`, plus the flagged ring that fraud classification traverses from
+`account_77310`. Fraud classification also reads each account's behavioural series
+(`_inflow` / `_outflow`), seeded below.
+
+```python
+ring = ["account_77310", "account_44120", "account_61885", "account_22907"] # a flagged laundering ring
+ordinary = [f"account_{i}" for i in range(1, 31)] # 30 ordinary accounts
+nodes = [intellistream_datahub_sdk.Resource(external_id=a, name=a.replace("_", " ").title(), labels=["Account"])
+ for a in ordinary + ring]
+# ordinary accounts: an acyclic transfer web — money only flows "forward", so no loops or pass-through mules
+edges = [intellistream_datahub_sdk.RelForm.by_external_ids(f"account_{i}", f"account_{i+step}", "sent_to")
+ for i in range(1, 31) for step in (1, 3) if i + step <= 30]
+# the flagged ring: a tight cycle fraud-classification walks out from account_77310 (funds loop back)
+edges += [intellistream_datahub_sdk.RelForm.by_external_ids(ring[k], ring[(k+1) % len(ring)], "sent_to")
+ for k in range(len(ring))]
+client.resources.create(nodes, edges)
+```
+
+Fraud classification turns the money *in* and *out* of each account into features, a
+pass-through mule shows inflow ≈ outflow, an ordinary account doesn't. Seed both series
+for every account in the graph:
+
+```python
+hrs = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=24*7, freq="1h")
+n = len(hrs)
+profiles = {a: (50_000, 0.97) for a in ring} # ring mules: high volume, in≈out
+profiles.update({a: (np.random.uniform(2_000, 8_000),
+ np.random.uniform(0.3, 0.6)) for a in ordinary}) # ordinary: only part flows back out
+for acct, (base, ratio) in profiles.items():
+ inflow = np.clip(base * (1 + 0.3*np.sin(2*np.pi*np.arange(n)/24)) + np.random.normal(0, base*0.05, n), 0, None)
+ outflow = np.clip(inflow * ratio + np.random.normal(0, base*0.03, n), 0, None) # ~ratio of it flows back out
+ ingest(f"{acct}_inflow", hrs, inflow, unit="usd")
+ ingest(f"{acct}_outflow", hrs, outflow, unit="usd")
+```
+
+## G. An electrical feeder load curve
+
+For [demand forecasting](/advanced/demand-forecasting): 120 days of hourly feeder load
+with the daily double-peak, a lighter weekend, and a slow seasonal drift, the shape the
+forecaster learns.
+
+```python
+def feeder_load(days=120):
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=days*24, freq="1h")
+ t = np.arange(days*24)
+ daily = 8 * np.sin(2*np.pi*((t % 24) - 6)/24) # morning & evening peaks
+ weekend = -3 * (((t // 24) % 7) >= 5) # weekends run lighter
+ season = 4 * np.sin(2*np.pi*t/(24*365)) # slow seasonal swing
+ return idx, np.clip(30 + daily + weekend + season + np.random.normal(0, 1, len(t)), 5, None)
+
+idx, load = feeder_load()
+ingest("feeder_f12_load_mw", idx, load, unit="mw")
+```
+
+## H. Fish-pen oxygen, with crash episodes
+
+For the [oxygen-crash early warning](/advanced/oxygen-crash-early-warning): 30 days of
+dissolved oxygen and water temperature at 15-minute resolution, with a few episodes where
+oxygen plunges, the crashes the model learns to see coming.
+
+```python
+def pen_oxygen(days=30):
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=days*24*4, freq="15min")
+ n = len(idx)
+ temp = 12 + 2*np.sin(2*np.pi*np.arange(n)/(24*4)) + np.random.normal(0, 0.2, n)
+ do = 8.5 - 0.15*(temp - 12) + np.random.normal(0, 0.2, n) # warmer water holds less O2
+ for start in (5*24*4, 14*24*4, 23*24*4, 27*24*4): # crash episodes (incl. a late one)
+ do[start:start+8] -= np.linspace(0, 4, 8)
+ do[start+8:start+20] -= 4
+ return idx, np.clip(do, 1, None), temp
+
+idx, do, temp = pen_oxygen()
+ingest("pen_h_07_dissolved_oxygen_mg_l", idx, do, unit="mg_l")
+ingest("pen_h_07_water_temp_c", idx, temp, unit="celsius")
+```
+
+## I. Operating channels, one unit's regimes and a peer fleet
+
+For [K-Means](/advanced/kmeans-clustering): four channels for `unit_3` as it swings
+between idle, ramp, steady and overload (the *operating regimes* section clusters on all
+four), plus just `load_mw` for a small pump fleet running at different duty levels (the
+*asset cohorts* section only clusters `load_mw` across the pumps).
+
+```python
+def operating_modes(asset, minutes=60*24*14, load_scale=1.0, phase=0.0):
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=minutes, freq="1min")
+ duty = (np.sin(2*np.pi*np.arange(minutes)/(60*8) + phase) + 1) / 2 # 0..1 duty cycle
+ load = np.clip(duty*80*load_scale + np.random.normal(0, 3, minutes), 0, None)
+ return idx, {
+ f"{asset}_load_mw": load,
+ f"{asset}_temp_c": 120 + 0.5*load + np.random.normal(0, 2, minutes),
+ f"{asset}_pressure_kpa": 180 + 0.7*load + np.random.normal(0, 3, minutes),
+ f"{asset}_flow_m3h": 20 + 0.4*load + np.random.normal(0, 1, minutes),
+ }
+
+# unit_3: all four channels — the operating-regimes section clusters on the full state.
+idx, channels = operating_modes("unit_3")
+for tag, values in channels.items():
+ ingest(tag, idx, values)
+
+# Peer fleet: the asset-cohorts section only clusters load_mw across pumps, so seed just that.
+for asset, scale, phase in [("pump_07", 0.9, 0.0), ("pump_08", 1.05, 0.4),
+ ("pump_11", 0.6, 0.8), ("pump_19", 1.3, 1.2)]:
+ idx, channels = operating_modes(asset, load_scale=scale, phase=phase)
+ ingest(f"{asset}_load_mw", idx, channels[f"{asset}_load_mw"])
+```
+
+## J. Correlated drilling channels, with a kick
+
+For the multivariate [LSTM autoencoder](/advanced/lstm-anomaly-detection): section C's
+exact shape with drilling names, mud flow-in, flow-out, pit volume and standpipe pressure
+move together until a late **kick** (gas influx) pushes returns and pit volume up while
+standpipe pressure sags, breaking the joint pattern the autoencoder learned.
+
+```python
+def drilling_process(seconds=60*60): # 1 h at 1 s — the LSTM autoencoder works in seconds
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=seconds, freq="1s")
+ flow_in = 600 + 40*np.sin(2*np.pi*np.arange(seconds)/3600) + np.random.normal(0, 4, seconds)
+ flow_out = 590 + 0.97*(flow_in - 600) + np.random.normal(0, 4, seconds) # returns track pumps-in
+ pit_vol = 240 + 0.02*(flow_in - 600) + np.random.normal(0, 0.5, seconds) # steady active pits
+ standpipe = 2800 + 3.0*(flow_in - 600) + np.random.normal(0, 15, seconds)
+ flow_out[-600:] += np.linspace(0, 60, 600) # kick over the last 10 min: returns gain over flow-in
+ pit_vol[-600:] += np.linspace(0, 8, 600) # pit volume rises with the influx
+ standpipe[-600:] -= np.linspace(0, 120, 600) # standpipe sags as gas lightens the column
+ return idx, {"rig_dw1_flow_in_gpm": flow_in, "rig_dw1_flow_out_gpm": flow_out,
+ "rig_dw1_pit_volume_bbl": pit_vol, "rig_dw1_standpipe_psi": standpipe}
+
+idx, channels = drilling_process()
+for tag, values in channels.items():
+ ingest(tag, idx, values)
+```
+
+## K. Messy raw sensors to clean
+
+For the [data pipeline & lineage](/advanced/data-cleaning-lineage) build: two *raw* engine
+sensors carrying the real-world defects that pipeline exists to fix, spike outliers,
+scattered `NaN` dropouts, a longer gap and a stuck/flatline stretch. (Everything
+downstream, the `*_clean`, feature and score series, is written *by* that recipe, so
+only these raw inputs are seeded here.)
+
+```python
+def messy_sensor(hours=24*7, base=90.0, swing=8.0, noise=1.0):
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=hours*60, freq="1min")
+ n = len(idx)
+ s = pd.Series(base + swing*np.sin(2*np.pi*np.arange(n)/(60*24)) + np.random.normal(0, noise, n), index=idx)
+ s.iloc[np.random.randint(0, n, n//200)] += np.random.normal(0, 12, n//200) # spike outliers
+ s.iloc[np.random.randint(0, n, n//150)] = np.nan # scattered dropouts
+ gap = np.random.randint(0, n - 240); s.iloc[gap:gap+180] = np.nan # a longer gap
+ stuck = np.random.randint(0, n - 240); s.iloc[stuck:stuck+120] = base # stuck / flatline
+ s = s.dropna() # dropouts/gaps become real gaps — NaN values can't be ingested
+ return s.index, s.to_numpy()
+
+idx, temp = messy_sensor(base=90, swing=8, noise=1.0)
+ingest("engine_temperature_raw", idx, temp, unit="celsius")
+idx, vib = messy_sensor(base=3.0, swing=1.2, noise=0.25)
+ingest("engine_vibration_raw", idx, vib, unit="mm_s")
+```
+
+## L. A noisy alarm: four transients and one real excursion
+
+For the [sustained-condition alarm](/advanced/sustained-alarm-window): a pump discharge
+pressure that spikes briefly four times and then stays above the limit for eighty minutes.
+The transients are deliberately **higher** than the sustained excursion, because duration
+rather than severity is what that rule judges, and a signal where the real one is also the
+tallest would let a plain threshold look good.
+
+```python
+LIMIT_BAR = 4.5
+
+def noisy_discharge(hours=6, per_minute=1):
+ n = hours * 60 * per_minute
+ step = f"{60 // per_minute}s"
+ idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=n, freq=step)
+ t = np.arange(n, dtype=float) / per_minute # minutes from the start
+ v = 3.6 + 0.25*np.sin(t/23) + 0.15*np.sin(t/7.5) # ordinary running
+ for centre in (30, 75, 140, 200): # four brief transients
+ v += 2.6 * np.exp(-((t - centre) / 2.0) ** 2)
+ v += np.where((t > 250) & (t < 330), 1.6, 0.0) # the one that matters
+ return idx, v + np.random.normal(0, 0.03, n)
+
+idx, bar = noisy_discharge()
+ingest("pump_p101_discharge_bar", idx, bar, unit="bar")
+print(f"{(bar > LIMIT_BAR).sum()} of {len(bar)} readings are above {LIMIT_BAR}")
+```
+
+---
+
+With the sandbox populated, every advanced scenario will find the series, events and
+graph it reads. As you swap in real data, the only thing that changes is where the
+numbers come from, the [ingestion guide](/guides/ingest-timeseries) covers doing it at
+production volume.
diff --git a/versioned_docs/version-1.0/advanced/kmeans-clustering.mdx b/versioned_docs/version-1.0/advanced/kmeans-clustering.mdx
new file mode 100644
index 0000000..3c3f314
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/kmeans-clustering.mdx
@@ -0,0 +1,211 @@
+---
+sidebar_position: 8
+title: K-Means, regimes, cohorts & communities
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# K-Means, operating regimes, asset cohorts & network communities
+
+:::info At a glance
+**Effort:** ~1–1.5 hours · **You'll build:** behavioural and graph feature vectors and
+cluster them three ways · **Stack:** the SDK for data and traversal, plus `numpy`,
+`pandas`, `scikit-learn`, and `networkx` for the graph part.
+:::
+
+K-Means finds structure with no labels: it groups points so each sits near its cluster's
+centre. That one idea answers three different operational questions, all covered here,
+*what modes does this asset run in?* (operating regimes), *which assets behave alike?*
+(cohorts), and *which parts of the network belong together?* (communities). The trick is
+always the same: turn the thing you want to group into a **feature vector**.
+
+:::info Needs a sandbox
+These steps read series (and a graph for section 3) that already exist.
+[Generate a sandbox](/advanced/generate-sample-data) first: sections I and F.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Asset cohorts, group assets that behave alike
+
+Summarise each asset by a behavioural signature, a few aggregates of its key series,
+then cluster. Assets land in peer groups you can benchmark within.
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+from sklearn.preprocessing import StandardScaler
+from sklearn.cluster import KMeans
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def signature(asset):
+ def agg(metric, *aggs):
+ rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts=f"{asset}_{metric}",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=30),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=list(aggs), granularity="30d")
+ return client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()[-1]
+ load = agg("load_mw", "avg", "max")
+ return {"avg_load": float(load.average), "peak_load": float(load.max)}
+
+assets = ["pump_07", "pump_08", "pump_11", "pump_19"] # ...your fleet
+X = pd.DataFrame([signature(a) for a in assets], index=assets)
+Xs = StandardScaler().fit_transform(X)
+
+# k must be well below the number of assets, or every asset is its own cluster
+km = KMeans(n_clusters=2, n_init=10, random_state=0).fit(Xs)
+X["cohort"] = km.labels_
+```
+
+
+
+
+```java
+// Java pulls each asset's aggregates the same way; the clustering below is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("pump_07_load_mw");
+filter.setStart(ZonedDateTime.now().minusDays(30));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg", "max"));
+filter.setGranularity("30d");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var sig = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust pulls each asset's aggregates the same way; the clustering below is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("pump_07_load_mw".into()),
+ start: Some(Utc::now() - chrono::Duration::days(30)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into(), "max".into()]),
+ granularity: Some("30d".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let sig = &page.get_items()[0];
+```
+
+
+
+
+### The payoff: the asset that doesn't fit its cohort
+
+An asset far from its own cluster's centre is behaving unlike its peers, a strong,
+label-free anomaly signal. Flag it.
+
+```python
+dist = np.linalg.norm(Xs - km.cluster_centers_[km.labels_], axis=1)
+for asset, d in zip(assets, dist):
+ if d > np.percentile(dist, 90):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"peer_outlier_{asset}_{int(pd.Timestamp.now().timestamp())}",
+ type="peer_outlier", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"asset": asset, "cohort": str(int(km.labels_[assets.index(asset)]))})])
+
+outliers = list(client.events.filter(type="peer_outlier", limit=100))
+print(f"{len(outliers)} peer outlier(s) on record")
+```
+
+## 2. Operating regimes, group an asset's *states*
+
+Now cluster *time* instead of assets. Each row is one moment described by the asset's
+sensor vector; the clusters are its operating modes, idle, ramp, steady, overload.
+Labelling history this way makes "we only see this fault in mode 3" answerable.
+
+```python
+def load_series(external_id): # one-minute averages over the recent window
+ rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts=external_id,
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=14),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1m", limit=100_000)
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pd.Series([float(p.average) for p in pts],
+ index=pd.to_datetime([p.timestamp for p in pts], utc=True)).sort_index()
+
+# one row per minute: [load, temp, pressure, flow] for a single asset
+states = pd.DataFrame({c: load_series(f"unit_3_{c}") for c in
+ ["load_mw", "temp_c", "pressure_kpa", "flow_m3h"]}).dropna()
+
+regimes = KMeans(n_clusters=4, n_init=10, random_state=0).fit_predict(
+ StandardScaler().fit_transform(states))
+states["regime"] = regimes # now every minute is tagged with its operating mode
+```
+
+## 3. Network communities, cluster the graph
+
+K-Means needs vectors, and a raw graph isn't one, so featurise it. Either **enrich the
+behavioural vector with graph-structural features** (so two assets are "alike" only if
+they behave *and* sit similarly in the network), or cluster the structure directly with
+**spectral clustering**, which is literally K-Means on the graph Laplacian's
+eigenvectors.
+
+```python
+import networkx as nx
+
+# (a) structural features per node, to add to the behavioural signature
+def graph_features(asset):
+ net = client.resources.fetch_related(external_id=asset, depth=2)
+ g = nx.Graph(); g.add_edges_from((e.start, e.end) for e in net.edges)
+ fid = next((n.id for n in net.nodes if n.external_id == asset), None)
+ pr = nx.pagerank(g) if g.number_of_edges() else {}
+ return {
+ "degree": g.degree(fid) if fid in g else 0,
+ "pagerank": pr.get(fid, 0.0),
+ "clustering": nx.clustering(g, fid) if fid in g else 0.0,
+ }
+
+# (b) or detect communities directly — spectral clustering = K-Means on the Laplacian
+from sklearn.cluster import SpectralClustering
+
+big_network = client.resources.fetch_related(external_id="account_1", depth=6)
+g = nx.Graph(); g.add_edges_from((e.start, e.end) for e in big_network.edges)
+A = nx.to_numpy_array(g)
+communities = SpectralClustering(n_clusters=2, affinity="precomputed", # k ≤ number of nodes
+ assign_labels="kmeans").fit_predict(A)
+```
+
+So yes, K-Means works **with** graphs: feed it structural features, or let spectral
+clustering turn the network into vectors for it.
+
+## Choosing k
+
+Don't guess the number of clusters, let the data say. The **silhouette score** peaks at
+the k that separates clusters best:
+
+```python
+from sklearn.metrics import silhouette_score
+scores = {k: silhouette_score(Xs, KMeans(k, n_init=10, random_state=0).fit_predict(Xs))
+ for k in range(2, len(Xs))} # the silhouette needs fewer clusters than points
+best_k = max(scores, key=scores.get)
+```
+
+## Further reading
+
+- **k-means**: [Wikipedia](https://en.wikipedia.org/wiki/K-means_clustering)
+- **Spectral clustering** (clustering a graph), [Wikipedia](https://en.wikipedia.org/wiki/Spectral_clustering)
+- **Silhouette score** (choosing k), [Wikipedia](https://en.wikipedia.org/wiki/Silhouette_%28clustering%29)
+
+## See also
+
+- [Correlate alarms with the graph](/guides/correlate-alarms): the traversal the graph features build on.
+- [Asset health scoring](/advanced/asset-health-score): score within a cohort once you have one.
+- [Fraud classification](/advanced/fraud-classification): graph features feeding a supervised model.
diff --git a/versioned_docs/version-1.0/advanced/lstm-anomaly-detection.mdx b/versioned_docs/version-1.0/advanced/lstm-anomaly-detection.mdx
new file mode 100644
index 0000000..3e61858
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/lstm-anomaly-detection.mdx
@@ -0,0 +1,192 @@
+---
+sidebar_position: 7
+title: LSTM autoencoder anomaly detection
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Multivariate anomaly detection with an LSTM autoencoder
+
+:::info At a glance
+**Effort:** ~2 hours · **You'll build:** an LSTM autoencoder that learns normal
+multivariate sequences and flags reconstruction error · **Stack:** the SDK for data
+in/out, plus `numpy`, `pandas`, `scikit-learn` (scaling) and `tensorflow`/`keras`.
+:::
+
+Some failures only show up in how *several* signals move *together over time*. A
+drilling kick is the classic case: no single channel is alarming, but the joint pattern
+of mud flow-in, flow-out, pit volume and standpipe pressure over a few seconds is
+unmistakable. An **LSTM autoencoder** learns to reconstruct that normal joint behaviour;
+when a real event arrives, it can't reconstruct it well, and the reconstruction error
+spikes. It needs **no examples of failure**, only normal operation.
+
+:::info Needs a sandbox
+These steps read series that already exist. [Generate a sandbox](/advanced/generate-sample-data)
+first, section J ingests the correlated drilling channels (with an injected kick) this build watches.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Load the normal multivariate window
+
+Pull several channels over a period the operation ran normally, align them, and scale.
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+from sklearn.preprocessing import StandardScaler
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+CHANNELS = ["rig_dw1_flow_in_gpm", "rig_dw1_flow_out_gpm",
+ "rig_dw1_pit_volume_bbl", "rig_dw1_standpipe_psi"]
+
+def load(external_id, start, end):
+ rf = intellistream_datahub_sdk.RetrieveFilter(ts=external_id, start=start, end=end, limit=100_000)
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pd.Series([float(p.value) for p in pts],
+ index=pd.to_datetime([p.timestamp for p in pts], utc=True)).sort_index()
+
+start = pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=1) # a stretch of normal running
+end = pd.Timestamp.now(tz="UTC") - pd.Timedelta(minutes=12) # ends before the recent kick
+frame = pd.DataFrame({c: load(c, start, end) for c in CHANNELS}).asfreq("1s").interpolate()
+
+scaler = StandardScaler()
+normal = scaler.fit_transform(frame.to_numpy())
+```
+
+
+
+
+```java
+// Java pulls each channel the same way; the model below is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("rig_dw1_flow_out_gpm");
+filter.setStart(ZonedDateTime.now().minusHours(1));
+filter.setEnd(ZonedDateTime.now().minusMinutes(12));
+filter.setLimit(100000);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var points = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust pulls each channel the same way; the model below is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("rig_dw1_flow_out_gpm".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(1)),
+ end: Some(Utc::now() - chrono::Duration::minutes(12)),
+ limit: Some(100_000),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+```
+
+
+
+
+## 2. Build sequences and the autoencoder
+
+Slice the multivariate signal into fixed windows of shape `(timesteps, channels)`. The
+autoencoder squeezes each window through a bottleneck and rebuilds it, learning only
+what normal sequences look like.
+
+```python
+from tensorflow.keras.models import Sequential
+from tensorflow.keras.layers import LSTM, RepeatVector, TimeDistributed, Dense
+
+STEPS, N_FEAT = 30, len(CHANNELS)
+
+def sequences(arr, n):
+ return np.stack([arr[i:i+n] for i in range(len(arr) - n)])
+
+X = sequences(normal, STEPS) # (samples, STEPS, N_FEAT)
+
+model = Sequential([
+ LSTM(32, activation="tanh", input_shape=(STEPS, N_FEAT)), # encoder
+ RepeatVector(STEPS),
+ LSTM(32, activation="tanh", return_sequences=True), # decoder
+ TimeDistributed(Dense(N_FEAT)),
+])
+model.compile(optimizer="adam", loss="mse")
+model.fit(X, X, epochs=30, batch_size=128, validation_split=0.1, verbose=0)
+```
+
+## 3. Score by reconstruction error and set a limit
+
+The per-window reconstruction error on normal data sets the alarm threshold; anything
+above is something the model has never seen.
+
+```python
+recon = model.predict(X, verbose=0)
+train_error = np.mean((X - recon) ** 2, axis=(1, 2))
+THRESHOLD = np.percentile(train_error, 99.5)
+```
+
+## 4. Watch live and raise an event
+
+Pull the most recent window, score it, write the anomaly score back as a series, and
+raise an event when it stays above the limit, catching the kick from the joint pattern,
+seconds in.
+
+```python
+recent = scaler.transform(
+ pd.DataFrame({c: load(c, pd.Timestamp.now(tz="UTC") - pd.Timedelta(minutes=10),
+ pd.Timestamp.now(tz="UTC")) for c in CHANNELS})
+ .asfreq("1s").interpolate().to_numpy())
+
+Xr = sequences(recent, STEPS)
+error = np.mean((Xr - model.predict(Xr, verbose=0)) ** 2, axis=(1, 2))
+score_index = pd.date_range(periods=len(error), end=pd.Timestamp.now(tz="UTC"), freq="1s")
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="rig_dw1_anomaly_score", name="Rig DW1 multivariate anomaly score", unit="score", value_type="float")])
+client.timeseries.insert_from_lists(timestamps=score_index, values=error, ts="rig_dw1_anomaly_score")
+
+if error[-3:].mean() > THRESHOLD:
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"kick_detected_dw1_{int(pd.Timestamp.now().timestamp())}",
+ type="kick_detected", status="critical",
+ event_time=score_index[-1],
+ metadata={"rig": "rig_deepwater_1", "score": f"{error[-1]:.3f}", "model": "lstm_ae_v1"})])
+
+# read the score back: one point per scored second
+stored = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts="rig_dw1_anomaly_score", start=score_index[0], end=pd.Timestamp.now(tz="UTC"),
+ limit=10_000))[0].get_datapoints()
+print(f"{len(stored)} scores stored, latest {stored[-1].value:.3f} against a limit of {THRESHOLD:.3f}")
+assert len(stored) == len(error), f"expected {len(error)}, got {len(stored)}"
+```
+
+## Where to take it further
+
+- **Attribute the anomaly.** Per-channel reconstruction error shows *which* signal broke
+ the pattern, flow-out vs. pit volume points to different failure modes.
+- **Compare with the simpler detector.** The [Isolation Forest](/advanced/predictive-maintenance)
+ on engineered features is cheaper; reach for the LSTM when the *temporal* pattern matters.
+
+## Further reading
+
+- **LSTM**: [Wikipedia](https://en.wikipedia.org/wiki/Long_short-term_memory)
+- **Autoencoder**: [Wikipedia](https://en.wikipedia.org/wiki/Autoencoder)
+- **Anomaly detection**: [Wikipedia](https://en.wikipedia.org/wiki/Anomaly_detection)
+
+## See also
+
+- [Oil & gas, drilling operations](/industries/oil-and-gas/drilling): the reactive version this upgrades.
+- [Predictive maintenance](/advanced/predictive-maintenance): feature-based anomaly detection.
+- [Turn readings into events](/guides/detect-events): emitting the alarm.
diff --git a/versioned_docs/version-1.0/advanced/lstm-forecasting.mdx b/versioned_docs/version-1.0/advanced/lstm-forecasting.mdx
new file mode 100644
index 0000000..4fd4726
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/lstm-forecasting.mdx
@@ -0,0 +1,188 @@
+---
+sidebar_position: 6
+title: Sequence forecasting with an LSTM
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Sequence forecasting with an LSTM
+
+:::info At a glance
+**Effort:** ~2 hours · **You'll build:** a sliding-window sequence model that learns
+temporal dynamics directly · **Stack:** the SDK for data in/out, plus `numpy`, `pandas`,
+`scikit-learn` (scaling) and `tensorflow`/`keras`.
+:::
+
+The [gradient-boosted forecaster](/advanced/demand-forecasting) predicts from
+hand-engineered lag features. An **LSTM** takes a different route: it reads the raw
+sequence and *learns* the temporal structure itself, useful when the dynamics are
+non-linear and the right lags aren't obvious, like an oil well's production decline,
+where rate, pressure and water-cut interact over time.
+
+We'll forecast a well's oil rate, but the pattern is the same for any signal with memory.
+
+:::info Needs a sandbox
+These steps read a series that already exists. [Generate a sandbox](/advanced/generate-sample-data)
+first, section B ingests the decline curve this build forecasts.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Load the history
+
+Pull a long, regularly-sampled history and scale it, neural nets train best on values
+in a small range.
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+from sklearn.preprocessing import MinMaxScaler
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="well_a12_oil_rate_bpd",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=730),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1d", limit=100_000)
+pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+
+rate = pd.Series([float(p.average) for p in pts],
+ index=pd.to_datetime([p.timestamp for p in pts], utc=True)).sort_index().asfreq("1d").interpolate()
+
+scaler = MinMaxScaler()
+scaled = scaler.fit_transform(rate.to_numpy().reshape(-1, 1))
+```
+
+
+
+
+```java
+// Java pulls the same daily history; the model below is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("well_a12_oil_rate_bpd");
+filter.setStart(ZonedDateTime.now().minusDays(730));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg"));
+filter.setGranularity("1d");
+filter.setLimit(100000);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var points = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust pulls the same daily history; the model below is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("well_a12_oil_rate_bpd".into()),
+ start: Some(Utc::now() - chrono::Duration::days(730)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into()]),
+ granularity: Some("1d".into()),
+ limit: Some(100_000),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+```
+
+
+
+
+## 2. Frame it as input window → forecast horizon
+
+An LSTM learns "given the last `LOOKBACK` days, what do the next `HORIZON` days look
+like?". Slice the series into overlapping pairs: each input is `LOOKBACK` days, each
+target is the whole `HORIZON` that follows. Predicting the horizon in **one shot** (rather
+than one day at a time, feeding each prediction back in) avoids the error build-up that
+makes recursive forecasts drift off the trend.
+
+```python
+LOOKBACK, HORIZON = 90, 90
+
+def windows(arr, lookback, horizon):
+ X = np.stack([arr[i:i+lookback, 0] for i in range(len(arr) - lookback - horizon)])
+ Y = np.stack([arr[i+lookback:i+lookback+horizon, 0] for i in range(len(arr) - lookback - horizon)])
+ return X[..., None], Y # X: (samples, LOOKBACK, 1), Y: (samples, HORIZON)
+
+X, Y = windows(scaled, LOOKBACK, HORIZON)
+split = int(len(X) * 0.85)
+Xtr, Xte, Ytr, Yte = X[:split], X[split:], Y[:split], Y[split:]
+```
+
+## 3. Train the LSTM
+
+A single LSTM layer feeding a dense layer with one output *per horizon step* learns the
+whole forecast at once.
+
+```python
+from tensorflow.keras.models import Sequential
+from tensorflow.keras.layers import LSTM, Dense
+from tensorflow.keras.callbacks import EarlyStopping
+
+model = Sequential([LSTM(64, input_shape=(LOOKBACK, 1)), Dense(HORIZON)])
+model.compile(optimizer="adam", loss="mse")
+model.fit(Xtr, Ytr, validation_data=(Xte, Yte),
+ epochs=60, batch_size=32,
+ callbacks=[EarlyStopping(patience=8, restore_best_weights=True)], verbose=0)
+```
+
+## 4. Forecast the horizon and write it back
+
+The model outputs the whole horizon in a single call, feed it the most recent
+`LOOKBACK` days, inverse-scale the result, and publish it as its own series.
+
+```python
+pred = model.predict(scaled[-LOOKBACK:].reshape(1, LOOKBACK, 1), verbose=0)
+forecast = scaler.inverse_transform(pred.reshape(-1, 1)).ravel()
+index = pd.date_range(rate.index[-1] + pd.Timedelta(days=1), periods=HORIZON, freq="1d")
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="well_a12_oil_rate_forecast", name="Well A-12 oil-rate forecast", unit="bpd", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=index, values=forecast, ts="well_a12_oil_rate_forecast")
+
+# read the horizon back: one point per forecast day
+stored = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts="well_a12_oil_rate_forecast",
+ start=index[0], end=index[-1] + pd.Timedelta(minutes=1), limit=1000))[0].get_datapoints()
+print(f"{len(stored)} forecast days stored")
+assert len(stored) == HORIZON, f"expected {HORIZON}, got {len(stored)}"
+```
+
+The decline curve is now a stored series, chart it against actuals, and a
+[threshold rule](/guides/detect-events) on it flags when the well is forecast to drop
+below its economic limit.
+
+## Where to take it further
+
+- **Go multivariate.** Feed rate, tubing pressure and water-cut together (input shape
+ `(N, 3)`) so the model uses their interaction, usually a big accuracy gain.
+- **Encoder–decoder.** For long horizons, a seq2seq LSTM decodes the horizon internally,
+ more expressive than a single dense layer when the forecast shape is complex.
+- **Quantiles.** Train with a pinball loss for a P10/P90 band, not just a point forecast.
+
+## Further reading
+
+- **LSTM**: [Wikipedia](https://en.wikipedia.org/wiki/Long_short-term_memory) · [Keras](https://keras.io/api/layers/recurrent_layers/lstm/)
+- **Recurrent neural networks**: [Wikipedia](https://en.wikipedia.org/wiki/Recurrent_neural_network)
+- **Decline curve analysis** (the domain), [Wikipedia](https://en.wikipedia.org/wiki/Decline_curve_analysis)
+
+## See also
+
+- [Demand forecasting](/advanced/demand-forecasting): the feature-based alternative.
+- [Oil & gas, production monitoring](/industries/oil-and-gas/production): the domain this serves.
+- [Turn readings into events](/guides/detect-events): alerting on the forecast.
diff --git a/versioned_docs/version-1.0/advanced/machine-learning-gently.mdx b/versioned_docs/version-1.0/advanced/machine-learning-gently.mdx
new file mode 100644
index 0000000..3c88de9
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/machine-learning-gently.mdx
@@ -0,0 +1,124 @@
+---
+sidebar_position: 0
+title: Machine learning, gently
+---
+
+# Machine learning, gently
+
+:::tip No maths background needed
+You do **not** need a statistics or data-science background to use the advanced
+scenarios. This page explains every idea they rely on in plain language, once, so the
+scenarios themselves can stay short. Read it first and the rest will make sense.
+:::
+
+The advanced scenarios all follow the same simple shape:
+
+> **get data out of the platform → learn a pattern from it → write a result back.**
+
+The SDK does the first and last parts (you already know those). The middle part, the
+"learn a pattern" bit, is what people call *machine learning*. Here's all you need to
+know about it to follow along.
+
+## Five terms
+
+- **Model**: a thing that has *learned a pattern* from past data and can apply it to new
+ data. Think of it as a function you didn't write by hand; you showed it examples and it
+ worked the rule out itself.
+- **Feature**: a single number that describes something useful about your data. "The
+ average vibration over the last minute" is a feature. Models work on lists of features,
+ not raw readings.
+- **Training**: showing the model lots of past examples so it can find the pattern. Done
+ once, up front; afterwards the model is fast.
+- **Label**: the answer you want to predict, when you have it. "This pump failed" is a
+ label. Some methods need labels (you teach by example); some don't.
+- **Score**: the model's output for a new example: a predicted number, a probability, or
+ a group it belongs to. The scenarios write this back to the platform as a new series or
+ an event.
+
+That's the whole vocabulary. Everything else is just *which* kind of pattern you're
+learning.
+
+## The four jobs, in plain English
+
+Every advanced scenario is doing one of these four things. Pick by the question you're
+asking:
+
+| You want to… | …it's called | Plain-English example |
+| --- | --- | --- |
+| Predict a **number** in the future | **forecasting** | "How much will this well produce next month?" |
+| Predict a **yes/no or a category** | **classification** | "Will this pump fail in the next week?" |
+| Spot the **weird / unusual** | **anomaly detection** | "Is this reading unlike anything normal?" |
+| **Group** similar things together | **clustering** | "Which assets behave alike?" |
+
+## The algorithms, one line each
+
+Each scenario uses one of these. You don't need to understand the maths, just the
+one-line idea and *when* you'd reach for it. The links go to plain explanations if you're
+curious.
+
+**For spotting the unusual (anomaly detection)**
+
+- **Isolation Forest**: learns what "normal" looks like and flags anything that stands
+ apart, with no examples of failure needed.
+ [What it is](https://en.wikipedia.org/wiki/Isolation_forest) ·
+ used in [Predictive maintenance](/advanced/predictive-maintenance).
+- **Autoencoder**: a model that learns to *redraw* normal data; when it can't redraw
+ something well, that thing is abnormal.
+ [What it is](https://en.wikipedia.org/wiki/Autoencoder) ·
+ used in [LSTM anomaly detection](/advanced/lstm-anomaly-detection).
+- **PCA**: boils dozens of related sensors down to a few summary numbers, then notices
+ when they stop relating to each other the normal way.
+ [What it is](https://en.wikipedia.org/wiki/Principal_component_analysis) ·
+ used in [Process monitoring with PCA](/advanced/pca-process-monitoring).
+
+**For predicting the future (forecasting)**
+
+- **LSTM**: a neural network with a memory, good at learning patterns that play out over
+ time (like a decline curve or a daily cycle).
+ [What it is](https://en.wikipedia.org/wiki/Long_short-term_memory) ·
+ used in [LSTM forecasting](/advanced/lstm-forecasting).
+- **Gradient boosting**: builds many tiny rules-of-thumb that together make accurate
+ predictions from tabular features; the dependable workhorse.
+ [What it is](https://en.wikipedia.org/wiki/Gradient_boosting) ·
+ used in [Demand forecasting](/advanced/demand-forecasting).
+
+**For predicting a label (classification)**
+
+- **XGBoost**: a fast, very popular version of gradient boosting, with a handy report of
+ which features mattered most.
+ [What it is](https://en.wikipedia.org/wiki/XGBoost) ·
+ used in [Failure prediction](/advanced/xgboost-failure-prediction).
+- **Random Forest**: averages the opinions of many decision trees; robust and almost
+ tuning-free. Works for labels *or* numbers.
+ [What it is](https://en.wikipedia.org/wiki/Random_forest) ·
+ used in [Soft sensor](/advanced/random-forest-soft-sensor).
+
+**For grouping (clustering)**
+
+- **K-Means**: sorts things into a chosen number of groups so that members of a group are
+ alike. [What it is](https://en.wikipedia.org/wiki/K-means_clustering) ·
+ used in [K-Means clustering](/advanced/kmeans-clustering).
+
+## What you actually need to run these
+
+The modelling code is **Python**, because that's where the data-science tools live,
+mainly [`scikit-learn`](https://scikit-learn.org/) (the standard ML toolbox),
+[`pandas`](https://pandas.pydata.org/) (tables of data) and, for the neural networks,
+[`tensorflow`/`keras`](https://keras.io/). Install with `pip`; each scenario lists what
+it needs at the top.
+
+The DataHub SDK's job is unchanged: it **gets the data out** (the same `retrieve` calls
+you've seen) and **writes results back** (the same `ingest` and event calls). The Java and
+Rust clients do those data steps too, the [reference](/reference/client) has the
+equivalents, but the learning step in the middle is Python.
+
+## A safe way to start
+
+1. **Populate a sandbox.** [Generate sample data](/advanced/generate-sample-data) creates
+ realistic signals to practise on, nothing touches real systems.
+2. **Pick the scenario that matches your question** from the four jobs above.
+3. **Run it top to bottom.** Each one is written to be followed step by step, with the
+ *why* spelled out as you go.
+
+You won't break anything by experimenting, and you don't have to understand the maths to
+get a useful result, that's rather the point.
diff --git a/versioned_docs/version-1.0/advanced/oxygen-crash-early-warning.mdx b/versioned_docs/version-1.0/advanced/oxygen-crash-early-warning.mdx
new file mode 100644
index 0000000..8c146f6
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/oxygen-crash-early-warning.mdx
@@ -0,0 +1,160 @@
+---
+sidebar_position: 3
+title: Early warning (predict before it happens)
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Early warning, predicting a crash before it happens
+
+:::info At a glance
+**Effort:** ~1 hour · **You'll build:** a forward-looking labelled dataset and a
+classifier that fires *ahead* of the event · **Stack:** the SDK for data in/out, plus
+`pandas`, `numpy`, `scikit-learn`.
+:::
+
+A threshold alarm tells you the bad thing is *already happening*. For some problems
+that's too late, by the time dissolved oxygen in a [salmon pen](/industries/agriculture-food/salmon-farming)
+hits the danger line, fish are already stressed. The goal here is to fire **before** the
+crash: learn the early signature of a developing crash and predict it 45 minutes out,
+while aeration can still prevent it.
+
+The technique, a classifier trained on a **forward-looking label**, generalises to any
+"predict the incident ahead of time" problem: a transformer about to trip, a server
+about to breach SLO, a line about to jam.
+
+:::info Needs a sandbox
+This trains on pen-oxygen history. [Generate a sandbox](/advanced/generate-sample-data)
+first, section H ingests `pen_h_07_dissolved_oxygen_mg_l` and `pen_h_07_water_temp_c`
+with the crash episodes the model learns.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Load the signals that precede a crash
+
+Pull the pen's history at a few-minute cadence, the oxygen itself plus the drivers that
+move it (temperature, and tide if you have it).
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def load(external_id, days=90):
+ rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts=external_id,
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=days),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="5m", limit=100_000)
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pd.Series([float(p.average) for p in pts],
+ index=pd.to_datetime([p.timestamp for p in pts], utc=True)).sort_index()
+
+do = load("pen_h_07_dissolved_oxygen_mg_l")
+temp = load("pen_h_07_water_temp_c")
+df = pd.DataFrame({"do": do, "temp": temp}).asfreq("5min").interpolate()
+```
+
+## 2. Engineer features and a *forward-looking* label
+
+The trick is the label: for each moment, look **ahead** and mark whether a crash happens
+in the next 45 minutes. The features only use the past, so the model learns to see a
+crash coming.
+
+```python
+CRITICAL = 5.0 # mg/L — below this is dangerous
+HORIZON = 9 # 9 × 5min = 45 minutes ahead
+
+# features: current level, recent slope, volatility, temperature, time of day
+df["do_slope"] = df["do"].diff().rolling(6).mean() # trend over ~30 min
+df["do_std"] = df["do"].rolling(6).std()
+df["temp_slope"] = df["temp"].diff().rolling(6).mean()
+df["hour"] = df.index.hour
+
+# label: will DO drop below CRITICAL within the next HORIZON steps?
+future_min = df["do"].shift(-1).rolling(HORIZON).min()
+df["will_crash"] = (future_min < CRITICAL).astype(int)
+
+frame = df.dropna()
+FEATURES = ["do", "do_slope", "do_std", "temp", "temp_slope", "hour"]
+```
+
+## 3. Train the classifier
+
+Crashes are rare, so the classes are imbalanced, weight them, and judge the model on
+precision/recall for the *crash* class, not raw accuracy.
+
+```python
+from sklearn.ensemble import HistGradientBoostingClassifier
+from sklearn.metrics import classification_report
+
+split = int(len(frame) * 0.8)
+train, test = frame.iloc[:split], frame.iloc[split:]
+
+# sample_weight up-weights the rare positive class
+sample_weight = np.where(train["will_crash"] == 1, 20.0, 1.0)
+clf = HistGradientBoostingClassifier(max_iter=300, learning_rate=0.05)
+clf.fit(train[FEATURES], train["will_crash"], sample_weight=sample_weight)
+
+print(classification_report(test["will_crash"], clf.predict(test[FEATURES])))
+```
+
+## 4. Score live and warn ahead of time
+
+Take the latest reading, predict the crash probability, and when it's high raise an
+`oxygen_crash_predicted` event, minutes before the [reactive low-oxygen
+alarm](/industries/agriculture-food/salmon-farming) would ever fire, so aeration starts in time.
+
+```python
+latest = frame.iloc[[-1]]
+prob = clf.predict_proba(latest[FEATURES])[0, 1]
+
+# publish the live risk as its own series
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="pen_h_07_crash_risk", name="Pen H-07 oxygen-crash risk", unit="score", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=[latest.index[-1]], values=[prob], ts="pen_h_07_crash_risk")
+
+if prob > 0.6:
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"oxygen_crash_predicted_h07_{int(pd.Timestamp.now().timestamp())}",
+ type="oxygen_crash_predicted", status="warning",
+ event_time=latest.index[-1],
+ metadata={"pen": "pen_h_07", "risk": f"{prob:.2f}", "lead_minutes": "45"})])
+
+# read the risk back
+pts = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts="pen_h_07_crash_risk",
+ start=latest.index[-1] - pd.Timedelta(minutes=1),
+ end=pd.Timestamp.now(tz="UTC")))[0].get_datapoints()
+print(f"{len(pts)} risk value(s) stored, latest {pts[-1].value:.2f}")
+assert len(pts) >= 1, "the risk score was not written"
+```
+
+The risk score is now a live series, and the event gives the farm a 45-minute head
+start, the difference between a near-miss and a lost pen.
+
+## Where to take it further
+
+- **Tune the lead time.** A longer `HORIZON` warns earlier but with more false alarms,
+ pick the trade-off the operation can act on.
+- **Add neighbours.** Pens sharing a water current crash together; add neighbouring pens'
+ oxygen as features, or correlate via [the graph](/guides/correlate-alarms).
+- **Close the loop.** Feed confirmed outcomes back as fresh labels and re-train on a
+ schedule so the model tracks the season.
+
+## Further reading
+
+- **Classification**: [Wikipedia](https://en.wikipedia.org/wiki/Statistical_classification)
+- **Gradient boosting**: [Wikipedia](https://en.wikipedia.org/wiki/Gradient_boosting)
+- **Precision & recall** (judging a rare-event model), [Wikipedia](https://en.wikipedia.org/wiki/Precision_and_recall)
+
+## See also
+
+- [Aquaculture, salmon farming](/industries/agriculture-food/salmon-farming): the reactive version this upgrades.
+- [Turn readings into events](/guides/detect-events): emitting the predictive warning.
+- [Query & aggregate](/guides/query-and-aggregate): assembling the training history.
diff --git a/versioned_docs/version-1.0/advanced/pca-process-monitoring.mdx b/versioned_docs/version-1.0/advanced/pca-process-monitoring.mdx
new file mode 100644
index 0000000..5830b98
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/pca-process-monitoring.mdx
@@ -0,0 +1,183 @@
+---
+sidebar_position: 11
+title: Process monitoring with PCA
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Multivariate process monitoring with PCA
+
+:::info At a glance
+**Effort:** ~1–1.5 hours · **You'll build:** a PCA model of normal operation plus the
+T² and Q monitoring statistics · **Stack:** the SDK for data in/out, plus `numpy`,
+`pandas` and `scikit-learn`.
+:::
+
+A process unit has dozens of sensors that all move together, when feed rises,
+temperatures, pressures and flows respond in a fixed, correlated dance. A fault breaks
+that dance: the sensors stop relating to each other the way they should, often *before*
+any single one crosses an alarm limit. **PCA** learns the normal correlation structure
+and reduces it to a few components; two statistics, **Hotelling's T²** and the **squared
+prediction error (Q / SPE)**, then flag when live data no longer fits, and a
+contribution check points at the sensor responsible.
+
+:::info Needs a sandbox
+These steps read correlated process sensors that already exist.
+[Generate a sandbox](/advanced/generate-sample-data) first, section C ingests them, fault and all.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Learn normal operation
+
+Pull a clean stretch of normal running across all the unit's sensors, standardise, and
+fit PCA keeping the components that capture the bulk of the variance.
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+from sklearn.preprocessing import StandardScaler
+from sklearn.decomposition import PCA
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+TAGS = ["cdu_1_top_temp_c", "cdu_1_bottom_temp_c", "cdu_1_pressure_kpa",
+ "cdu_1_reflux_ratio", "cdu_1_feed_bpd", "cdu_1_reboiler_duty_gj"]
+
+def series(tag, start, end):
+ rf = intellistream_datahub_sdk.RetrieveFilter(ts=tag, start=start, end=end,
+ aggregates=["avg"], granularity="1m", limit=100_000)
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pd.Series([float(p.average) for p in pts],
+ index=pd.to_datetime([p.timestamp for p in pts], utc=True)).sort_index()
+
+s = pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=14) # a clean stretch of normal running
+e = pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1) # ends before the recent fault window
+normal = pd.DataFrame({t: series(t, s, e) for t in TAGS}).dropna()
+
+scaler = StandardScaler().fit(normal)
+Z = scaler.transform(normal)
+pca = PCA(n_components=4).fit(Z) # keep 4 components
+```
+
+
+
+
+```java
+// Java pulls each tag the same way; the PCA below is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("cdu_1_top_temp_c");
+filter.setStart(ZonedDateTime.now().minusDays(14));
+filter.setEnd(ZonedDateTime.now().minusDays(1));
+filter.setAggregates(List.of("avg"));
+filter.setGranularity("1m");
+filter.setLimit(100000);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var pts = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust pulls each tag the same way; the PCA below is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("cdu_1_top_temp_c".into()),
+ start: Some(Utc::now() - chrono::Duration::days(14)),
+ end: Some(Utc::now() - chrono::Duration::days(1)),
+ aggregates: Some(vec!["avg".into()]),
+ granularity: Some("1m".into()),
+ limit: Some(100_000),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let pts = &page.get_items()[0];
+```
+
+
+
+
+## 2. Define the monitoring statistics and their limits
+
+**T²** measures how far a point sits inside the model's space (an unusual but in-pattern
+state); **Q/SPE** measures how far it sits *outside* the model (a broken correlation,
+usually the real fault). Set each limit from the normal data.
+
+```python
+lam = pca.explained_variance_
+
+def t2(z): # Hotelling's T² in component space
+ t = pca.transform(z)
+ return np.sum(t**2 / lam, axis=1)
+
+def spe(z): # squared prediction error (residual)
+ t = pca.transform(z)
+ resid = z - pca.inverse_transform(t)
+ return np.sum(resid**2, axis=1)
+
+T2_LIMIT = np.percentile(t2(Z), 99)
+SPE_LIMIT = np.percentile(spe(Z), 99)
+```
+
+## 3. Monitor live and name the culprit sensor
+
+Score the latest data; when either statistic exceeds its limit the process has moved off
+normal. The biggest term in the residual is the sensor most responsible, attach it to
+the event so operators know *where* to look.
+
+```python
+recent = pd.DataFrame({t: series(t, pd.Timestamp.now(tz="UTC") - pd.Timedelta(minutes=30),
+ pd.Timestamp.now(tz="UTC")) for t in TAGS}).dropna()
+znew = scaler.transform(recent)
+
+if t2(znew)[-1] > T2_LIMIT or spe(znew)[-1] > SPE_LIMIT:
+ z = znew[-1:]
+ residual = (z - pca.inverse_transform(pca.transform(z)))[0]
+ culprit = TAGS[int(np.argmax(residual**2))] # contribution analysis
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"process_deviation_cdu1_{int(pd.Timestamp.now().timestamp())}",
+ type="process_deviation", status="open",
+ event_time=recent.index[-1],
+ metadata={"unit": "crude_unit_1",
+ "t2": f"{t2(znew)[-1]:.1f}", "spe": f"{spe(znew)[-1]:.1f}",
+ "top_contributor": culprit})])
+
+# the seeded fault sits in the last ten hours, so the deviation must be on record
+found = list(client.events.filter(type="process_deviation", limit=10))
+assert found, "the seeded fault should have raised a process_deviation event"
+print(f"{len(found)} process deviation(s) on record, latest names {found[-1].metadata['top_contributor']}")
+```
+
+You can also publish T² and SPE as their own series for a live "process health" chart,
+two lines that capture the state of dozens of sensors at once.
+
+## Where to take it further
+
+- **Dimensionality first.** PCA is also a feature-reduction step *before* clustering or a
+ classifier, feed the components into [K-Means](/advanced/kmeans-clustering) instead of
+ raw tags.
+- **Dynamic PCA.** Include lagged sensor values so the model captures process dynamics,
+ not just instantaneous correlation.
+
+## Further reading
+
+- **Principal component analysis**: [Wikipedia](https://en.wikipedia.org/wiki/Principal_component_analysis)
+- **Hotelling's T²**: [Wikipedia](https://en.wikipedia.org/wiki/Hotelling%27s_T-squared_distribution)
+- **Multivariate process monitoring**: [Statistical process control](https://en.wikipedia.org/wiki/Statistical_process_control)
+
+## See also
+
+- [Oil & gas, refinery operations](/industries/oil-and-gas/refining): the unit this watches.
+- [Predictive maintenance](/advanced/predictive-maintenance) · [LSTM anomaly detection](/advanced/lstm-anomaly-detection), other ways to flag the abnormal.
+- [Turn readings into events](/guides/detect-events): emitting the deviation.
diff --git a/versioned_docs/version-1.0/advanced/predictive-maintenance.mdx b/versioned_docs/version-1.0/advanced/predictive-maintenance.mdx
new file mode 100644
index 0000000..990fbd2
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/predictive-maintenance.mdx
@@ -0,0 +1,240 @@
+---
+sidebar_position: 1
+title: Predictive maintenance (anomaly detection)
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Predictive maintenance, vibration anomaly detection
+
+:::info At a glance
+**Effort:** ~1–2 hours · **What you'll build:** a model that learns a machine's *healthy*
+vibration and warns you when it starts to drift · **Stack:** the SDK for data in and out,
+plus three Python libraries, `numpy`, `scipy`, `scikit-learn`.
+:::
+
+**The idea in one paragraph.** A failing bearing vibrates differently long before it
+breaks, but not *louder*, so a simple "is it above X?" alarm misses it. The difference
+is in the *texture* of the vibration: tiny repeated shocks, energy shifting into certain
+frequencies. We'll let a model look at lots of examples of the machine running *healthy*,
+learn what healthy texture looks like, and then raise a flag when new data stops looking
+like it. Nothing here needs labelled failures, only normal running.
+
+The [quick examples](/industries/transport-logistics/aerospace) raise an alarm on a raw threshold; this
+version learns normal first.
+
+:::info Needs a sandbox
+These steps read a vibration series that already exists. [Generate a sandbox](/advanced/generate-sample-data)
+first, section A ingests the degrading `pump_07_vibration_mm_s` this build scores, one reading
+a second.
+:::
+
+:::tip New to machine learning?
+No background needed. If a word here is unfamiliar, the
+[gentle primer](/advanced/machine-learning-gently) explains it in one line.
+:::
+
+## Step 1: Get a stretch of healthy data
+
+First we need examples of the machine running well. We pull a stretch of vibration readings
+from a period the machine was known to be healthy, and load them into a plain array of
+numbers we can do maths on. This data step is identical in every language:
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def load_series(external_id, start, end):
+ rf = intellistream_datahub_sdk.RetrieveFilter(ts=external_id, start=start, end=end, limit=100_000)
+ points = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ idx = pd.to_datetime([p.timestamp for p in points], utc=True)
+ return pd.Series([float(p.value) for p in points], index=idx).sort_index()
+
+healthy = load_series(
+ "pump_07_vibration_mm_s",
+ pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=24),
+ pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=6))
+```
+
+
+
+
+```java
+// Java extracts the same data; the learning step (Step 3) is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("pump_07_vibration_mm_s");
+filter.setStart(ZonedDateTime.now().minusHours(24));
+filter.setEnd(ZonedDateTime.now().minusHours(6));
+filter.setLimit(100000);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var points = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust extracts the same data; the learning step (Step 3) is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("pump_07_vibration_mm_s".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(24)),
+ end: Some(Utc::now() - chrono::Duration::hours(6)),
+ limit: Some(100_000),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+```
+
+
+
+
+## Step 2: Turn the raw wiggle into a few meaningful numbers
+
+A model can't learn much from one raw vibration value, it has no context. So we chop the
+signal into short **windows** (two minutes each, at the seed's one reading a second) and,
+for each window, calculate a few numbers that capture its *texture*. These numbers are called **features**. We use
+four, and here's what each one means in plain terms:
+
+- **RMS**: roughly the overall "loudness" or energy of the window.
+- **Kurtosis**: how *spiky* it is. A healthy machine hums smoothly; a failing bearing
+ adds sharp little shocks that push this number up.
+- **Crest factor**: the biggest peak compared to the average level; another way to catch
+ those shocks.
+- **Band energy**: how much of the vibration sits in a particular frequency range, found
+ with an [FFT](https://en.wikipedia.org/wiki/Fast_Fourier_transform) (a standard tool
+ that splits a wiggle into the pure tones it's made of). Bearing faults ring at
+ characteristic frequencies, so energy showing up there is a red flag.
+
+```python
+from scipy.stats import kurtosis
+from scipy.fft import rfft, rfftfreq
+
+SAMPLE_HZ = 1 # the seed writes one reading per second
+WINDOW = SAMPLE_HZ * 120 # work in 2-minute windows
+
+def features(window, fs=SAMPLE_HZ):
+ x = window - window.mean() # remove the average so we see the wobble
+ rms = np.sqrt(np.mean(x**2)) # overall energy
+ crest = np.max(np.abs(x)) / rms if rms else 0.0 # biggest peak vs. average
+ spectrum = np.abs(rfft(x)) # the FFT: how much of each frequency
+ freqs = rfftfreq(len(x), 1 / fs)
+ band = spectrum[(freqs >= 0.1) & (freqs <= 0.4)] # the fault band; the healthy tone sits below it
+ band_energy = float(np.sum(band**2))
+ return [rms, kurtosis(x), crest, band_energy] # four numbers describe this window
+
+def feature_matrix(series):
+ vals = series.to_numpy()
+ rows = [features(vals[i:i+WINDOW]) for i in range(0, len(vals) - WINDOW, WINDOW)]
+ return np.array(rows)
+
+X_healthy = feature_matrix(healthy) # one row of four features per window
+```
+
+At one reading a second the highest frequency a window can resolve is 0.5 Hz, so the fault
+band sits below that; a real 1 kHz sensor would put it at the bearing's fault frequencies
+instead. Each two-minute window is now just four numbers. Eighteen hours of healthy data
+become a table of 540 rows that says "this is what normal looks like."
+
+## Step 3: Let the model learn "normal"
+
+Now we hand that table to an **[Isolation Forest](https://en.wikipedia.org/wiki/Isolation_forest)**.
+The one-line idea: it learns the shape of the normal cloud of points and can then tell, for
+any new point, how *far outside* that cloud it sits. It needs no examples of failure, a
+big deal, because you rarely have many. (We also scale the features first, so that one
+feature with big numbers doesn't drown out the others, a routine tidy-up step.)
+
+```python
+from sklearn.ensemble import IsolationForest
+from sklearn.preprocessing import StandardScaler
+
+scaler = StandardScaler().fit(X_healthy) # remember each feature's scale
+model = IsolationForest(contamination=0.01, random_state=0) # 0.01 = expect ~1% odd points
+model.fit(scaler.transform(X_healthy)) # learn what healthy looks like
+
+def anomaly_score(X):
+ # the model returns "how normal"; we flip the sign so higher = more abnormal
+ return -model.decision_function(scaler.transform(X))
+
+# the alarm level comes from the healthy data itself: the score 99% of normal windows stay under
+ALARM = float(np.percentile(anomaly_score(X_healthy), 99))
+```
+
+## Step 4: Score new data, and write the answer back
+
+Finally we pull the *latest* hour, turn it into the same four features, and ask the model
+how abnormal each window is. We **write that score back to the platform as a new series**,
+so it's a normal, chartable signal, and if the score stays high (not just a one-off
+blip), we raise an event.
+
+```python
+recent = load_series("pump_07_vibration_mm_s",
+ pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=1),
+ pd.Timestamp.now(tz="UTC"))
+X_recent = feature_matrix(recent)
+scores = anomaly_score(X_recent)
+score_index = recent.index[WINDOW::WINDOW][:len(scores)] # timestamp each window's score
+
+try:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="pump_07_vibration_anomaly", name="Pump 07 vibration anomaly score", unit="score", value_type="float")])
+except intellistream_datahub_sdk.DataHubException as e:
+ if e.status_code != 409: # the seed's stand-in series may already be there
+ raise
+client.timeseries.insert_from_lists(
+ timestamps=score_index, values=scores, ts="pump_07_vibration_anomaly")
+
+if scores[-5:].mean() > ALARM: # sustained over the last ten minutes, not a single spike
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"degradation_predicted_pump_07_{int(pd.Timestamp.now().timestamp())}",
+ type="degradation_predicted", status="open",
+ event_time=score_index[-1],
+ metadata={"asset": "pump_07", "score": f"{scores[-1]:.3f}", "model": "iforest_v1"})])
+
+# read the scores back: the seed's fault ramps in over the last few hours, so the latest hour is abnormal
+stored = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts="pump_07_vibration_anomaly", start=score_index[0], end=pd.Timestamp.now(tz="UTC"),
+ limit=1000))[0].get_datapoints()
+print(f"{len(stored)} scores stored, latest {stored[-1].value:.3f} against an alarm level of {ALARM:.3f}")
+assert len(stored) >= len(scores), f"expected {len(scores)} scores, found {len(stored)}"
+assert scores[-5:].mean() > ALARM, "the seeded fault should score as abnormal"
+```
+
+That's the whole loop. The anomaly score is now a first-class series, chart it beside the
+raw vibration, [fold it into a health score](/advanced/asset-health-score), or alert on
+it like any other signal.
+
+## Where to take it further
+
+- **Name the failure.** Once you've collected a few real failures (labels), a
+ [classifier](/advanced/xgboost-failure-prediction) can predict the *type* of failure,
+ not just "something's off."
+- **Estimate time-to-failure.** Fit a line to the rising anomaly score and extrapolate to
+ the alarm level for a rough "days left."
+- **Cover the fleet.** When one machine fails, [walk the graph](/guides/correlate-alarms)
+ to every identical sibling and check them too.
+
+## Further reading
+
+- **Isolation Forest**: the anomaly detector used here:
+ [Wikipedia](https://en.wikipedia.org/wiki/Isolation_forest) ·
+ [scikit-learn guide](https://scikit-learn.org/stable/modules/outlier_detection.html#isolation-forest)
+- **FFT (frequency analysis)**: [Wikipedia](https://en.wikipedia.org/wiki/Fast_Fourier_transform)
+- **Kurtosis (spikiness)**: [Wikipedia](https://en.wikipedia.org/wiki/Kurtosis)
+- **New to the ideas?**: [Machine learning, gently](/advanced/machine-learning-gently)
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate): extracting the training windows.
+- [Asset health scoring](/advanced/asset-health-score): fold this score into a composite.
+- [Failure prediction with XGBoost](/advanced/xgboost-failure-prediction): the labelled, predict-the-type cousin.
diff --git a/versioned_docs/version-1.0/advanced/random-forest-soft-sensor.mdx b/versioned_docs/version-1.0/advanced/random-forest-soft-sensor.mdx
new file mode 100644
index 0000000..91f8301
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/random-forest-soft-sensor.mdx
@@ -0,0 +1,171 @@
+---
+sidebar_position: 10
+title: Soft sensor with Random Forest
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Soft sensor with a Random Forest
+
+:::info At a glance
+**Effort:** ~1 hour · **You'll build:** a regression model that infers a hard-to-measure
+quantity from cheap online sensors · **Stack:** the SDK for data in/out, plus `pandas`,
+`numpy` and `scikit-learn`.
+:::
+
+Some of the most important quantities are the hardest to measure. Product purity,
+produced-water oil content, melt viscosity, they come from a lab, hours apart, while
+the process runs continuously. A **soft sensor** closes that gap: a model that infers
+the lab value in real time from the cheap online sensors that *are* measured every
+second. Random Forest is a great fit, robust, little tuning, and an out-of-bag score
+that estimates accuracy without a separate test set.
+
+:::info Needs a sandbox
+These steps read process sensors and sparse lab samples that already exist.
+[Generate a sandbox](/advanced/generate-sample-data) first, section D ingests both.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Assemble lab targets against sensor features
+
+The training signal is the infrequent lab samples (the target) paired with the online
+sensor readings *at the moment each sample was taken* (the features).
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def series(external_id, start, end, granularity="5m"):
+ rf = intellistream_datahub_sdk.RetrieveFilter(ts=external_id, start=start, end=end,
+ aggregates=["avg"], granularity=granularity, limit=100_000)
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pd.Series([float(p.average) for p in pts],
+ index=pd.to_datetime([p.timestamp for p in pts], utc=True)).sort_index()
+
+start, end = pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=14), pd.Timestamp.now(tz="UTC")
+SENSORS = ["cdu_1_top_temp_c", "cdu_1_reflux_ratio", "cdu_1_feed_bpd", "cdu_1_pressure_kpa"]
+features = pd.DataFrame({s: series(s, start, end) for s in SENSORS})
+
+# lab samples are sparse — align each to the nearest sensor row
+lab = series("cdu_1_product_sulfur_ppm_lab", start, end, granularity="1h").dropna()
+train = features.reindex(lab.index, method="nearest").join(lab.rename("y")).dropna()
+```
+
+
+
+
+```java
+// Java pulls the sensor and lab series the same way; the model below is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("cdu_1_top_temp_c");
+filter.setStart(ZonedDateTime.now().minusDays(14));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg"));
+filter.setGranularity("5m");
+filter.setLimit(100000);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var pts = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust pulls the sensor and lab series the same way; the model below is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("cdu_1_top_temp_c".into()),
+ start: Some(Utc::now() - chrono::Duration::days(14)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into()]),
+ granularity: Some("5m".into()),
+ limit: Some(100_000),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let pts = &page.get_items()[0];
+```
+
+
+
+
+## 2. Train the regressor
+
+A Random Forest needs little tuning, and `oob_score` gives an honest accuracy estimate
+from the trees' out-of-bag samples, handy when labels are scarce.
+
+```python
+from sklearn.ensemble import RandomForestRegressor
+
+rf = RandomForestRegressor(n_estimators=400, max_depth=None,
+ oob_score=True, n_jobs=-1, random_state=0)
+rf.fit(train[SENSORS], train["y"])
+print(f"out-of-bag R²: {rf.oob_score_:.3f}")
+
+# which sensors drive the quality?
+for name, imp in sorted(zip(SENSORS, rf.feature_importances_),
+ key=lambda kv: kv[1], reverse=True):
+ print(f" {name}: {imp:.2f}")
+```
+
+## 3. Infer continuously and publish the virtual sensor
+
+Run the model on the live sensor rows and write the prediction back as its own
+series, a continuous estimate of the lab value, updated every few minutes instead of
+once a shift. Now it can be charted, alerted on, and fed into control like any real tag.
+
+```python
+recent = pd.DataFrame({s: series(s,
+ pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=6),
+ pd.Timestamp.now(tz="UTC")) for s in SENSORS}).dropna()
+virtual = rf.predict(recent[SENSORS])
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="cdu_1_product_sulfur_ppm_soft", name="CDU-1 product sulfur (soft sensor)",
+ unit="ppm", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=recent.index, values=virtual, ts="cdu_1_product_sulfur_ppm_soft")
+
+# read the virtual sensor back: one point per sensor row
+stored = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts="cdu_1_product_sulfur_ppm_soft",
+ start=recent.index[0], end=recent.index[-1] + pd.Timedelta(minutes=1),
+ limit=1000))[0].get_datapoints()
+print(f"{len(stored)} soft-sensor values stored, latest {stored[-1].value:.2f} ppm")
+assert len(stored) == len(virtual), f"expected {len(virtual)}, got {len(stored)}"
+```
+
+When the next lab sample lands, append it to the training set and re-fit, the soft
+sensor stays calibrated as the process and feedstock change.
+
+## Where to take it further
+
+- **Quantify uncertainty.** The spread across the forest's trees gives a per-prediction
+ confidence, widen alerts when the model is unsure.
+- **Drift watch.** Compare each new lab sample to the soft sensor's prediction; a growing
+ gap is a [retrain trigger](/guides/detect-events).
+
+## Further reading
+
+- **Random forest**: [Wikipedia](https://en.wikipedia.org/wiki/Random_forest)
+- **Soft sensor**: [Wikipedia](https://en.wikipedia.org/wiki/Soft_sensor)
+- **Out-of-bag error**: [Wikipedia](https://en.wikipedia.org/wiki/Out-of-bag_error)
+
+## See also
+
+- [Oil & gas, refinery operations](/industries/oil-and-gas/refining): where soft sensors earn their keep.
+- [XGBoost failure prediction](/advanced/xgboost-failure-prediction): the boosted-tree cousin.
+- [Query & aggregate](/guides/query-and-aggregate): assembling the feature history.
diff --git a/versioned_docs/version-1.0/advanced/sustained-alarm-window.mdx b/versioned_docs/version-1.0/advanced/sustained-alarm-window.mdx
new file mode 100644
index 0000000..a4656b2
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/sustained-alarm-window.mdx
@@ -0,0 +1,531 @@
+---
+sidebar_position: 12
+title: Sustained-condition alarms (sliding window)
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Sustained-condition alarms (sliding window)
+
+A limit on a single reading fires on every transient. A pump's discharge pressure spikes on
+every start-up and every slug of gas, the alarm fires four hundred times a month, and
+somebody switches it off; on the one occasion it mattered, nobody was watching.
+
+The question worth acting on is not *is it above 4.5* but **has it stayed above 4.5**, and
+that is a question about a stretch of time. This page builds that rule end to end: seed a
+signal, replay it to prove the rule behaves, then run the same code against a live
+subscription.
+
+:::info Needs a sandbox
+Run generator **L** on [Generate sample data](/advanced/generate-sample-data#l-a-noisy-alarm-four-transients-and-one-real-excursion).
+It writes `pump_p101_discharge_bar`: six hours at one reading a minute, four brief spikes
+well above the limit, and one eighty-minute excursion just above it. Seeding is Python, as
+it is for every scenario here; the rule itself is below in all three languages.
+:::
+
+## 1. The rule, in four numbers
+
+| Setting | Value | Why |
+| --- | --- | --- |
+| `LIMIT` | 4.5 bar | The existing alarm limit. Unchanged, deliberately |
+| `WINDOW` | 20 minutes | Longer than any transient worth ignoring, shorter than the time you need to act |
+| `CLEAR_BELOW` | 4.2 bar | Hysteresis. Raise and clear thresholds must differ or the rule flaps at the boundary |
+| `MIN_READINGS` | 15 | A window with holes proves nothing. At one reading a minute, twenty minutes should bring about twenty |
+
+## 2. The detector
+
+One object, holding the last `WINDOW` of readings and whether a finding is currently open.
+Nothing in it knows where the readings came from, which is what lets the same code run over
+history and over a live stream.
+
+
+
+
+```java
+import java.time.*;
+import java.util.*;
+
+record Reading(Instant at, double value) {}
+
+class SustainedHigh {
+ static final double LIMIT = 4.5, CLEAR_BELOW = 4.2;
+ static final int MIN_READINGS = 15;
+ static final Duration WINDOW = Duration.ofMinutes(20);
+
+ private final Deque recent = new ArrayDeque<>();
+ private final String seriesId;
+ private boolean open = false;
+
+ SustainedHigh(String seriesId) { this.seriesId = seriesId; }
+
+ EventModel push(Instant at, double value) {
+ if (recent.stream().anyMatch(r -> r.at().equals(at))) {
+ return null; // a redelivery, not a new reading
+ }
+ recent.addLast(new Reading(at, value));
+ Instant newest = recent.peekLast().at();
+ while (recent.peekFirst().at().isBefore(newest.minus(WINDOW))) {
+ recent.removeFirst(); // evict on carried time, not the clock
+ }
+ return evaluate();
+ }
+
+ private EventModel evaluate() {
+ if (recent.size() < MIN_READINGS) return null;
+ Duration span = Duration.between(
+ recent.peekFirst().at(), recent.peekLast().at());
+ if (span.toMillis() < WINDOW.toMillis() * 0.9) return null; // not full
+ if (!open && recent.stream().allMatch(r -> r.value() > LIMIT)) {
+ open = true;
+ return raiseEvent();
+ }
+ if (open && recent.stream().allMatch(r -> r.value() < CLEAR_BELOW)) {
+ open = false; // clear only once it is properly back
+ }
+ return null;
+ }
+
+ private EventModel raiseEvent() {
+ Instant at = recent.peekLast().at();
+ double peak = recent.stream()
+ .mapToDouble(Reading::value).max().orElseThrow();
+
+ EventModel event = new EventModel();
+ event.setExternalId(
+ "sustained_high_" + seriesId + "_" + at.getEpochSecond());
+ event.setType("sustained_high");
+ event.setStatus("open");
+ event.setEventTime(ZonedDateTime.ofInstant(at, ZoneOffset.UTC));
+ event.setMetadata(Map.of(
+ "series", seriesId, "limit", String.valueOf(LIMIT),
+ "window_minutes", "20", "peak", String.format("%.2f", peak)));
+ client.events().create(List.of(event));
+ return event;
+ }
+}
+```
+
+
+
+
+```python
+from collections import deque
+from datetime import timedelta
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+LIMIT, CLEAR_BELOW, MIN_READINGS = 4.5, 4.2, 15
+WINDOW = timedelta(minutes=20)
+
+class SustainedHigh:
+ def __init__(self, series_id):
+ self.series_id, self.recent, self.open = series_id, deque(), False
+
+ def push(self, ts, value):
+ if any(t == ts for t, _ in self.recent):
+ return None # a redelivery, not a new reading
+ self.recent.append((ts, value))
+ while self.recent and self.recent[0][0] < self.recent[-1][0] - WINDOW:
+ self.recent.popleft() # evict on carried time, not the clock
+ return self.evaluate()
+
+ def evaluate(self):
+ if len(self.recent) < MIN_READINGS:
+ return None # not enough of the window to judge
+ span = self.recent[-1][0] - self.recent[0][0]
+ if span < WINDOW * 0.9:
+ return None # the window has not filled yet
+ if not self.open and all(v > LIMIT for _, v in self.recent):
+ self.open = True
+ return self.raise_event()
+ if self.open and all(v < CLEAR_BELOW for _, v in self.recent):
+ self.open = False # clear only once it is properly back
+ return None
+
+ def raise_event(self):
+ at, peak = self.recent[-1][0], max(v for _, v in self.recent)
+ event = intellistream_datahub_sdk.Event(
+ external_id=f"sustained_high_{self.series_id}_{int(at.timestamp())}",
+ type="sustained_high",
+ status="open",
+ event_time=at,
+ metadata={"series": self.series_id, "limit": str(LIMIT),
+ "window_minutes": "20", "peak": f"{peak:.2f}"})
+ client.events.create([event])
+ return event
+```
+
+
+
+
+```rust
+use chrono::{DateTime, Duration, Utc};
+use intellistream_datahub_sdk::events::Event;
+use std::collections::VecDeque;
+use intellistream_datahub_sdk::ApiService;
+
+const LIMIT: f64 = 4.5;
+const CLEAR_BELOW: f64 = 4.2;
+const MIN_READINGS: usize = 15;
+
+fn window() -> Duration { Duration::minutes(20) }
+
+struct SustainedHigh {
+ series_id: String,
+ recent: VecDeque<(DateTime, f64)>,
+ open: bool,
+}
+
+impl SustainedHigh {
+ fn new(series_id: &str) -> Self {
+ Self { series_id: series_id.into(), recent: VecDeque::new(), open: false }
+ }
+
+ async fn push(&mut self, api: &ApiService, at: DateTime, value: f64)
+ -> Option {
+ if self.recent.iter().any(|(t, _)| *t == at) {
+ return None; // a redelivery, not a new reading
+ }
+ self.recent.push_back((at, value));
+ let newest = self.recent.back()?.0;
+ while self.recent.front().map_or(false, |(t, _)| *t < newest - window()) {
+ self.recent.pop_front(); // evict on carried time, not the clock
+ }
+ self.evaluate(api).await
+ }
+
+ async fn evaluate(&mut self, api: &ApiService) -> Option {
+ if self.recent.len() < MIN_READINGS { return None; }
+ let span = self.recent.back()?.0 - self.recent.front()?.0;
+ if span < window() * 9 / 10 { return None; } // not full yet
+ if !self.open && self.recent.iter().all(|(_, v)| *v > LIMIT) {
+ self.open = true;
+ return self.raise_event(api).await;
+ }
+ if self.open && self.recent.iter().all(|(_, v)| *v < CLEAR_BELOW) {
+ self.open = false; // clear only once it is properly back
+ }
+ None
+ }
+
+ async fn raise_event(&self, api: &ApiService) -> Option {
+ let at = self.recent.back()?.0;
+ let peak = self.recent.iter().map(|(_, v)| *v).fold(f64::MIN, f64::max);
+
+ let mut event = Event::new(
+ format!("sustained_high_{}_{}", self.series_id, at.timestamp()),
+ "sustained_high".into(),
+ at);
+ event.status = Some("open".into());
+ event.add_metadata("series".into(), self.series_id.clone());
+ event.add_metadata("limit".into(), LIMIT.to_string());
+ event.add_metadata("window_minutes".into(), "20".into());
+ event.add_metadata("peak".into(), format!("{peak:.2}"));
+ api.events.create(&vec![event.clone()]).await.ok()?;
+ Some(event)
+ }
+}
+```
+
+
+
+
+The "window not filled" check is worth a second look. The buffer only ever holds readings
+within `WINDOW` of the newest one, so its span can never exceed the window; what it *can* be
+is short, either because the process has only just started or because readings stopped
+arriving. Comparing against most of a window rather than exactly one keeps the rule working
+at any sampling rate.
+
+## 3. Replay the history, and see what fires
+
+The seeded six hours are the test. Pull them back and push each reading through the detector
+in order. A datapoint carries its timestamp and value as strings, so both are parsed on the
+way in.
+
+
+
+
+```java
+final String SERIES = "pump_p101_discharge_bar";
+
+RetrieveFilter filter = new RetrieveFilter();
+filter.setExternalId(SERIES);
+filter.setStart(ZonedDateTime.now().minusHours(7));
+filter.setEnd(ZonedDateTime.now());
+filter.setLimit(10000);
+
+DataRetriever request = new DataRetriever<>();
+request.setItems(List.of(filter));
+var points = client.timeseries().retrieve(request)
+ .getItems().get(0).getDatapoints();
+
+SustainedHigh detector = new SustainedHigh(SERIES);
+final double LIMIT = SustainedHigh.LIMIT;
+int raised = 0, crossings = 0;
+double previous = 0.0;
+for (var p : points) {
+ Instant at = Instant.ofEpochMilli(Long.parseLong(p.getTimestamp()));
+ double value = Double.parseDouble(p.getValue());
+ if (previous <= LIMIT && value > LIMIT) crossings++;
+ previous = value;
+ if (detector.push(at, value) != null) raised++;
+}
+System.out.printf("%d readings, %d crossings, %d event(s) raised%n",
+ points.size(), crossings, raised);
+// 360 readings, 5 crossings, 1 event(s) raised
+```
+
+
+
+
+```python
+import pandas as pd
+
+SERIES = "pump_p101_discharge_bar"
+
+points = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts=SERIES,
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=7),
+ end=pd.Timestamp.now(tz="UTC")))[0].get_datapoints()
+
+readings = [(dp.timestamp, float(dp.value)) for dp in points]
+crossings = sum(1 for a, b in zip(readings, readings[1:])
+ if a[1] <= LIMIT < b[1]) # what a plain alarm would do
+
+detector = SustainedHigh(SERIES)
+raised = [e for ts, v in readings if (e := detector.push(ts, v))]
+
+print(f"{len(readings)} readings, {crossings} crossings, "
+ f"{len(raised)} event(s) raised")
+# 360 readings, 5 crossings, 1 event(s) raised
+```
+
+
+
+
+```rust
+use chrono::TimeZone;
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+
+const SERIES: &str = "pump_p101_discharge_bar";
+
+let filter = RetrieveFilter {
+ external_id: Some(SERIES.into()),
+ start: Some(Utc::now() - Duration::hours(7)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+
+let mut detector = SustainedHigh::new(SERIES);
+let (mut raised, mut crossings, mut previous) = (0, 0, 0.0_f64);
+for p in &series.datapoints {
+ let at = p.timestamp;
+ let value = p.value.unwrap_or(0.0);
+ if previous <= LIMIT && value > LIMIT { crossings += 1; }
+ previous = value;
+ if detector.push(&api, at, value).await.is_some() { raised += 1; }
+}
+println!("{} readings, {} crossings, {} event(s) raised",
+ series.datapoints.len(), crossings, raised);
+// 360 readings, 5 crossings, 1 event(s) raised
+```
+
+
+
+
+:::tip What "working" looks like
+**Five crossings, one event.** The four transients each cross the limit and each would have
+fired the old alarm; none of them survives twenty minutes, so none of them raises anything
+here. The excursion does, twenty minutes after it began, which is the price the window
+charges and the reason it is worth paying.
+:::
+
+## 4. Verify it landed
+
+The detector writes real events, so the check is a query rather than a print.
+
+
+
+
+```java
+EventRetreiver retriever = new EventRetreiver();
+retriever.setLimit(100);
+retriever.getFilter().setType(List.of("sustained_high"));
+
+DataWrapper found = client.events().filter(retriever);
+found.getItems().forEach(e ->
+ System.out.println(e.getExternalId() + " " + e.getMetadata()));
+
+if (found.getItems().size() != 1) {
+ throw new AssertionError(
+ "expected one finding, found " + found.getItems().size());
+}
+```
+
+
+
+
+```python
+found = list(client.events.filter(type="sustained_high", limit=100))
+
+for e in found:
+ print(e.external_id, e.metadata)
+# sustained_high_pump_p101_discharge_bar_… {'series': 'pump_p101_discharge_bar',
+# 'limit': '4.5', 'window_minutes': '20', 'peak': '5.24'}
+
+assert len(found) == 1, f"expected one finding, found {len(found)}"
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::{EventFilter, EventFilterForm};
+
+let filter = EventFilterForm::default()
+ .set_filter(EventFilter {
+ r#type: Some(vec!["sustained_high".into()]), ..Default::default() })
+ .set_limit(100)
+ .build();
+
+let found = api.events.filter(&filter).await?;
+assert_eq!(found.get_items().len(), 1, "expected exactly one finding");
+```
+
+
+
+
+Re-running the replay is instructive: the detector raises the same finding again with the
+same external id, and the platform stores it as a second event rather than collapsing it, so
+the check above then finds two. An event's external id is not unique; each create without an
+`id` gets a new one. Only the datapoints collapse, keyed by series and timestamp.
+
+## 5. Run the same detector live
+
+Swap the source. Nothing in the detector changes: `push` is fed by a
+[subscription](/guides/realtime-subscriptions) instead of by a list, and each message is
+acknowledged **after** it has been handled, so a crash replays it rather than losing it.
+
+`datapoints(...)` below is your own unpacking of the delivered payload, which carries the
+action and the datapoints it affected; the SDK hands it to you rather than deciding what
+your rule wants from it.
+
+
+
+
+```java
+Subscription sub = new Subscription();
+sub.setExternalId("pump_p101");
+sub.setName("Pump P-101");
+sub.setTimeseries(List.of(IdCollection.createFromExternalId(SERIES)));
+client.subscriptions().create(List.of(sub));
+
+SustainedHigh detector = new SustainedHigh(SERIES);
+
+try (SubscriptionListener listener =
+ client.subscriptions().listen(List.of("pump_p101"))) {
+ while (running) {
+ SubscriptionMessage msg = listener.poll(Duration.ofSeconds(5));
+ if (msg == null) continue;
+ for (Reading r : datapoints(msg.payload())) {
+ EventModel event = detector.push(r.at(), r.value());
+ if (event != null) {
+ System.out.println("raised " + event.getExternalId());
+ }
+ }
+ listener.ack(msg.messageId()); // only once it is durably handled
+ }
+}
+```
+
+
+
+
+```python
+sub = intellistream_datahub_sdk.Subscription(
+ external_id="pump_p101", name="Pump P-101", timeseries=[SERIES])
+client.subscriptions.create([sub])
+
+detector = SustainedHigh(SERIES)
+
+with client.subscriptions.listen(["pump_p101"]) as listener:
+ for msg in listener:
+ for ts, value in datapoints(msg.payload):
+ if event := detector.push(ts, value):
+ print("raised", event.external_id)
+ listener.ack([msg.message_id]) # only once it is durably handled
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+use intellistream_datahub_sdk::subscriptions::Subscription;
+
+let sub = Subscription::new(
+ "pump_p101".into(), "Pump P-101".into(),
+ vec![IdAndExtId::from_external_id(SERIES)]);
+api.subscriptions.create(&sub).await?;
+
+let mut detector = SustainedHigh::new(SERIES);
+let mut listener = api.subscriptions.listen(&["pump_p101"]).await?;
+
+while let Some(result) = listener.next().await {
+ let msg = match result {
+ Ok(m) => m,
+ Err(e) => { eprintln!("listen: {e}"); continue }
+ };
+ for (at, value) in datapoints(&msg.payload) {
+ if detector.push(&api, at, value).await.is_some() {
+ println!("raised a sustained_high finding");
+ }
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?; // only once handled
+}
+```
+
+
+
+
+Two habits make the difference between this working on a desk and working on a plant:
+
+- **Fill the window before trusting it.** On start-up, replay the last twenty minutes
+ through `push` exactly as section 3 does. Without it the rule is blind for its first
+ twenty minutes, which is precisely when somebody has restarted it during an incident.
+- **Treat a thin window as a finding of its own.** `MIN_READINGS` makes the detector stay
+ quiet when readings stop arriving, which is right, but silence is not the same as health.
+ Raise a separate, quieter event about the instrument or the link.
+
+## 6. The same shape, other rules
+
+Only the evaluate step changes:
+
+| Question | The change |
+| --- | --- |
+| Sustained *below* a limit | Invert the comparisons |
+| Rate of change over the window | Compare first and last, divide by the span |
+| A ratio between two series drifting | One detector fed by two series, keyed on the nearest timestamps |
+| Above the limit for most of the window, not all of it | Count instead of "all", and raise on a fraction |
+
+When no threshold over any window can express what you are looking for, because the
+readings never leave their limits and it is the shape that changed, that is where
+[LSTM anomaly detection](/advanced/lstm-anomaly-detection) starts earning its keep.
+
+## Further reading
+
+- **Stream processing**: [Wikipedia](https://en.wikipedia.org/wiki/Stream_processing)
+- **Hysteresis**: [Wikipedia](https://en.wikipedia.org/wiki/Hysteresis)
+- **Idempotence**: [Wikipedia](https://en.wikipedia.org/wiki/Idempotence)
+- Stream processing, in the platform documentation, windows, event time against arrival time, and what a computation has to remember
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions): the delivery mechanics this builds on.
+- [Turn readings into events](/guides/detect-events): the single-reading version of the same job.
+- [Generate sample data](/advanced/generate-sample-data#l-a-noisy-alarm-four-transients-and-one-real-excursion): the signal this page runs against.
+- [LSTM anomaly detection](/advanced/lstm-anomaly-detection): for the excursions no limit can describe.
diff --git a/versioned_docs/version-1.0/advanced/xgboost-failure-prediction.mdx b/versioned_docs/version-1.0/advanced/xgboost-failure-prediction.mdx
new file mode 100644
index 0000000..aad6e6a
--- /dev/null
+++ b/versioned_docs/version-1.0/advanced/xgboost-failure-prediction.mdx
@@ -0,0 +1,206 @@
+---
+sidebar_position: 9
+title: Failure prediction with XGBoost
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Failure prediction with XGBoost
+
+:::info At a glance
+**Effort:** ~1–2 hours · **You'll build:** an engineered tabular feature set and a
+gradient-boosted classifier with feature importance · **Stack:** the SDK for data in/out,
+plus `pandas`, `numpy` and `xgboost`.
+:::
+
+Some questions are best answered from **tabular features**, not raw sequences: *will
+this asset fail in the next week?* XGBoost is the workhorse here, it eats engineered
+features, handles missing values natively, trains fast, and tells you which features
+drive the prediction. We'll predict an electric submersible pump (ESP) failure seven
+days out, but the recipe fits any "label the history, predict the window" problem.
+
+:::info Needs a sandbox
+These steps read series and failure events that already exist. [Generate a sandbox](/advanced/generate-sample-data)
+first, section E ingests the labelled failure history this build trains on.
+:::
+
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
+## 1. Engineer one feature row per asset per day
+
+For each pump on each day, summarise the recent week of its sensors into a feature row.
+Gaps stay as `NaN`, XGBoost handles missing values without imputation.
+
+
+
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def feature_row(pump, day):
+ def agg(metric, *aggs):
+ rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts=f"{pump}_{metric}", start=day - pd.Timedelta(days=7), end=day,
+ aggregates=list(aggs), granularity="7d")
+ pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+ return pts[-1] if pts else None
+ ip, mt, vb = agg("intake_pressure_bar", "avg", "min"), \
+ agg("motor_temp_c", "avg", "max"), \
+ agg("vibration_mm_s", "avg", "max")
+ return {
+ "intake_avg": float(ip.average) if ip else np.nan,
+ "intake_min": float(ip.min) if ip else np.nan,
+ "motor_avg": float(mt.average) if mt else np.nan,
+ "motor_max": float(mt.max) if mt else np.nan,
+ "vib_avg": float(vb.average) if vb else np.nan,
+ "vib_max": float(vb.max) if vb else np.nan,
+ }
+```
+
+
+
+
+```java
+// Java pulls each pump's weekly aggregates; the model below is Python.
+var filter = new RetrieveFilter();
+filter.setExternalId("pump_esp_a12_motor_temp_c");
+filter.setStart(ZonedDateTime.now().minusDays(7));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg", "max"));
+filter.setGranularity("7d");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+var agg = client.timeseries().retrieve(request).getItems().get(0).getDatapoints();
+```
+
+
+
+
+```rust
+// Rust pulls each pump's weekly aggregates; the model below is Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("pump_esp_a12_motor_temp_c".into()),
+ start: Some(Utc::now() - chrono::Duration::days(7)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into(), "max".into()]),
+ granularity: Some("7d".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let agg = &page.get_items()[0];
+```
+
+
+
+
+## 2. Label from the failure history
+
+A row is positive if the pump failed within the next 7 days. Pull historical failure
+events and label each `(pump, day)` accordingly.
+
+```python
+# failures: DataFrame of (pump, failed_on), pulled from /events (type = esp_failure)
+failures = pd.DataFrame([
+ {"pump": e.metadata.get("pump"), "failed_on": pd.to_datetime(e.event_time, utc=True)}
+ for e in client.events.filter(type="esp_failure", limit=1000)])
+
+# one feature row per pump per day across the history you have
+training_grid = [(p, d) for p in ["pump_esp_a12"]
+ for d in pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=260, freq="1d")]
+
+def label(pump, day, failures):
+ upcoming = failures[(failures["pump"] == pump) &
+ (failures["failed_on"] > day) &
+ (failures["failed_on"] <= day + pd.Timedelta(days=7))]
+ return int(len(upcoming) > 0)
+
+rows = [(feature_row(p, d), label(p, d, failures)) for p, d in training_grid]
+X = pd.DataFrame([r for r, _ in rows])
+y = np.array([lab for _, lab in rows])
+```
+
+## 3. Train, with class imbalance and early stopping
+
+Failures are rare, so weight the positive class and optimise for ranking (`aucpr`).
+Early stopping on a validation split avoids over-fitting.
+
+```python
+from xgboost import XGBClassifier
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import average_precision_score
+
+Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)
+
+clf = XGBClassifier(
+ n_estimators=600, max_depth=5, learning_rate=0.03,
+ subsample=0.8, colsample_bytree=0.8,
+ eval_metric="aucpr", early_stopping_rounds=40,
+ scale_pos_weight=(y == 0).sum() / max((y == 1).sum(), 1))
+clf.fit(Xtr, ytr, eval_set=[(Xte, yte)], verbose=False)
+
+print(f"avg precision: {average_precision_score(yte, clf.predict_proba(Xte)[:, 1]):.3f}")
+```
+
+## 4. Score live, with the reasons attached
+
+Score today's row per pump, write the risk back as a series, and raise a
+`failure_predicted` event for high-risk pumps, including the **feature importances** so
+the planner sees *why* (rising vibration vs. falling intake pressure point to different
+fixes).
+
+```python
+import_ = dict(sorted(zip(X.columns, clf.feature_importances_),
+ key=lambda kv: kv[1], reverse=True)[:3])
+
+today = pd.DataFrame([feature_row("pump_esp_a12", pd.Timestamp.now(tz="UTC"))])
+risk = float(clf.predict_proba(today)[0, 1])
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="pump_esp_a12_failure_risk", name="ESP A-12 7-day failure risk", unit="score", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=[pd.Timestamp.now(tz="UTC")], values=[risk], ts="pump_esp_a12_failure_risk")
+
+if risk > 0.6:
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"failure_predicted_a12_{int(pd.Timestamp.now().timestamp())}",
+ type="failure_predicted", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"pump": "pump_esp_a12", "risk": f"{risk:.2f}",
+ "top_drivers": ", ".join(import_)})])
+
+# read the risk back
+stored = client.timeseries.retrieve_latest_datapoints(["pump_esp_a12_failure_risk"])[0].get_datapoints()
+print(f"7-day failure risk {stored[-1].value:.2f}")
+assert abs(stored[-1].value - risk) < 0.01, "the risk was not written"
+```
+
+## Where to take it further
+
+- **Real explainability.** Use SHAP for per-prediction attributions, not just global
+ importance, the planner sees what drove *this* pump's score.
+- **Monotonic constraints.** Tell XGBoost that higher vibration only ever raises risk
+ (`monotone_constraints`) for a model that matches engineering intuition.
+- **Compare.** Benchmark against [Random Forest](/advanced/random-forest-soft-sensor) and
+ the sklearn gradient booster in [fraud classification](/advanced/fraud-classification).
+
+## Further reading
+
+- **XGBoost**: [Wikipedia](https://en.wikipedia.org/wiki/XGBoost) · [docs](https://xgboost.readthedocs.io/)
+- **Gradient boosting**: [Wikipedia](https://en.wikipedia.org/wiki/Gradient_boosting)
+- **Feature importance**: [scikit-learn guide](https://scikit-learn.org/stable/modules/permutation_importance.html)
+
+## See also
+
+- [Oil & gas, production monitoring](/industries/oil-and-gas/production): the ESPs this protects.
+- [Predictive maintenance](/advanced/predictive-maintenance): the unsupervised counterpart.
+- [Turn readings into events](/guides/detect-events): emitting the prediction.
diff --git a/versioned_docs/version-1.0/guides/_category_.json b/versioned_docs/version-1.0/guides/_category_.json
new file mode 100644
index 0000000..0cd5757
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/_category_.json
@@ -0,0 +1,9 @@
+{
+ "label": "Examples",
+ "position": 5,
+ "link": {
+ "type": "generated-index",
+ "title": "Examples",
+ "description": "Task-focused recipes for the things you'll do most: ingest at volume, model assets as a graph, query and aggregate, consume live data, and turn readings into events. For domain walkthroughs that combine these, see the Industry scenarios."
+ }
+}
diff --git a/versioned_docs/version-1.0/guides/attach-files.mdx b/versioned_docs/version-1.0/guides/attach-files.mdx
new file mode 100644
index 0000000..1a7216c
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/attach-files.mdx
@@ -0,0 +1,199 @@
+---
+sidebar_position: 6
+title: Attach files to assets
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Attach files to assets
+
+Not everything is a number. A pump has a calibration certificate, a plant has a
+piping diagram, a daily run produces a CSV export. Store these as **files**,
+organized in folders, tagged with metadata, and linked to the data set they belong to.
+
+## Upload a file
+
+Give the file a destination path and some metadata. The server stores one full path, name
+included; the Java client sends the `path` you give it as-is, and the Python and Rust
+clients join `destination_path` and `name` into the same thing. The Java client uploads raw
+bytes; the Python and Rust clients read a local file.
+
+
+
+
+```java
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+byte[] content = Files.readAllBytes(Path.of("calibration_a12.pdf"));
+
+var uploaded = client.files().upload(
+ FileUploadRequest.builder()
+ .path("/certificates/2026/calibration_a12.pdf") // destination, name included
+ .content(content)
+ .contentType("application/pdf")
+ .externalId("calibration_pump_esp_a12")
+ .dataSetId(42L)
+ .description("ESP A-12 calibration certificate")
+ .build());
+
+System.out.println("stored as id " + uploaded.getItems().get(0).getId());
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+uploaded = client.files.upload_file(intellistream_datahub_sdk.FileUpload(
+ path="calibration_a12.pdf", # local file
+ destination_path="/certificates/2026/",
+ external_id="calibration_pump_esp_a12",
+ name="calibration_a12.pdf",
+ data_set_id=42,
+ description="ESP A-12 calibration certificate"))
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::files::FileUpload;
+
+// MIME type is inferred from the file content
+let mut upload = FileUpload::new_with_destination_path(
+ "calibration_a12.pdf", "/certificates/2026/")?;
+upload.set_external_id("calibration_pump_esp_a12".into());
+upload.set_file_name("calibration_a12.pdf".into());
+upload.set_data_set_id(42);
+upload.set_description("ESP A-12 calibration certificate".into());
+let uploaded = api.files.upload_file(upload).await?;
+```
+
+
+
+
+:::tip Link a file to what it documents
+Set `dataSetId` so a file is discoverable from the data set it documents. To point it at the
+resource it documents, the calibration certificate at the pump it certifies, the Python and
+Rust clients take `related_resources`, a list of numeric resource ids; the Java client has no
+field for it.
+:::
+
+## List a directory
+
+Browse the folder tree. Each entry is a file or a folder (`type`), with its name,
+path and size.
+
+
+
+
+```java
+var listing = client.files().list("/certificates/2026");
+listing.getItems().forEach(node ->
+ System.out.println(node.getType() + " " + node.getName() + " " + node.getSize() + " bytes"));
+
+boolean present = listing.getItems().stream()
+ .anyMatch(node -> "calibration_a12.pdf".equals(node.getName()));
+if (!present) {
+ throw new AssertionError("upload not found in /certificates/2026");
+}
+```
+
+
+
+
+```python
+listing = client.files.list_directory_by_path("/certificates/2026")
+for node in listing:
+ print(node.type, node.name, node.size)
+
+assert "calibration_a12.pdf" in [node.name for node in listing], "upload not found"
+
+roots = client.files.list_root_directory() # top of the tree
+```
+
+
+
+
+```rust
+let listing = api.files.list_directory_by_path("/certificates/2026").await?;
+for node in listing.get_items() {
+ println!("{} {} {} bytes", node.r#type.as_deref().unwrap_or(""), node.name, node.size);
+}
+
+assert!(listing.get_items().iter().any(|node| node.name == "calibration_a12.pdf"),
+ "upload not found in /certificates/2026");
+```
+
+
+
+
+## Download
+
+Download by the id the upload echoed. Python and Rust take it as an integer, Java as a
+string (the endpoint also accepts an external id in that position).
+
+
+
+
+```java
+long id = uploaded.getItems().get(0).getId();
+byte[] bytes = client.files().download(String.valueOf(id));
+Files.write(Path.of("calibration_a12.pdf"), bytes);
+```
+
+
+
+
+```python
+from pathlib import Path
+
+download = client.files.download(uploaded[0].id)
+Path("calibration_a12.pdf").write_bytes(download.content)
+```
+
+
+
+
+```rust
+let id = uploaded.get_items()[0].id.expect("the upload echo carries the id");
+let download = api.files.download(id).await?;
+std::fs::write("calibration_a12.pdf", &download.bytes)?;
+```
+
+
+
+
+## Delete
+
+
+
+
+```java
+client.files().delete(List.of(IdCollection.createFromExternalId("calibration_pump_esp_a12")));
+```
+
+
+
+
+```python
+client.files.delete(["calibration_pump_esp_a12"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+
+api.files.delete(&DataWrapper::from(
+ vec![IdAndExtId::from_external_id("calibration_pump_esp_a12")])).await?;
+```
+
+
+
+
+See the [Files reference](/reference/files) for every method and the upload metadata
+fields.
diff --git a/versioned_docs/version-1.0/guides/correlate-alarms.mdx b/versioned_docs/version-1.0/guides/correlate-alarms.mdx
new file mode 100644
index 0000000..5d2f7e1
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/correlate-alarms.mdx
@@ -0,0 +1,156 @@
+---
+sidebar_position: 8
+title: Correlate alarms with the graph
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Correlate alarms with the graph
+
+Two alarms fire within seconds of each other. Are they one incident or two? A flat
+lookup can't tell you, but the **relationship graph** can. If both alarmed sensors
+sit under the same subsystem, the two alarms share a subsystem.
+
+Graph traversal (`fetch_related` in Python and Rust, `fetchRelated` in Java) walks the graph
+outward from a node and returns the connected sub-graph. Walk the neighbourhood of each alarm
+and intersect the two: a shared ancestor, say a `cooling_system` both sensors are `PART_OF`, is
+the common cause.
+
+:::info Needs a sandbox
+This guide reads data it does not create. [Seed a sandbox](/guides/seed-a-sandbox) writes it
+in one go.
+:::
+
+:::note Why a flat read isn't enough
+A flat read (`by_ids`, `get_by_id`, `filter`, `search`) returns the resource alone, with an
+empty `relatedResources`: it carries no edges at all. A shared subsystem might be several hops
+away (sensor → pump → skid → cooling_system), and only graph traversal follows the chain.
+:::
+
+## 1. From alarms to resources
+
+An alarm is an [event](./detect-events.mdx) that names the resource it concerns. The two
+alarms the sandbox seeds carry the sensor in `metadata.resource`, so the ids are given here:
+`sensor_a` and `sensor_b`. In production an alarm carries the resource in `relatedResources`,
+and a read of the event gives you the id.
+
+## 2. Walk each alarm's neighbourhood
+
+Fetch the sub-graph around each sensor. Bounding to `PART_OF` edges keeps the walk to
+the *structural* hierarchy (containment), ignoring incidental links.
+
+
+
+
+```java
+RelatedResourcesForm a = new RelatedResourcesForm();
+a.setExternalId("sensor_a");
+a.setDepth(5);
+a.setRelationshipTypes(List.of("PART_OF"));
+
+RelatedResourcesForm b = new RelatedResourcesForm();
+b.setExternalId("sensor_b");
+b.setDepth(5);
+b.setRelationshipTypes(List.of("PART_OF"));
+
+ResourceNetwork na = client.resources().fetchRelated(a);
+ResourceNetwork nb = client.resources().fetchRelated(b);
+```
+
+
+
+
+```python
+na = client.resources.fetch_related(
+ external_id="sensor_a", depth=5, relationship_types=["PART_OF"])
+nb = client.resources.fetch_related(
+ external_id="sensor_b", depth=5, relationship_types=["PART_OF"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let na = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("sensor_a")
+ .with_depth(5)
+ .with_relationship_types(vec!["PART_OF".into()])).await?;
+let nb = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("sensor_b")
+ .with_depth(5)
+ .with_relationship_types(vec!["PART_OF".into()])).await?;
+```
+
+
+
+
+## 3. Find the shared subsystem
+
+Intersect the two node sets. The walk follows edges in both directions, so once it reaches
+the shared skid it also comes back down the other branch: besides the sensors themselves, the
+intersection holds both pumps as well as `skid_1` and `cooling_system`, the subsystems both
+alarms belong to and the likely common cause.
+
+
+
+
+```java
+Set aNodes = na.nodes().stream()
+ .map(NodeModel::getExternalId).collect(Collectors.toSet());
+
+Set shared = nb.nodes().stream()
+ .map(NodeModel::getExternalId)
+ .filter(aNodes::contains)
+ .collect(Collectors.toSet());
+shared.remove("sensor_a");
+shared.remove("sensor_b");
+
+System.out.println("Both alarms are part of: " + shared); // [cooling_system, skid_1, pump_a, pump_b]
+if (!shared.contains("cooling_system")) {
+ throw new AssertionError("expected cooling_system, found " + shared);
+}
+```
+
+
+
+
+```python
+a_nodes = {n.external_id for n in na.nodes}
+shared = {n.external_id for n in nb.nodes} & a_nodes
+shared -= {"sensor_a", "sensor_b"}
+
+print("Both alarms are part of:", shared) # {'cooling_system', 'skid_1', 'pump_a', 'pump_b'}
+assert "cooling_system" in shared, f"expected cooling_system, found {shared}"
+```
+
+
+
+
+```rust
+use std::collections::HashSet;
+
+let a_nodes: HashSet<&str> = na.nodes().iter().map(|n| n.external_id()).collect();
+let mut shared: HashSet<&str> = nb.nodes().iter()
+ .map(|n| n.external_id())
+ .filter(|id| a_nodes.contains(id))
+ .collect();
+shared.remove("sensor_a");
+shared.remove("sensor_b");
+
+println!("Both alarms are part of: {:?}", shared); // {"cooling_system", "skid_1", "pump_a", "pump_b"}
+assert!(shared.contains("cooling_system"), "expected cooling_system, found {:?}", shared);
+```
+
+
+
+
+:::tip Deeper networks, smarter correlation
+Raise `depth` to correlate across a deeper hierarchy, a whole plant rather than one
+skid. Drop the relationship-type filter to follow *any* connection (power feeds, data
+flows), not just containment, when a fault can propagate sideways.
+:::
+
+The same pattern answers "what else might this failure affect?", walk out from a
+failing component and the returned `nodes` are its blast radius.
diff --git a/versioned_docs/version-1.0/guides/detect-events.mdx b/versioned_docs/version-1.0/guides/detect-events.mdx
new file mode 100644
index 0000000..e9d4986
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/detect-events.mdx
@@ -0,0 +1,153 @@
+---
+sidebar_position: 5
+title: Turn readings into events
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Turn readings into events
+
+Raw datapoints answer "what is the value?"; **events** answer "what happened?". A
+common pipeline reads recent readings, checks them against a rule, and records a
+discrete, queryable event when the rule fires, a threshold breach, a state change,
+an alarm. Events carry a type, a time, and metadata, and can reference the resources
+they concern.
+
+:::info Needs a sandbox
+This guide reads data it does not create. [Seed a sandbox](/guides/seed-a-sandbox) writes it
+in one go.
+:::
+
+## Detect a threshold breach and record it
+
+Read the latest hour, and if any reading exceeds a limit, create an event.
+
+
+
+
+```java
+import java.time.ZonedDateTime;
+
+var filter = new RetrieveFilter();
+filter.setExternalId("engine_temperature");
+filter.setStart(ZonedDateTime.now().minusHours(1));
+filter.setEnd(ZonedDateTime.now());
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+var series = client.timeseries().retrieve(request).getItems().get(0);
+boolean tooHot = series.getDatapoints().stream()
+ .anyMatch(p -> Double.parseDouble(p.getValue()) > 110.0);
+
+if (tooHot) {
+ EventModel event = new EventModel();
+ event.setExternalId("overheat_press_07_" + System.currentTimeMillis());
+ event.setType("overheat");
+ event.setStatus("open");
+ event.setMetadata(Map.of("series", "engine_temperature", "limit", "110"));
+ event.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(event));
+}
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=1),
+ end=pd.Timestamp.now(tz="UTC"))
+
+points = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+if any(dp.value is not None and dp.value > 110.0 for dp in points):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"overheat_press_07_{int(pd.Timestamp.now().timestamp())}",
+ type="overheat",
+ status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"series": "engine_temperature", "limit": "110"})])
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use intellistream_datahub_sdk::events::Event;
+
+let filter = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(1)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let response = api.time_series
+ .retrieve_datapoints(&DataWrapper::from_vec(vec![filter])).await?;
+let series = response.get_items().first().ok_or("engine_temperature returned nothing")?;
+
+let too_hot = series.datapoints.iter()
+ .any(|p| p.value.is_some_and(|v| v > 110.0));
+
+if too_hot {
+ let mut event = Event::new(
+ format!("overheat_press_07_{}", Utc::now().timestamp()),
+ "overheat".into(),
+ Utc::now());
+ event.status = Some("open".into());
+ event.add_metadata("series".into(), "engine_temperature".into());
+ event.add_metadata("limit".into(), "110".into());
+ api.events.create(&vec![event]).await?;
+}
+```
+
+
+
+
+## Query the events later
+
+Events are first-class records, filter them by type, time or metadata for an audit
+trail or an incident timeline.
+
+
+
+
+```java
+EventRetreiver retriever = new EventRetreiver();
+retriever.setLimit(100);
+retriever.getFilter().setType(List.of("overheat"));
+DataWrapper overheats = client.events().filter(retriever);
+```
+
+
+
+
+```python
+overheats = client.events.filter(type="overheat", limit=100)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::{EventFilter, EventFilterForm};
+
+let filter = EventFilterForm::default()
+ .set_filter(EventFilter { r#type: Some(vec!["overheat".into()]), ..Default::default() })
+ .set_limit(100)
+ .build();
+let overheats = api.events.filter(&filter).await?;
+```
+
+
+
+
+:::tip Run it on the live stream
+Polling the last hour is fine for a cron job. To react the moment a reading crosses
+the line, run this rule inside a [live subscription loop](./realtime-subscriptions.mdx)
+instead of on a timer.
+:::
diff --git a/versioned_docs/version-1.0/guides/ingest-timeseries.mdx b/versioned_docs/version-1.0/guides/ingest-timeseries.mdx
new file mode 100644
index 0000000..cb0dbf0
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/ingest-timeseries.mdx
@@ -0,0 +1,313 @@
+---
+sidebar_position: 1
+title: High-throughput ingestion
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# High-throughput ingestion
+
+A common job: push a steady, high-volume stream of sensor readings in so a dashboard
+or alerting rule can act on them. Every SDK **chunks the data into batches, sends them
+concurrently and retries transient failures**: Java in process, Python and Rust through the
+durable spool.
+
+:::info Needs a sandbox
+This guide writes into `engine_temperature`, which section **A** of
+[Seed a sandbox](/guides/seed-a-sandbox) creates.
+:::
+
+:::note How it works, and the caps it fits
+Datapoints are split into batches (the store is optimised for large batches) and sent
+concurrently up to a bounded in-flight limit. The caps below are the API's and bind every client.
+Java sends **10,000** datapoints per request by default, retries transient failures (HTTP
+429/5xx, network) in process and returns an `IngestResult` summarising what landed and what
+didn't. Python and Rust send up to **100,000** per request, four at once (one at a time with
+buffering on), and retry through the
+[durable spool](/reference/client#durable-ingest-buffering): with buffering on, a transient
+failure (429, 5xx, network, or a `401`/`403`) writes the input to disk, and the next ingest call
+sends the spool again, oldest first, before its own data. With buffering off, the first request
+to fail is the call's error. The defaults fit the caps, so the table only matters if you have
+tuned something, with one exception: Python and Rust split by count whatever the value type, so
+keep a `TEXT` or `MIXED` series to 10,000 points per call.
+
+| Cap | Value |
+| --- | --- |
+| `items` in one request | **10,000** |
+| Datapoints in one collection, numeric | 100,000 |
+| Datapoints in one collection, `TEXT` / `MIXED` | **10,000** |
+| Body of `POST /timeseries/data` | 16 MiB (10,000 numeric points is roughly 500 KB) |
+
+Over a cap is a `400`; an oversized body is a `413`, which no retry will fix. Sustained
+volume is bounded separately by a per-minute rate limit and a daily ingest quota, both `429`.
+Java retries them in process; Python and Rust retry them through the spool when buffering is on,
+and return them as the call's error when it is off. See [Limits & quotas](/reference/limits). The [binary path](#binary-ingest) has
+caps of its own, and each SDK stays under them by itself.
+:::
+
+## Ingest a million readings
+
+
+
+
+`ingest` groups datapoints by series external id and returns an `IngestResult`.
+
+```java
+import java.time.Instant;
+import java.util.*;
+
+var client = DatahubClient.fromEnv();
+
+// a million readings, one per second from the start of 2026
+Instant start = Instant.parse("2026-01-01T00:00:00Z");
+List points = new ArrayList<>(1_000_000);
+for (int i = 0; i < 1_000_000; i++) {
+ points.add(Datapoint.of(start.plusSeconds(i), 90 + 5 * Math.random()));
+}
+Map> readings = Map.of("engine_temperature", points);
+
+IngestResult result = client.timeseries().ingest(readings,
+ IngestOptions.builder()
+ .batchSize(10_000) // datapoints per request
+ .parallelism(16) // concurrent in-flight batches
+ .maxRetries(3)
+ .build());
+
+System.out.printf("ingested %,d, failed %,d%n", result.succeeded(), result.failed());
+if (!result.isComplete()) {
+ result.errors().forEach(e -> System.err.println(e.statusCode() + ": " + e.message()));
+ throw new IllegalStateException("ingest incomplete");
+}
+```
+
+
+
+
+`insert_from_lists` takes whole arrays (NumPy / pandas) and batches them for you.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+# a million readings, one per second from the start of 2026
+client.timeseries.insert_from_lists(
+ timestamps=pd.date_range("2026-01-01", periods=1_000_000, freq="s", tz="UTC"),
+ values=90 + 5 * np.random.rand(1_000_000),
+ ts="engine_temperature")
+
+# read the first hour back: 3 600 readings
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=pd.Timestamp("2026-01-01 00:00", tz="UTC"),
+ end=pd.Timestamp("2026-01-01 01:00", tz="UTC"),
+ limit=10_000)
+points = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+print(f"{len(points)} readings in the first hour")
+assert len(points) >= 3600, len(points)
+```
+
+
+
+
+`insert_datapoints` splits large inputs into requests of at most 100 000 datapoints.
+
+```rust
+use chrono::{Duration, TimeZone, Utc};
+use intellistream_datahub_sdk::create_api_service;
+use intellistream_datahub_sdk::generic::{DataWrapper, DatapointString, DatapointsCollection, RetrieveFilter};
+
+let api = create_api_service();
+
+// a million readings, one per second from the start of 2026
+let start = Utc.with_ymd_and_hms(2026, 1, 1, 0, 0, 0).unwrap();
+let readings: Vec = (0..1_000_000)
+ .map(|i| DatapointString::from_datetime(
+ start + Duration::seconds(i),
+ &format!("{:.2}", 90.0 + (i % 50) as f64 / 10.0)))
+ .collect();
+
+let mut dw = DataWrapper::new();
+dw.add_item(DatapointsCollection {
+ external_id: Some("engine_temperature".into()),
+ datapoints: readings,
+ ..Default::default()
+});
+api.time_series.insert_datapoints(&mut dw).await?;
+
+// read the first hour back: 3 600 readings
+let first_hour = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(start),
+ end: Some(start + Duration::hours(1)),
+ limit: Some(10_000),
+ ..Default::default()
+};
+let points = api.time_series.retrieve_datapoints(&DataWrapper::from(vec![first_hour])).await?;
+let n = points.get_items()[0].datapoints.len();
+println!("{n} readings in the first hour");
+assert!(n >= 3600, "expected 3600 readings, got {n}");
+```
+
+
+
+
+## Tuning
+
+Java's `ingest` knobs let you trade throughput against load on the server:
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `batchSize` | `10_000` | Datapoints per request. Do not raise it above `10_000`, the server's [`items` cap](/reference/limits#batch-caps). |
+| `parallelism` | `8` | Concurrent in-flight requests. |
+| `maxRetries` | `3` | Retries for transient failures. |
+| `failFast` | `false` | Abort on the first failed batch instead of collecting errors. |
+
+When `failFast` is off, inspect `result.errors()` for the per-batch failures. See the
+[Time series reference](/reference/timeseries#ingestresult) for the full result shape.
+
+Python and Rust have one knob, the number of requests in flight (default 4). Set
+`DATAPOINT_INSERT_PARALLELISM` in the environment, which `create_api_service()` and Python's
+`from_env()` read, or call `set_datapoint_insert_parallelism` on a Rust `DataHubConfig`. More is
+faster only while the API has cores to parse the requests on.
+
+## Binary ingest {#binary-ingest}
+
+Past a few thousand points a second, JSON is the bottleneck: every value is text the API has
+to parse. `ingestBinary` in Java and `insert_datapoints_binary` in Python and Rust send the same
+readings as zstd-compressed Arrow frames to
+[`POST /timeseries/data/binary`](/reference/binary-datapoints), a million points or more per
+request, parsed and sorted on the client. Pick it when the volume is sustained; keep the JSON
+path when you want the [durable spool](/reference/client#durable-ingest-buffering), which the
+binary path does not have. It resolves each series through `/timeseries/byids` once and caches
+it, so the token needs **read** access to the data set as well as write. Python has it on the
+synchronous `DataHubClient` only.
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.ingest.BinaryIngestOptions;
+import java.time.ZoneOffset;
+
+IngestResult result = client.timeseries().ingestBinary(readings, // the same map as above
+ BinaryIngestOptions.builder()
+ .zstdLevel(3) // 1, 3 or 9 (default 9): client CPU against bytes on the wire
+ .parallelism(16) // concurrent in-flight requests
+ .build());
+
+System.out.printf("ingested %,d, failed %,d%n", result.succeeded(), result.failed());
+if (!result.isComplete()) {
+ result.errors().forEach(e -> System.err.println(e.statusCode() + ": " + e.message()));
+ throw new IllegalStateException("ingest incomplete");
+}
+
+// read the first hour back: 3 600 readings
+RetrieveFilter firstHour = new RetrieveFilter();
+firstHour.setExternalId("engine_temperature");
+firstHour.setStart(start.atZone(ZoneOffset.UTC));
+firstHour.setEnd(start.plusSeconds(3600).atZone(ZoneOffset.UTC));
+firstHour.setLimit(10_000);
+DataRetriever request = new DataRetriever<>();
+request.setItems(List.of(firstHour));
+
+int n = client.timeseries().retrieve(request).getItems().get(0).getDatapoints().size();
+System.out.printf("%d readings in the first hour%n", n);
+if (n < 3600) throw new IllegalStateException("expected 3600 readings, got " + n);
+```
+
+
+
+
+`insert_from_lists_binary` takes the same arrays as `insert_from_lists`.
+
+```python
+# the same million readings, as Arrow frames
+client.timeseries.insert_from_lists_binary(
+ timestamps=pd.date_range("2026-01-01", periods=1_000_000, freq="s", tz="UTC"),
+ values=90 + 5 * np.random.rand(1_000_000),
+ ts="engine_temperature",
+ zstd_level=3) # 1, 3 or 9 (default 9): client CPU against bytes on the wire
+
+# read the first hour back: 3 600 readings
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=pd.Timestamp("2026-01-01 00:00", tz="UTC"),
+ end=pd.Timestamp("2026-01-01 01:00", tz="UTC"),
+ limit=10_000)
+points = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
+print(f"{len(points)} readings in the first hour")
+assert len(points) >= 3600, len(points)
+```
+
+
+
+
+`insert_datapoints_binary` takes the same `DataWrapper` as `insert_datapoints`, by reference.
+
+```rust
+use intellistream_datahub_sdk::timeseries::BinaryIngestOptions;
+
+// the same million readings, built again: insert_datapoints drains the wrapper as it sends
+let mut dw = DataWrapper::new();
+dw.add_item(DatapointsCollection {
+ external_id: Some("engine_temperature".into()),
+ datapoints: (0..1_000_000)
+ .map(|i| DatapointString::from_datetime(
+ start + Duration::seconds(i),
+ &format!("{:.2}", 90.0 + (i % 50) as f64 / 10.0)))
+ .collect(),
+ ..Default::default()
+});
+api.time_series
+ .insert_datapoints_binary(&dw, &BinaryIngestOptions::new().zstd_level(3)) // 1, 3 or 9 (default 9)
+ .await?;
+
+// read the first hour back: 3 600 readings
+let first_hour = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(start),
+ end: Some(start + Duration::hours(1)),
+ limit: Some(10_000),
+ ..Default::default()
+};
+let points = api.time_series.retrieve_datapoints(&DataWrapper::from(vec![first_hour])).await?;
+let n = points.get_items()[0].datapoints.len();
+println!("{n} readings in the first hour");
+assert!(n >= 3600, "expected 3600 readings, got {n}");
+```
+
+
+
+
+A series that does not exist, or a value that does not fit its type, is caught without a request
+(status 404 and 422): Java reports it in `result.errors()`, Python and Rust return it as the
+call's error. A request the server refuses because a series was renamed or removed since it was
+cached is rebuilt once after re-resolving, and `429`, `5xx` and network failures are retried.
+
+For readings that arrive one at a time, Java has a buffer that batches them into frames and
+sends once 10 000 points have accumulated or the oldest is 200 ms old. `add` never blocks on the
+network, and closing the buffer flushes what is left:
+
+```java
+import ai.intellistream.datahub.sdk.client.BinaryIngestBuffer;
+
+try (BinaryIngestBuffer buffer = client.timeseries().binaryBuffer()) {
+ buffer.add("engine_temperature", Instant.now(), 92.4);
+ buffer.add("engine_rpm", Instant.now(), 1500L);
+}
+```
+
+Python and Rust have no such buffer: collect the readings and send them with
+`insert_datapoints_binary`. The options and the buffer's other methods are in the
+[Time series reference](/reference/timeseries#binary-ingest).
+
+## Then chart the trend
+
+Once the data is in, roll it up to hourly or daily buckets for a dashboard, see
+[Query & aggregate time series](./query-and-aggregate.mdx).
+
+:::warning Ordering
+Batches are sent in parallel, so there is **no cross-batch ordering guarantee**,
+which is fine for time-stamped data, since each datapoint carries its own timestamp.
+:::
diff --git a/versioned_docs/version-1.0/guides/model-assets-graph.mdx b/versioned_docs/version-1.0/guides/model-assets-graph.mdx
new file mode 100644
index 0000000..f0e2f49
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/model-assets-graph.mdx
@@ -0,0 +1,231 @@
+---
+sidebar_position: 3
+title: Model assets as a graph
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Model assets as a graph
+
+Most domains are a hierarchy of things that contain or feed each other, a plant
+contains lines, a line contains machines, a machine emits sensor readings. Model
+that as **resources** (the nodes) and **relations** (the edges), and the SDK returns
+the persisted graph in one call.
+
+## Create the graph
+
+Create the nodes and the edges between them together. Each node needs **at least one
+label** (a type such as `Plant` or `Machine`). Relationship-type names and labels are
+upper-cased by the server; **external ids are not touched at all** and are stored exactly as
+you send them, so you can mirror the tags your operation already maintains
+(`COM-99-PT-1034`, `=K1-M3+B02`). The example below uses snake_case because that reads
+well in code, not because the server requires it.
+[External ids & naming →](/reference/external-ids)
+
+
+
+
+```java
+Resource plant = new Resource();
+plant.setExternalId("plant_oslo");
+plant.setName("Oslo Plant");
+plant.setLabels(List.of("Plant")); // at least one label is required
+
+Resource line = new Resource();
+line.setExternalId("line_a");
+line.setName("Assembly line A");
+line.setLabels(List.of("Line"));
+
+Resource press = new Resource();
+press.setExternalId("press_07");
+press.setName("Hydraulic press 07");
+press.setLabels(List.of("Machine"));
+
+RelForm plantLine = new RelForm();
+plantLine.setName("contains");
+plantLine.setFromExternalId("plant_oslo");
+plantLine.setToExternalId("line_a");
+
+RelForm lineMachine = new RelForm();
+lineMachine.setName("contains");
+lineMachine.setFromExternalId("line_a");
+lineMachine.setToExternalId("press_07");
+
+var graph = client.resources().create(
+ List.of(plant, line, press), List.of(plantLine, lineMachine));
+
+System.out.println(graph.getNodes().size() + " nodes, "
+ + graph.getRelations().size() + " edges");
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+nodes = [
+ intellistream_datahub_sdk.Resource(external_id="plant_oslo", name="Oslo Plant", labels=["Plant"]),
+ intellistream_datahub_sdk.Resource(external_id="line_a", name="Assembly line A", labels=["Line"]),
+ intellistream_datahub_sdk.Resource(external_id="press_07", name="Hydraulic press 07", labels=["Machine"]),
+]
+edges = [
+ intellistream_datahub_sdk.RelForm.by_external_ids("plant_oslo", "line_a", "contains"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("line_a", "press_07", "contains"),
+]
+
+graph = client.resources.create(nodes, edges)
+print(len(graph.nodes), "nodes,", len(graph.relations), "edges")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut plant = Resource::new();
+plant.external_id = "plant_oslo".into();
+plant.name = "Oslo Plant".into();
+plant.labels = Some(vec!["Plant".into()]); // at least one label is required
+
+let mut line = Resource::new();
+line.external_id = "line_a".into();
+line.name = "Assembly line A".into();
+line.labels = Some(vec!["Line".into()]);
+
+let mut press = Resource::new();
+press.external_id = "press_07".into();
+press.name = "Hydraulic press 07".into();
+press.labels = Some(vec!["Machine".into()]);
+
+let graph = api.resources.create(
+ vec![plant, line, press],
+ vec![
+ RelForm::by_external_ids("plant_oslo", "line_a", "contains"),
+ RelForm::by_external_ids("line_a", "press_07", "contains"),
+ ],
+).await?;
+
+println!("{} nodes, {} edges",
+ graph.nodes().map_or(0, |n| n.len()),
+ graph.relations().map_or(0, |r| r.len()));
+```
+
+
+
+
+## Attach a time series to a node
+
+A machine's sensors are time series. Create the series, tie it to the machine that produces
+it with a relationship (`press_07` → `PRODUCES` → `press_07_oil_temp`), then
+[ingest its datapoints](./ingest-timeseries.mdx).
+
+
+
+
+```java
+Timeseries oilTemp = new Timeseries();
+oilTemp.setExternalId("press_07_oil_temp");
+oilTemp.setName("Press 07 oil temperature");
+oilTemp.setUnit("celsius");
+oilTemp.setUnitExternalId("temperature_deg_c");
+client.timeseries().create(List.of(oilTemp));
+
+RelForm produces = new RelForm();
+produces.setRelationshipType("PRODUCES");
+produces.setFromExternalId("press_07");
+produces.setToExternalId("press_07_oil_temp");
+client.edges().create(List.of(produces));
+```
+
+
+
+
+```python
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="press_07_oil_temp", name="Press 07 oil temperature", value_type="float",
+ unit="celsius", unit_external_id="temperature_deg_c")])
+
+client.edges.create([
+ intellistream_datahub_sdk.RelForm.by_external_ids("press_07", "press_07_oil_temp", "PRODUCES")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::relations::RelForm;
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+let mut oil_temp = TimeSeries::new("press_07_oil_temp", "Press 07 oil temperature");
+oil_temp.value_type = Some("float".into());
+oil_temp.unit = Some("celsius".into());
+oil_temp.unit_external_id = Some("temperature_deg_c".into());
+api.time_series.create_one(&oil_temp).await?;
+
+api.edges.create(&vec![
+ RelForm::by_external_ids("press_07", "press_07_oil_temp", "PRODUCES"),
+]).await?;
+```
+
+
+
+
+## Read the graph back
+
+Fetch a node's neighbourhood with `fetch_related` (`fetchRelated` in Java); the returned network
+carries the nodes it found and the edges between them. (`by_ids` returns the node alone:
+`relatedResources` is empty on that read.) One hop out from the press finds the two edges
+created above.
+
+
+
+
+```java
+ResourceNetwork net = client.resources().fetchRelated("press_07", 1);
+net.edges().forEach(edge ->
+ System.out.println(edge.getStart() + " -" + edge.getType() + "-> " + edge.getEnd()));
+
+if (net.edges().size() != 2) {
+ throw new AssertionError("expected CONTAINS and PRODUCES, found " + net.edges().size());
+}
+```
+
+
+
+
+```python
+net = client.resources.fetch_related(external_id="press_07", depth=1)
+for edge in net.edges:
+ print(edge.start, edge.relationship_type, edge.end)
+
+types = {edge.relationship_type for edge in net.edges}
+assert types == {"CONTAINS", "PRODUCES"}, types
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let net = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("press_07").with_depth(1)).await?;
+for edge in net.edges() {
+ println!("{:?} -{:?}-> {:?}", edge.start, edge.relationship_type, edge.end);
+}
+
+assert_eq!(net.edges().len(), 2, "expected CONTAINS and PRODUCES");
+```
+
+
+
+
+:::tip Group with data sets
+Relationships model *structure*. To slice assets by ownership, environment or tenant,
+"everything in the Oslo plant", also put them in a **data set** (see the
+[Data sets reference](/reference/datasets)). A resource can sit in a graph and a
+data set at once.
+:::
diff --git a/versioned_docs/version-1.0/guides/query-and-aggregate.mdx b/versioned_docs/version-1.0/guides/query-and-aggregate.mdx
new file mode 100644
index 0000000..a757061
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/query-and-aggregate.mdx
@@ -0,0 +1,234 @@
+---
+sidebar_position: 2
+title: Query & aggregate time series
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Query & aggregate time series
+
+Read raw datapoints back for a window, then roll them up to fixed buckets
+(hourly averages, daily maxima) for a chart or report. Aggregates and the bucket
+size are part of the retrieve filter, no separate endpoint.
+
+:::info Needs a sandbox
+This guide reads data it does not create. [Seed a sandbox](/guides/seed-a-sandbox) writes it
+in one go.
+:::
+
+## Raw datapoints for a window
+
+
+
+
+```java
+import java.time.ZonedDateTime;
+
+var filter = new RetrieveFilter();
+filter.setExternalId("engine_temperature");
+filter.setStart(ZonedDateTime.now().minusDays(1));
+filter.setEnd(ZonedDateTime.now());
+filter.setLimit(10_000);
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().forEach(series ->
+ series.getDatapoints().forEach(p ->
+ System.out.println(p.getTimestamp() + " = " + p.getValue())));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1),
+ end=pd.Timestamp.now(tz="UTC"),
+ limit=10_000)
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ print(dp.timestamp, dp.value)
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+
+let filter = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(Utc::now() - chrono::Duration::days(1)),
+ end: Some(Utc::now()),
+ limit: Some(10_000),
+ ..Default::default()
+};
+let result = api.time_series.retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+for series in result.get_items() {
+ for p in &series.datapoints {
+ println!("{} = {:?}", p.timestamp, p.value); // value is Option
+ }
+}
+```
+
+
+
+
+## Hourly averages
+
+Set `aggregates` and `granularity` and the server returns one value per bucket.
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("engine_temperature");
+filter.setStart(ZonedDateTime.now().minusDays(1));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg", "max"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+// NOTE: Java can't read aggregate values yet — getValue() returns null here (see warning below)
+client.timeseries().retrieve(request).getItems()
+ .forEach(s -> s.getDatapoints().forEach(p ->
+ System.out.println(p.getTimestamp() + " = " + p.getValue())));
+```
+
+
+
+
+```python
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg", "max"],
+ granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ print(dp.timestamp, dp.average, dp.max)
+```
+
+
+
+
+```rust
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(Utc::now() - chrono::Duration::days(1)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into(), "max".into()]),
+ granularity: Some("1h".into()),
+ ..Default::default()
+};
+let result = api.time_series.retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+for series in result.get_items() {
+ for p in &series.datapoints {
+ println!("{} avg={:?} max={:?}", p.timestamp, p.average, p.max);
+ }
+}
+```
+
+
+
+
+:::note The aggregates and granularities
+The aggregates are `avg`, `sum`, `min` and `max`, lower-case; a name outside that set is
+dropped, not rejected. Granularity is a number plus a unit: `s`, `m`, `h`, `d`, `w`, `mo`, `y`,
+or the words `sec`, `min`, `hour`, `day`, `week`, `month`, `year` and their plurals (`30s`,
+`5m`, `1h`, `1d`). Bare `m` is a minute; a month is `mo`. The Python and Rust datapoint types
+expose each requested aggregate as its own field: `min`/`max`/`sum` by name, and `avg` read
+back on `.average` (`dp.average`, `dp.max`, …).
+[Retrieve datapoints →](/reference/timeseries#retrieve-datapoints)
+:::
+
+:::warning Java aggregate reads
+The Java client does **not** expose aggregate values yet: `getValue()` returns `null`
+for aggregated datapoints (the server sends `avg`/`max`/… as named fields, but the Java
+datapoint type carries only `timestamp`/`value`). For now use **Python or Rust** for
+aggregates, or in Java, fetch raw datapoints and aggregate them in your application.
+:::
+
+## Paging large windows
+
+A wide window can exceed one response. Each returned collection carries a `nextCursor` when
+there is more (`getNextCursor()` in Java, `next_cursor` in Python and Rust); pass it back as
+`cursor` to fetch the next page.
+
+
+
+
+```java
+String cursor = null;
+do {
+ var filter = new RetrieveFilter();
+ filter.setExternalId("engine_temperature");
+ filter.setStart(start);
+ filter.setEnd(end);
+ filter.setCursor(cursor);
+
+ var request = new DataRetriever();
+ request.setItems(List.of(filter));
+
+ var page = client.timeseries().retrieve(request).getItems().get(0);
+ page.getDatapoints().forEach(p -> handle(p));
+ cursor = page.getNextCursor();
+} while (cursor != null);
+```
+
+
+
+
+```python
+end = pd.Timestamp.now(tz="UTC")
+start = end - pd.Timedelta(days=1)
+cursor = None
+while True:
+ rf = intellistream_datahub_sdk.RetrieveFilter(ts="engine_temperature",
+ start=start, end=end, cursor=cursor)
+ page = client.timeseries.retrieve_datapoints(rf)[0]
+ for dp in page.get_datapoints():
+ handle(dp)
+ cursor = page.next_cursor
+ if not cursor:
+ break
+```
+
+
+
+
+```rust
+use chrono::Utc;
+
+let (start, end) = (Utc::now() - chrono::Duration::days(1), Utc::now());
+let mut cursor: Option = None;
+loop {
+ let filter = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(start),
+ end: Some(end),
+ cursor: cursor.clone(),
+ ..Default::default()
+ };
+ let result = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+ let page = &result.get_items()[0];
+ for p in &page.datapoints { handle(p); }
+ cursor = page.next_cursor.clone();
+ if cursor.is_none() { break; }
+}
+```
+
+
+
diff --git a/versioned_docs/version-1.0/guides/realtime-subscriptions.mdx b/versioned_docs/version-1.0/guides/realtime-subscriptions.mdx
new file mode 100644
index 0000000..303cb61
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/realtime-subscriptions.mdx
@@ -0,0 +1,268 @@
+---
+sidebar_position: 4
+title: Consume live data
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Consume live data
+
+Push-based monitoring: a **subscription** names a set of time series, and a live
+connection delivers each new datapoint as it lands, no polling. A control-room
+dashboard, an alerting worker, or a downstream pipeline drives the loop and **acks**
+what it has handled. Anything unacked is redelivered, so a crash is followed by redelivery.
+
+:::info Needs a sandbox
+This guide reads data it does not create. [Seed a sandbox](/guides/seed-a-sandbox) writes it
+in one go. Nothing arrives on the socket until something writes to the series, so once the
+listener below is running, re-run section **A** of the seed in a second shell and watch the
+messages arrive.
+:::
+
+## 1. Create a subscription
+
+
+
+
+```java
+Subscription sub = new Subscription();
+sub.setExternalId("engine_room");
+sub.setName("Engine room");
+sub.setTimeseries(List.of(
+ IdCollection.createFromExternalId("engine_temperature"),
+ IdCollection.createFromExternalId("engine_rpm")));
+client.subscriptions().create(List.of(sub));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.subscriptions.create([
+ intellistream_datahub_sdk.Subscription(
+ external_id="engine_room",
+ name="Engine room",
+ timeseries=["engine_temperature", "engine_rpm"])])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::subscriptions::Subscription;
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let sub = Subscription::new(
+ "engine_room".into(), "Engine room".into(),
+ vec![
+ IdAndExtId::from_external_id("engine_temperature"),
+ IdAndExtId::from_external_id("engine_rpm"),
+ ]);
+api.subscriptions.create(&sub).await?;
+```
+
+
+
+
+## 2. Listen and ack
+
+Drive a loop over the listener (Java can also hand each message to a handler), and ack once
+you've durably handled it. Make the handler **idempotent**, a redelivery after a crash will replay
+the last unacked messages.
+
+
+
+
+Register a handler with `stream`, a dedicated virtual thread delivers each message **as
+it arrives** and acks it once your handler returns (or nacks it if the handler throws).
+The returned handle stops delivery and closes the listener, so use try-with-resources.
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("engine_room"))
+ .stream((SubscriptionMessage msg) -> handle(msg.payload()))) { // auto-acks on return
+ awaitShutdown(); // your app's lifecycle; closing the handle ends the stream
+}
+```
+
+Prefer a loop, or want to ack on your own schedule? Drive `poll` yourself, it blocks up
+to the timeout and returns `null` on a quiet interval (it is a blocking queue hand-off,
+**not** network polling):
+
+```java
+import ai.intellistream.datahub.sdk.client.SubscriptionListener;
+import java.time.Duration;
+
+try (SubscriptionListener listener = client.subscriptions().listen(List.of("engine_room"))) {
+ while (running) {
+ SubscriptionMessage msg = listener.poll(Duration.ofSeconds(5));
+ if (msg == null) continue;
+ handle(msg.payload()); // event action + affected datapoints
+ listener.ack(msg.messageId());
+ }
+}
+```
+
+
+
+
+The listener is iterable and a context manager. The loop does not end on its own, since a
+dropped socket is reconnected for you, and an error such as a refused subscription is raised
+from it:
+
+```python
+with client.subscriptions.listen(["engine_room"]) as listener:
+ for msg in listener:
+ handle(msg.payload)
+ listener.ack([msg.message_id])
+```
+
+
+
+
+`next().await` never returns `None`: a dropped or closed socket is reconnected for you. An
+error comes back as `Some(Err(..))`, either a refused subscription while the socket stays open
+or a reconnect that ran out of retries, so log it and keep going.
+
+```rust
+let mut listener = api.subscriptions.listen(&["engine_room"]).await?;
+while let Some(result) = listener.next().await {
+ match result {
+ Ok(msg) => {
+ handle(&msg.payload);
+ listener.ack(&[msg.message_id.as_str()]).await?;
+ }
+ Err(e) => eprintln!("listen error: {}", e),
+ }
+}
+```
+
+
+
+
+## 3. Count what arrives
+
+Re-run section **A** of the seed in a second shell. Each batch it writes to
+`engine_temperature` or `engine_rpm` becomes a message here, so the count climbs.
+
+
+
+
+```java
+import java.util.concurrent.atomic.AtomicInteger;
+
+AtomicInteger seen = new AtomicInteger();
+try (var stream = client.subscriptions().listen(List.of("engine_room"))
+ .stream((SubscriptionMessage msg) ->
+ System.out.println(seen.incrementAndGet() + " messages so far"))) {
+ Thread.sleep(60_000); // long enough to re-run the seed in another shell
+}
+```
+
+
+
+
+```python
+seen = 0
+with client.subscriptions.listen(["engine_room"]) as listener:
+ for msg in listener:
+ listener.ack([msg.message_id])
+ seen += 1
+ print(f"{seen} messages so far")
+```
+
+
+
+
+```rust
+use std::time::Duration;
+use tokio::time::{timeout_at, Instant};
+
+let mut seen = 0;
+let mut listener = api.subscriptions.listen(&["engine_room"]).await?;
+let deadline = Instant::now() + Duration::from_secs(60); // long enough to re-run the seed in another shell
+while let Ok(Some(result)) = timeout_at(deadline, listener.next()).await {
+ match result {
+ Ok(msg) => {
+ listener.ack(&[msg.message_id.as_str()]).await?;
+ seen += 1;
+ println!("{seen} messages so far");
+ }
+ Err(e) => eprintln!("listen error: {e}"),
+ }
+}
+listener.close().await?;
+```
+
+
+
+
+## Change the interest set at runtime
+
+A long-lived listener doesn't need to reconnect to follow more (or fewer) series:
+`subscribe` / `unsubscribe` adjust it in place. They take **subscription** external ids, not
+series, so to follow another series create a subscription for it first. This swaps
+`engine_room` for one that follows the oil pressure:
+
+
+
+
+```java
+Subscription oil = new Subscription();
+oil.setExternalId("engine_oil");
+oil.setName("Engine oil");
+oil.setTimeseries(List.of(IdCollection.createFromExternalId("engine_oil_pressure")));
+client.subscriptions().create(List.of(oil));
+
+listener.subscribe(List.of("engine_oil"));
+listener.unsubscribe(List.of("engine_room"));
+```
+
+
+
+
+```python
+client.subscriptions.create([
+ intellistream_datahub_sdk.Subscription(
+ external_id="engine_oil",
+ name="Engine oil",
+ timeseries=["engine_oil_pressure"])])
+
+listener.subscribe(["engine_oil"])
+listener.unsubscribe(["engine_room"])
+```
+
+
+
+
+```rust
+let oil = Subscription::new(
+ "engine_oil".into(), "Engine oil".into(),
+ vec![IdAndExtId::from_external_id("engine_oil_pressure")]);
+api.subscriptions.create(&oil).await?;
+
+listener.subscribe(&["engine_oil"]).await?;
+listener.unsubscribe(&["engine_room"]).await?;
+```
+
+
+
+
+:::tip nack to retry later
+If a message can't be processed right now (a downstream system is down), `nack` it
+instead of acking, it will be redelivered rather than dropped. In Java, throwing from a
+`stream` handler nacks for you; to control acks explicitly use
+`stream(handler, AckMode.MANUAL)` and call `ack`/`nack` yourself, and with `poll`, just call
+`nack` instead of `ack`. Python and Rust have no `stream` or `poll`: in the loops above, call
+`listener.nack(...)` in place of `listener.ack(...)`.
+:::
+
+## See also
+
+- [Sustained-condition alarms](/advanced/sustained-alarm-window): a full worked example that keeps a sliding window over this stream, seeds its own data and then verifies what it found.
+- [Turn readings into events](/guides/detect-events): recording what a rule decides.
+- [Seed a sandbox](/guides/seed-a-sandbox): the series this guide subscribes to.
+- Stream processing, in the platform documentation, windows, event time against arrival time, and what has to be remembered.
diff --git a/versioned_docs/version-1.0/guides/seed-a-sandbox.mdx b/versioned_docs/version-1.0/guides/seed-a-sandbox.mdx
new file mode 100644
index 0000000..3974cd9
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/seed-a-sandbox.mdx
@@ -0,0 +1,152 @@
+---
+sidebar_position: 0
+title: Seed a sandbox
+---
+
+# Seed a sandbox
+
+Most guides in this section read series, resources and events that have to exist first. This
+page creates them, once, using the SDK's own calls, so every guide can be run end to end
+against a sandbox rather than read.
+
+:::info What needs seeding, and what does not
+| Guide | Needs |
+| --- | --- |
+| [High-throughput ingestion](/guides/ingest-timeseries) | **A** (the series must exist) |
+| [Query & aggregate](/guides/query-and-aggregate) | **A** the engine series |
+| [Model assets as a graph](/guides/model-assets-graph) | Nothing. It creates the graph it reads |
+| [Consume live data](/guides/realtime-subscriptions) | **A** the engine series |
+| [Turn readings into events](/guides/detect-events) | **A** the engine series, which include a hot stretch so the rule actually fires |
+| [Attach files to assets](/guides/attach-files) | Nothing. It uploads what it then reads |
+| [Work with units](/guides/work-with-units) | Nothing. It creates the series it tags |
+| [Correlate alarms](/guides/correlate-alarms) | **B** the cooling system, its two sensors and their alarms |
+:::
+
+Everything below is Python (`numpy` + `pandas` + the SDK), the same shape as
+[Generate sample data](/advanced/generate-sample-data) for the advanced scenarios. Java and
+Rust reach the same result through the calls each guide already shows.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+def ingest(external_id, index, values, unit=None, name=None):
+ try:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id=external_id, name=name or external_id,
+ unit=unit or "value", value_type="float")])
+ except intellistream_datahub_sdk.DataHubException as e:
+ if e.status_code != 409: # a second run finds the series already there
+ raise
+ client.timeseries.insert_from_lists(
+ timestamps=index, values=np.asarray(values), ts=external_id)
+ print(f"ingested {len(values):,} points → {external_id}")
+```
+
+Every block below can be run again. A second run of a series or resource create answers
+`409` on the duplicate external id: the helper catches that and goes on to write the
+datapoints, which dedup on `(series, timestamp)`, and block B catches it around the graph
+create. Events are the exception: an event's external id is not unique, so a second run raises
+two more alarms.
+
+## A. An engine, with the last twenty minutes running hot
+
+Three series for one engine: a day of readings at one a minute. The temperature ends in a
+climb past 110 °C, because [Turn readings into events](/guides/detect-events) looks at the
+last hour and fires above that limit. Seed this and the guide has something to find.
+
+```python
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=24*60, freq="1min")
+n, t = len(idx), np.arange(24*60, dtype=float)
+
+hot = np.where(t > n - 20, (t - (n - 20)) * 1.6, 0.0) # the last twenty minutes
+ingest("engine_temperature", idx,
+ 88 + 4*np.sin(t/90) + np.random.normal(0, 0.8, n) + hot, unit="celsius")
+ingest("engine_rpm", idx,
+ 1500 + 60*np.sin(t/140) + np.random.normal(0, 12, n), unit="rpm")
+ingest("engine_oil_pressure", idx,
+ 320 - 0.4*hot + np.random.normal(0, 4, n), unit="kpa")
+```
+
+Check that the temperature really does cross the limit the events guide uses, before
+running that guide and concluding it does not work:
+
+```python
+print(f"peak {(88 + 4*np.sin(t/90) + hot).max():.1f} °C") # > 110
+```
+
+## B. A cooling system, two sensors and two alarms
+
+[Correlate alarms](/guides/correlate-alarms) walks outward from two alarmed sensors and
+looks for what they have in common. That only works if the chain exists, so this builds
+`sensor_a` and `sensor_b` three hops below a shared `cooling_system`, and raises one
+alarm on each. The relationship name has to match the one the guide filters on, `PART_OF`,
+or the traversal comes back empty and the guide looks broken.
+
+```python
+nodes = [
+ intellistream_datahub_sdk.Resource(external_id="cooling_system", name="Cooling system", labels=["System"]),
+ intellistream_datahub_sdk.Resource(external_id="skid_1", name="Cooling skid 1", labels=["Skid"]),
+ intellistream_datahub_sdk.Resource(external_id="pump_a", name="Cooling pump A", labels=["Pump"]),
+ intellistream_datahub_sdk.Resource(external_id="pump_b", name="Cooling pump B", labels=["Pump"]),
+ intellistream_datahub_sdk.Resource(external_id="sensor_a", name="Pump A discharge sensor", labels=["Sensor"]),
+ intellistream_datahub_sdk.Resource(external_id="sensor_b", name="Pump B discharge sensor", labels=["Sensor"]),
+]
+edges = [
+ intellistream_datahub_sdk.RelForm.by_external_ids("skid_1", "cooling_system", "PART_OF"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("pump_a", "skid_1", "PART_OF"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("pump_b", "skid_1", "PART_OF"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("sensor_a", "pump_a", "PART_OF"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("sensor_b", "pump_b", "PART_OF"),
+]
+
+try:
+ graph = client.resources.create(nodes, edges)
+ print(len(graph.nodes), "nodes,", len(graph.relations), "edges")
+except intellistream_datahub_sdk.DataHubException as e:
+ if e.status_code != 409: # a second run finds the graph already there
+ raise
+```
+
+Two alarms, seconds apart, one on each sensor. They look like two incidents and are one.
+
+```python
+now = pd.Timestamp.now(tz="UTC")
+client.events.create([
+ intellistream_datahub_sdk.Event(external_id=f"alarm_sensor_a_{int(now.timestamp())}",
+ type="alarm", status="open", event_time=now,
+ metadata={"resource": "sensor_a", "reason": "high discharge temperature"}),
+ intellistream_datahub_sdk.Event(external_id=f"alarm_sensor_b_{int(now.timestamp())}",
+ type="alarm", status="open", event_time=now + pd.Timedelta(seconds=8),
+ metadata={"resource": "sensor_b", "reason": "high discharge temperature"}),
+])
+```
+
+:::tip Attach the alarm to the resource properly
+The metadata above is enough for the correlation walk, which starts from the sensors. In
+production an alarm should carry `relatedResources` so the link is a real edge rather than a
+string somebody has to parse. [Events reference →](/reference/events)
+:::
+
+## Check the sandbox is populated
+
+```python
+points = client.timeseries.retrieve_datapoints(intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=1),
+ end=pd.Timestamp.now(tz="UTC")))[0].get_datapoints()
+
+alarms = client.events.filter(type="alarm", external_id="alarm_sensor_*", limit=10)
+
+print(f"{len(points)} readings in the last hour, {len(alarms)} alarms")
+# 60 readings in the last hour, 2 alarms (2 more for each re-run of the alarm block)
+```
+
+If both numbers are non-zero, every guide on this site has something to run against.
+
+## See also
+
+- [Generate sample data](/advanced/generate-sample-data): the equivalent for the advanced scenarios.
+- [Sustained-condition alarms](/advanced/sustained-alarm-window): a full example that seeds, runs and then verifies itself.
+- [High-throughput ingestion](/guides/ingest-timeseries): doing this at production volume.
diff --git a/versioned_docs/version-1.0/guides/work-with-units.mdx b/versioned_docs/version-1.0/guides/work-with-units.mdx
new file mode 100644
index 0000000..89003e0
--- /dev/null
+++ b/versioned_docs/version-1.0/guides/work-with-units.mdx
@@ -0,0 +1,154 @@
+---
+sidebar_position: 7
+title: Work with units of measure
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Work with units of measure
+
+Units are shared reference data: `temperature_deg_c`, `pressure_bar`, `mass_flow_rate_kghr`.
+Tagging a time series with a unit makes its values self-describing: a chart can label its
+axis, and a consumer knows that `248.6` is bar, not psi. Browse the catalogue, then
+reference a unit by its external id when you create a series.
+
+## Browse the catalogue
+
+
+
+
+```java
+client.units().list().getItems().forEach(u ->
+ System.out.println(u.getExternalId() + " " + u.getName() + " (" + u.getSymbol() + ") - " + u.getQuantity()));
+```
+
+
+
+
+```python
+for u in client.units.list():
+ print(u.external_id, u.name, u.symbol, "-", u.quantity)
+```
+
+
+
+
+```rust
+for u in api.units.list().await?.get_items() {
+ println!("{} {} ({}) - {}", u.external_id, u.name, u.symbol, u.quantity);
+}
+```
+
+
+
+
+## Tag a series with a unit
+
+A series carries two unit fields. `unit` is free text of at most 64 characters, the label a
+chart shows (`bar`, `°C`). `unitExternalId` is the catalogue key (`pressure_bar`); set it when a
+consumer should be able to resolve the unit's symbol, quantity and conversion. A create needs at
+least one of them:
+
+| You send | Result |
+| --- | --- |
+| `unitExternalId` only | Accepted if the catalogue has it; `unit` is filled from the entry's symbol (its name if it has no symbol). |
+| `unit` and `unitExternalId` | Both kept as sent, as long as the catalogue has the `unitExternalId`. |
+| `unit` only | Accepted as is; the catalogue is not consulted. |
+| A blank `unitExternalId`, or one the catalogue does not have | `400`, with a field error on `unitExternalId`, whether or not `unit` is set. |
+| Neither | `400`. |
+
+The lookup normalises the id to lower snake_case first, so `Temperature_Deg_C` finds
+`temperature_deg_c`. An update that sets `unitExternalId` is checked the same way before
+anything is written; clearing it with `setNull` is still allowed. The unit leaves the value type
+alone (the Python and Rust examples set `float`); it describes what the numbers mean.
+
+
+
+
+```java
+var series = new Timeseries();
+series.setExternalId("wellhead_pressure_bar");
+series.setName("Wellhead pressure");
+series.setUnit("bar");
+series.setUnitExternalId("pressure_bar");
+client.timeseries().create(series);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="wellhead_pressure_bar",
+ name="Wellhead pressure",
+ unit="bar",
+ unit_external_id="pressure_bar",
+ value_type="float")]) # Python's default is bigint
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+let mut ts = TimeSeries::new("wellhead_pressure_bar", "Wellhead pressure");
+ts.unit = Some("bar".into());
+ts.unit_external_id = Some("pressure_bar".into());
+ts.set_value_type("float");
+api.time_series.create_one(&ts).await?;
+```
+
+
+
+
+## Look up a specific unit
+
+Resolve a unit you already know, by external id, or by numeric id.
+
+
+
+
+`byIds` takes `IdCollection`, by external id or by numeric id:
+
+```java
+UnitModel celsius = client.units().byIds(List.of(
+ IdCollection.createFromExternalId("temperature_deg_c"))).getItems().get(0);
+UnitModel byId = client.units().byIds(List.of(
+ IdCollection.createFromId(7))).getItems().get(0);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+celsius = client.units.by_external_id("temperature_deg_c")
+by_id = client.units.by_ids([intellistream_datahub_sdk.IdCollection(id=7)])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+
+let celsius = api.units.by_external_id("temperature_deg_c").await?;
+let by_id = api.units.by_ids(&DataWrapper::from(vec![IdAndExtId::from_id(7)])).await?;
+```
+
+
+
+
+:::note Unit external ids are `_` in snake_case
+The seeded unit catalogue names each unit by its quantity and symbol: `temperature_deg_c` for
+degrees Celsius, `pressure_bar` for bar, `mass_flow_rate_kghr` for kilograms per hour. Browse
+the catalogue to find the exact id. That is a property of the catalogue, not a rule about
+external ids in general: the ones *you* create are
+[stored exactly as you send them](/reference/external-ids).
+:::
+
+See the [Units reference](/reference/units) for the full method list.
diff --git a/versioned_docs/version-1.0/industries/_category_.json b/versioned_docs/version-1.0/industries/_category_.json
new file mode 100644
index 0000000..8b1eb0e
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/_category_.json
@@ -0,0 +1,8 @@
+{
+ "label": "Industry scenarios",
+ "position": 6,
+ "link": {
+ "type": "doc",
+ "id": "industries/overview"
+ }
+}
diff --git a/versioned_docs/version-1.0/industries/agriculture-food/_category_.json b/versioned_docs/version-1.0/industries/agriculture-food/_category_.json
new file mode 100644
index 0000000..333e8ac
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/agriculture-food/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Agriculture & food",
+ "position": 8,
+ "link": { "type": "generated-index", "title": "Agriculture & food" }
+}
diff --git a/versioned_docs/version-1.0/industries/agriculture-food/precision-farming.mdx b/versioned_docs/version-1.0/industries/agriculture-food/precision-farming.mdx
new file mode 100644
index 0000000..713b248
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/agriculture-food/precision-farming.mdx
@@ -0,0 +1,198 @@
+---
+sidebar_position: 1
+title: Agriculture — precision farming
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Agriculture — precision farming
+
+**The problem.** A farm is a patchwork of fields, each with soil sensors streaming
+moisture and temperature, and each periodically photographed from a drone or satellite
+to spot disease and growth variation. The agronomist needs the sensor trends *and* the
+imagery in one place, tied to the field they belong to, so a dry-soil reading and a
+stressed-crop image line up.
+
+This scenario is the one where **files** matter as much as numbers: attach imagery and
+scouting reports to the field, alongside its sensor series.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the field's soil series with two days of
+readings that dry out toward the irrigation threshold. (The drone-scan upload in step 2
+needs a local image file — any file works.) Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+for s, u in [("field_north_40_soil_moisture", "pct"), ("field_north_40_soil_temp_c", "deg_c")]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(value_type="float", external_id=s, name=s, unit=u)])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=48, freq="1h")
+moisture = 35 - np.linspace(0, 15, 48) + np.random.normal(0, 1, 48) # drying out
+client.timeseries.insert_from_lists(timestamps=idx, values=moisture, ts="field_north_40_soil_moisture")
+```
+
+## 1. Model fields and stream soil sensors
+
+A farm *contains* fields; each field's sensors are series.
+
+
+
+
+```java
+var moisture = new Timeseries();
+moisture.setExternalId("field_north_40_soil_moisture");
+moisture.setName("North 40 soil moisture");
+moisture.setUnit("pct");
+var soilTemp = new Timeseries();
+soilTemp.setExternalId("field_north_40_soil_temp_c");
+soilTemp.setName("North 40 soil temperature");
+soilTemp.setUnit("deg_c");
+client.timeseries().create(moisture, soilTemp);
+
+client.timeseries().ingest(Map.of(
+ "field_north_40_soil_moisture", moistureReadings)); // List
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.timeseries.create([
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="field_north_40_soil_moisture",
+ name="North 40 soil moisture", unit="pct"),
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="field_north_40_soil_temp_c",
+ name="North 40 soil temperature", unit="deg_c")])
+client.timeseries.insert_from_lists(
+ timestamps=timestamps, values=moisture, ts="field_north_40_soil_moisture")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+let mut moisture = TimeSeries::new("field_north_40_soil_moisture", "North 40 soil moisture");
+moisture.unit = Some("pct".into());
+api.time_series.create_one(&moisture).await?;
+
+let mut soil_temp = TimeSeries::new("field_north_40_soil_temp_c", "North 40 soil temperature");
+soil_temp.unit = Some("deg_c".into());
+api.time_series.create_one(&soil_temp).await?;
+
+api.time_series
+ .insert_datapoint(None, Some("field_north_40_soil_moisture".into()), ts, "21.5".into())
+ .await?;
+```
+
+
+
+
+## 2. Attach the drone imagery to the field
+
+Upload each scan into the field's folder, tagged so it's discoverable next to the
+field's sensor data. See [Attach files to assets](/guides/attach-files) for the upload
+details.
+
+
+
+
+```java
+byte[] scan = Files.readAllBytes(Path.of("north_40_2026_06_28.tif"));
+
+client.files().upload(
+ FileUploadRequest.builder()
+ .path("fields/north_40/scans/2026_06_28.tif")
+ .content(scan)
+ .contentType("image/tiff")
+ .externalId("scan_north_40_2026_06_28")
+ .description("Drone NDVI scan — North 40")
+ .source("drone_survey")
+ .build());
+```
+
+
+
+
+```python
+client.files.upload_file(intellistream_datahub_sdk.FileUpload(
+ path="north_40_2026_06_28.tif", # local file
+ destination_path="/fields/north_40/scans/",
+ external_id="scan_north_40_2026_06_28",
+ name="2026_06_28.tif"))
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::files::FileUpload;
+
+let mut upload = FileUpload::new_with_destination_path(
+ "north_40_2026_06_28.tif", "/fields/north_40/scans/")?;
+upload.external_id = "scan_north_40_2026_06_28".into();
+upload.name = "2026_06_28.tif".into();
+api.files.upload_file(upload).await?;
+```
+
+
+
+
+Later, list a field's folder to pull up its scan history alongside the soil trend:
+
+
+
+
+```java
+client.files().list("/fields/north_40/scans")
+ .getItems().forEach(node -> System.out.println(node.getName()));
+```
+
+
+
+
+```python
+for node in client.files.list_directory_by_path("/fields/north_40/scans"):
+ print(node.name)
+```
+
+
+
+
+```rust
+let scans = api.files.list_directory_by_path("/fields/north_40/scans").await?;
+for node in scans.get_items() { println!("{}", node.name); }
+```
+
+
+
+
+## 3. Trigger irrigation on a dry trend
+
+When soil moisture stays below target through the heat of the day, raise an
+`irrigation_needed` [event](/guides/detect-events) the irrigation controller acts on.
+
+## See the result
+
+Chart the soil moisture — the downward trend is what triggers irrigation:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="field_north_40_soil_moisture",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=2), end=pd.Timestamp.now(tz="UTC"))
+s = pd.Series({pd.to_datetime(p.timestamp): float(p.value)
+ for p in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()}).sort_index()
+s.plot(title="North 40 soil moisture (%)"); plt.show()
+```
+
+## See also
+
+- [Attach files to assets](/guides/attach-files) — uploading and listing imagery.
+- [High-throughput ingestion](/guides/ingest-timeseries) — soil-sensor data.
+- [Turn readings into events](/guides/detect-events) — the irrigation rule.
diff --git a/versioned_docs/version-1.0/industries/agriculture-food/salmon-farming.mdx b/versioned_docs/version-1.0/industries/agriculture-food/salmon-farming.mdx
new file mode 100644
index 0000000..d6ecda7
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/agriculture-food/salmon-farming.mdx
@@ -0,0 +1,189 @@
+---
+sidebar_position: 2
+title: Aquaculture — salmon farming
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Aquaculture — salmon farming
+
+**The problem.** In a sea-pen salmon farm, the margin between a healthy harvest and a
+catastrophe is thin and fast. Dissolved oxygen can fall through a critical level on a
+warm, still afternoon and suffocate an entire pen within hours. Sea lice spread between
+pens and trip regulatory limits. And feed — the single largest cost — is wasted the
+moment fish stop eating, fouling the water as it sinks. Farmers need to **see a pen
+turning dangerous in time to act**: aerate before an oxygen crash, treat before lice
+escalate, and stop feeding the moment appetite drops.
+
+What we solve here is preventing mass mortality and protecting feed margin on a farm
+where conditions change by the minute.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the pen's oxygen/temperature series,
+**create the subscription before we listen**, and feed in oxygen that falls below the
+6 mg/L danger line — so the loop raises a `low_oxygen` alarm. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+for s, u in [("pen_h_07_dissolved_oxygen_mg_l", "mg_l"), ("pen_h_07_water_temp_c", "deg_c")]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=s, name=s, unit=u, value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="farm_hardanger_pens", name="Hardanger pens",
+ timeseries=["pen_h_07_dissolved_oxygen_mg_l"])])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=120, freq="1min")
+client.timeseries.insert_from_lists(timestamps=idx, values=8 - np.linspace(0, 3, 120),
+ ts="pen_h_07_dissolved_oxygen_mg_l")
+```
+
+## 1. Watch pen conditions live — and aerate before a crash
+
+Dissolved oxygen and temperature per pen stream into a subscription. The instant
+oxygen dips toward the danger line, raise a `low_oxygen` alarm so aerators start while
+there's still time. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("farm_hardanger_pens"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (oxygenBelow(msg.payload(), 6.0)) { // mg/L — danger approaching
+ EventModel alarm = new EventModel();
+ alarm.setExternalId("low_oxygen_h07_" + System.currentTimeMillis());
+ alarm.setType("low_oxygen");
+ alarm.setStatus("critical");
+ alarm.setMetadata(Map.of("pen", "pen_h_07", "do_mg_l", "5.6"));
+ alarm.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(alarm)); // triggers aeration
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["farm_hardanger_pens"]) as listener:
+ for msg in listener:
+ if oxygen_below(msg.payload, 6.0): # mg/L — danger approaching
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"low_oxygen_h07_{int(pd.Timestamp.now().timestamp())}",
+ type="low_oxygen", status="critical",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"pen": "pen_h_07", "do_mg_l": "5.6"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["farm_hardanger_pens"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if oxygen_below(&msg.payload, 6.0) { // mg/L — danger approaching
+ let mut alarm = Event::new(
+ format!("low_oxygen_h07_{}", Utc::now().timestamp()),
+ "low_oxygen".into(),
+ Utc::now());
+ alarm.status = Some("critical".into());
+ alarm.add_metadata("pen".into(), "pen_h_07".into());
+ alarm.add_metadata("do_mg_l".into(), "5.6".into());
+ api.events.create(&vec![alarm]).await?; // triggers aeration
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Protect feed margin
+
+Feed dispensed and biomass are series per pen. Roll them up to a feed-conversion ratio
+(feed in vs growth) so a pen quietly going off its feed shows up before the cost does.
+See [Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var feed = new Timeseries();
+feed.setExternalId("pen_h_07_feed_kg");
+feed.setName("Pen H-07 feed");
+feed.setUnit("kg");
+var biomass = new Timeseries();
+biomass.setExternalId("pen_h_07_biomass_kg");
+biomass.setName("Pen H-07 biomass");
+biomass.setUnit("kg");
+client.timeseries().create(feed, biomass);
+
+client.timeseries().ingest(Map.of(
+ "pen_h_07_feed_kg", List.of(Datapoint.of(Instant.now(), 420.0))));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.timeseries.create([
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="pen_h_07_feed_kg", name="Pen H-07 feed", unit="kg"),
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="pen_h_07_biomass_kg", name="Pen H-07 biomass", unit="kg")])
+client.timeseries.insert_from_lists(
+ timestamps=[pd.Timestamp.now(tz="UTC")], values=[420.0], ts="pen_h_07_feed_kg")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use chrono::Utc;
+
+let mut feed = TimeSeries::new("pen_h_07_feed_kg", "Pen H-07 feed");
+feed.unit = Some("kg".into());
+api.time_series.create_one(&feed).await?;
+
+let mut biomass = TimeSeries::new("pen_h_07_biomass_kg", "Pen H-07 biomass");
+biomass.unit = Some("kg".into());
+api.time_series.create_one(&biomass).await?;
+
+api.time_series
+ .insert_datapoint(None, Some("pen_h_07_feed_kg".into()), Utc::now(), "420.0".into())
+ .await?;
+```
+
+
+
+
+## 3. Keep the compliance record
+
+Sea-lice counts and any mortality are `lice_count` and `mortality` events tied to the
+pen — the auditable record the regulator and the vet both need.
+
+## See the result
+
+The falling oxygen trips the loop before a crash:
+
+```text
+low_oxygen_h07_… → critical (dissolved oxygen fell below 6 mg/L)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the live pen-monitoring loop.
+- [Turn readings into events](/guides/detect-events) — the low-oxygen and lice rules.
+- [Query & aggregate](/guides/query-and-aggregate) — feed-conversion roll-ups.
+- **[Early warning (advanced)](/advanced/oxygen-crash-early-warning)** — predict the oxygen crash *before* it happens, instead of reacting to it.
diff --git a/versioned_docs/version-1.0/industries/built-environment/_category_.json b/versioned_docs/version-1.0/industries/built-environment/_category_.json
new file mode 100644
index 0000000..c4e383d
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/built-environment/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Built environment",
+ "position": 10,
+ "link": { "type": "generated-index", "title": "Built environment" }
+}
diff --git a/versioned_docs/version-1.0/industries/built-environment/construction.mdx b/versioned_docs/version-1.0/industries/built-environment/construction.mdx
new file mode 100644
index 0000000..85b1745
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/built-environment/construction.mdx
@@ -0,0 +1,189 @@
+---
+sidebar_position: 1
+title: Construction — site & safety
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Construction — site operations, safety & documents
+
+**The problem.** A construction site runs on documents as much as data: drawings,
+permits, method statements, inspection sign-offs — each belonging to a specific zone of
+the site, each needing to be on hand the moment an inspector or crew lead asks. On top
+of that, equipment utilisation and safety incidents need tracking. The challenge is
+keeping the paperwork, the machines, and the incidents all tied to the right part of
+the site.
+
+This scenario pairs **document management** with safety events on a modelled site.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a crane-utilisation series; step 1
+builds the site graph itself. (The drawing upload in step 1 needs a local file — any PDF works.)
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="crane_02_hours", name="Crane 02 hours", unit="h", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=14, freq="1d")
+client.timeseries.insert_from_lists(timestamps=idx, values=np.random.uniform(4, 9, 14), ts="crane_02_hours")
+```
+
+## 1. Model the site and attach its documents
+
+A site *contains* zones. Each zone's drawings and permits live in its folder, tagged so
+a crew can find the current revision instantly. See [Attach files](/guides/attach-files).
+
+
+
+
+```java
+Resource site = new Resource();
+site.setExternalId("site_harbour_tower");
+site.setName("Harbour Tower");
+site.setLabels(List.of("Site"));
+
+Resource zone = new Resource();
+zone.setExternalId("zone_level_12");
+zone.setName("Level 12");
+zone.setLabels(List.of("Zone"));
+
+RelForm contains = new RelForm();
+contains.setName("contains");
+contains.setFromExternalId("site_harbour_tower");
+contains.setToExternalId("zone_level_12");
+
+client.resources().create(List.of(site, zone), List.of(contains));
+
+byte[] drawing = Files.readAllBytes(Path.of("level_12_rev_c.pdf"));
+client.files().upload(
+ FileUploadRequest.builder()
+ .path("harbour_tower/level_12/drawings/structural_rev_c.pdf")
+ .content(drawing)
+ .contentType("application/pdf")
+ .externalId("drawing_level_12_structural_rev_c")
+ .description("Level 12 structural — revision C")
+ .build());
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id="site_harbour_tower", name="Harbour Tower", labels=["Site"]),
+ intellistream_datahub_sdk.Resource(external_id="zone_level_12", name="Level 12", labels=["Zone"])],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("site_harbour_tower", "zone_level_12", "contains")])
+
+client.files.upload_file(intellistream_datahub_sdk.FileUpload(
+ path="level_12_rev_c.pdf",
+ destination_path="/harbour_tower/level_12/drawings/",
+ external_id="drawing_level_12_structural_rev_c",
+ name="structural_rev_c.pdf"))
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+use intellistream_datahub_sdk::files::FileUpload;
+
+let mut site = Resource::new();
+site.external_id = "site_harbour_tower".into();
+site.name = "Harbour Tower".into();
+site.labels = Some(vec!["Site".into()]);
+let mut zone = Resource::new();
+zone.external_id = "zone_level_12".into();
+zone.name = "Level 12".into();
+zone.labels = Some(vec!["Zone".into()]);
+
+api.resources.create(
+ vec![site, zone],
+ vec![RelForm::by_external_ids("site_harbour_tower", "zone_level_12", "contains")]).await?;
+
+let mut upload = FileUpload::new_with_destination_path(
+ "level_12_rev_c.pdf", "/harbour_tower/level_12/drawings/")?;
+upload.external_id = "drawing_level_12_structural_rev_c".into();
+upload.name = "structural_rev_c.pdf".into();
+api.files.upload_file(upload).await?;
+```
+
+
+
+
+## 2. Record a safety incident against its zone
+
+A near-miss or incident becomes a `safety_incident` event, tied to the zone via
+metadata so the HSE team can see a zone's full history and trend.
+
+
+
+
+```java
+EventModel incident = new EventModel();
+incident.setExternalId("safety_incident_l12_" + System.currentTimeMillis());
+incident.setType("safety_incident");
+incident.setSubType("near_miss");
+incident.setStatus("open");
+incident.setMetadata(Map.of("zone", "zone_level_12", "category", "dropped_object"));
+incident.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(incident));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"safety_incident_l12_{int(pd.Timestamp.now().timestamp())}",
+ type="safety_incident", sub_type="near_miss", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"zone": "zone_level_12", "category": "dropped_object"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut incident = Event::new(
+ format!("safety_incident_l12_{}", Utc::now().timestamp()),
+ "safety_incident".into(),
+ Utc::now());
+incident.sub_type = Some("near_miss".into());
+incident.status = Some("open".into());
+incident.add_metadata("zone".into(), "zone_level_12".into());
+incident.add_metadata("category".into(), "dropped_object".into());
+api.events.create(&vec![incident]).await?;
+```
+
+
+
+
+## 3. Track equipment use
+
+Crane and hoist utilisation are series (`crane_02_hours`,
+`hoist_north_cycles`); roll them up to see idle plant and plan hire returns with
+[Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The recorded near-miss lands as an event against its zone:
+
+```text
+safety_incident_l12_… → open (dropped-object near-miss on zone_level_12)
+```
+
+## See also
+
+- [Attach files to assets](/guides/attach-files) — drawings, permits, sign-offs.
+- [Turn readings into events](/guides/detect-events) — recording incidents.
+- [Model assets as a graph](/guides/model-assets-graph) — site/zone/equipment structure.
diff --git a/versioned_docs/version-1.0/industries/built-environment/smart-buildings.mdx b/versioned_docs/version-1.0/industries/built-environment/smart-buildings.mdx
new file mode 100644
index 0000000..79c8750
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/built-environment/smart-buildings.mdx
@@ -0,0 +1,193 @@
+---
+sidebar_position: 2
+title: Smart buildings — BMS & energy
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Smart buildings — BMS & energy
+
+**The problem.** A commercial building's management system runs hundreds of points —
+zone temperatures, CO₂, air-handler status, sub-metered energy. Facilities teams want
+two things at once: a live comfort view that flags a stuffy or overheating zone
+*before* the complaints come in, and an energy picture that shows where the kilowatt-
+hours actually go so they can cut waste.
+
+This scenario pairs **live comfort monitoring** with **energy aggregation** on the same
+building model.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a zone's CO₂ series, **create the
+subscription before we listen** (with a stuffy spell), and an energy series for the
+roll-up. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="zone_l8_co2_ppm", name="L8 open-plan CO2", unit="ppm", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="tower_a_comfort", name="Tower A comfort", timeseries=["zone_l8_co2_ppm"])])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=60, freq="1min")
+co2 = np.full(60, 600.0); co2[-10:] = 1180.0 # out of comfort band
+client.timeseries.insert_from_lists(timestamps=idx, values=co2, ts="zone_l8_co2_ppm")
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="tower_a_l8_energy_kwh", name="Tower A L8 energy", unit="kwh", value_type="float")])
+idx2 = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=30, freq="1d")
+client.timeseries.insert_from_lists(timestamps=idx2, values=np.random.uniform(200, 400, 30), ts="tower_a_l8_energy_kwh")
+```
+
+## 1. Watch comfort live
+
+Subscribe to the zone comfort series and react as readings land — raise a
+`comfort_alert` when a zone drifts out of band. See
+[Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("tower_a_comfort"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (outsideComfort(msg.payload())) { // temp or CO2 out of band
+ EventModel alert = new EventModel();
+ alert.setExternalId("comfort_alert_l8_" + System.currentTimeMillis());
+ alert.setType("comfort_alert");
+ alert.setStatus("open");
+ alert.setMetadata(Map.of("zone", "zone_l8_open_plan", "co2_ppm", "1180"));
+ alert.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(alert));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["tower_a_comfort"]) as listener:
+ for msg in listener:
+ if outside_comfort(msg.payload): # temp or CO2 out of band
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"comfort_alert_l8_{int(pd.Timestamp.now().timestamp())}",
+ type="comfort_alert", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"zone": "zone_l8_open_plan", "co2_ppm": "1180"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["tower_a_comfort"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if outside_comfort(&msg.payload) { // temp or CO2 out of band
+ let mut alert = Event::new(
+ format!("comfort_alert_l8_{}", Utc::now().timestamp()),
+ "comfort_alert".into(),
+ Utc::now());
+ alert.status = Some("open".into());
+ alert.add_metadata("zone".into(), "zone_l8_open_plan".into());
+ alert.add_metadata("co2_ppm".into(), "1180".into());
+ api.events.create(&vec![alert]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. See where the energy goes
+
+Sub-meters report per floor and system. Roll consumption up to daily totals per meter
+to rank the biggest users and spot overnight waste. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("tower_a_l8_energy_kwh");
+filter.setStart(ZonedDateTime.now().minusDays(30));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("sum"));
+filter.setGranularity("1d");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> chartDailyKwh(p.getTimestamp(), p.getValue()));
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="tower_a_l8_energy_kwh",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=30),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1d")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ chart_daily_kwh(dp.timestamp, dp.sum)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("tower_a_l8_energy_kwh".into()),
+ aggregates: Some(vec!["sum".into()]),
+ granularity: Some("1d".into()),
+ start: Some(Utc::now() - chrono::Duration::days(30)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { chart_daily_kwh(&p.timestamp, p.sum); }
+```
+
+
+
+
+:::tip Correlate comfort and equipment
+When several zones go uncomfortable together, walk the graph from each to the air
+handler they share — the same [alarm correlation](/guides/correlate-alarms) trick finds
+the one unit behind the complaints.
+:::
+
+## See the result
+
+The stuffy zone trips the loop before the complaints come in:
+
+```text
+comfort_alert_l8_… → open (CO₂ 1180 ppm, above the comfort band)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the comfort-monitoring loop.
+- [Query & aggregate](/guides/query-and-aggregate) — energy roll-ups.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — find a shared air handler.
diff --git a/versioned_docs/version-1.0/industries/energy-utilities/_category_.json b/versioned_docs/version-1.0/industries/energy-utilities/_category_.json
new file mode 100644
index 0000000..0d7b279
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/energy-utilities/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Energy & utilities",
+ "position": 2,
+ "link": { "type": "generated-index", "title": "Energy & utilities" }
+}
diff --git a/versioned_docs/version-1.0/industries/energy-utilities/air-quality.mdx b/versioned_docs/version-1.0/industries/energy-utilities/air-quality.mdx
new file mode 100644
index 0000000..5815dae
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/energy-utilities/air-quality.mdx
@@ -0,0 +1,155 @@
+---
+sidebar_position: 6
+title: Environmental — air quality
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Environmental — air-quality monitoring
+
+**The problem.** An air-quality network mixes sensor types: particulates in
+micrograms per cubic metre, nitrogen dioxide in parts per billion, carbon monoxide in
+parts per million. If a series doesn't carry its unit, `42` is meaningless — is that a
+healthy CO reading or a hazardous one? Getting **units** right, consistently, across a
+heterogeneous fleet is what makes the data comparable and the dashboards trustworthy.
+
+This is the scenario where **units** are the hero: pick the right unit for each
+pollutant and tag every series with it.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the station's pollutant series (each with
+its unit) and a day of readings — so the daily-average roll-up has data. The unit
+catalogue browsed in step 1 is built-in reference data. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+for s, u in [("station_kirkeveien_pm25", "ug_m3"), ("station_kirkeveien_no2", "ppb"),
+ ("station_kirkeveien_co", "ppm")]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(value_type="float", external_id=s, name=s, unit=u)])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=24, freq="1h")
+client.timeseries.insert_from_lists(timestamps=idx, values=np.random.uniform(5, 45, 24), ts="station_kirkeveien_pm25")
+```
+
+## 1. Find the right unit
+
+Browse the unit catalogue to get the canonical external id for each pollutant's unit.
+See the [Units reference](/reference/units).
+
+
+
+
+```java
+client.units().list().getItems().forEach(u ->
+ System.out.println(u.getExternalId() + " " + u.getName() + " (" + u.getSymbol() + ")"));
+// e.g. ug_m3 micrograms per cubic metre (µg/m³)
+```
+
+
+
+
+```python
+for u in client.units.list():
+ print(u.external_id, u.name, u.symbol)
+
+ugm3 = client.units.by_external_id("ug_m3") # confirm the one you'll use
+```
+
+
+
+
+```rust
+for u in api.units.list().await?.get_items() {
+ println!("{} {} ({})", u.external_id, u.name, u.symbol);
+}
+let ugm3 = api.units.by_external_id("ug_m3").await?;
+```
+
+
+
+
+## 2. Tag each series with its unit
+
+Create one series per pollutant per station, each carrying its unit's external id — so
+every value is self-describing and a chart can label its axis correctly.
+
+
+
+
+```java
+var pm25 = new Timeseries();
+pm25.setExternalId("station_kirkeveien_pm25");
+pm25.setName("Kirkeveien PM2.5");
+pm25.setUnit("ug_m3");
+var no2 = new Timeseries();
+no2.setExternalId("station_kirkeveien_no2");
+no2.setName("Kirkeveien NO2");
+no2.setUnit("ppb");
+var co = new Timeseries();
+co.setExternalId("station_kirkeveien_co");
+co.setName("Kirkeveien CO");
+co.setUnit("ppm");
+client.timeseries().create(pm25, no2, co);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.timeseries.create([
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="station_kirkeveien_pm25", name="Kirkeveien PM2.5", unit="ug_m3"),
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="station_kirkeveien_no2", name="Kirkeveien NO2", unit="ppb"),
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="station_kirkeveien_co", name="Kirkeveien CO", unit="ppm")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+for (id, name, unit) in [
+ ("station_kirkeveien_pm25", "Kirkeveien PM2.5", "ug_m3"),
+ ("station_kirkeveien_no2", "Kirkeveien NO2", "ppb"),
+ ("station_kirkeveien_co", "Kirkeveien CO", "ppm"),
+] {
+ let mut ts = TimeSeries::new(id, name);
+ ts.unit = Some(unit.into());
+ api.time_series.create_one(&ts).await?;
+}
+```
+
+
+
+
+## 3. Daily averages and exceedances
+
+Roll each pollutant up to daily averages for the public index, and raise an
+`exceedance` [event](/guides/detect-events) when a station crosses a regulatory limit —
+the metadata carrying the pollutant, its unit, and the threshold so the record is
+unambiguous. See [Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+Chart the day's PM2.5 — now self-describing, because the series carries its `ug_m3` unit:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="station_kirkeveien_pm25",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+vals = [dp.average for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.plot(vals); plt.title("Kirkeveien PM2.5 (µg/m³)"); plt.show()
+```
+
+## See also
+
+- [Units reference](/reference/units) — the unit catalogue and lookups.
+- [Work with units of measure](/guides/work-with-units) — tagging series with units.
+- [Query & aggregate](/guides/query-and-aggregate) — daily-average index.
diff --git a/versioned_docs/version-1.0/industries/energy-utilities/ev-charging.mdx b/versioned_docs/version-1.0/industries/energy-utilities/ev-charging.mdx
new file mode 100644
index 0000000..858e1ae
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/energy-utilities/ev-charging.mdx
@@ -0,0 +1,127 @@
+---
+sidebar_position: 3
+title: EV charging — network uptime
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# EV charging — network uptime & load
+
+**The problem.** A charging network's reputation is its **availability**: a driver who
+rolls up to a dead charger doesn't come back. At the same time, a bank of fast chargers
+all drawing at once can exceed a site's grid connection and trip it — taking every
+charger down. The operator needs to **know a charger has failed the moment it does**,
+and to keep total site draw inside its limit.
+
+What we solve here is uptime drivers can trust and grid limits the site never breaches.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a charger's status series, **create the
+subscription before we listen**, and push a fault reading — so the loop below raises a
+`charger_down`. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="charger_oslo_14_status", name="Charger Oslo 14 status", unit="status", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="network_status", name="Charging network status", timeseries=["charger_oslo_14_status"])])
+
+# status 1 = available; push a 0 (faulted) for the loop to catch
+client.timeseries.insert_from_lists(timestamps=[pd.Timestamp.now(tz="UTC")], values=[0.0], ts="charger_oslo_14_status")
+```
+
+## 1. Detect a dead charger immediately
+
+Each charge point reports status and power. A connector that stops responding or faults
+raises a `charger_down` event straight into the dispatch queue. See
+[Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("network_status"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (faulted(msg.payload())) {
+ EventModel down = new EventModel();
+ down.setExternalId("charger_down_oslo_14_" + System.currentTimeMillis());
+ down.setType("charger_down");
+ down.setStatus("open");
+ down.setMetadata(Map.of("charger", "charger_oslo_14", "connector", "2"));
+ down.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(down));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["network_status"]) as listener:
+ for msg in listener:
+ if faulted(msg.payload):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"charger_down_oslo_14_{int(pd.Timestamp.now().timestamp())}",
+ type="charger_down", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"charger": "charger_oslo_14", "connector": "2"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::events::Event;
+
+let mut listener = api.subscriptions.listen(&["network_status"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if faulted(&msg.payload) {
+ let mut down = Event::new(
+ format!("charger_down_oslo_14_{}", Utc::now().timestamp()),
+ "charger_down".into(),
+ Utc::now());
+ down.status = Some("open".into());
+ down.add_metadata("charger".into(), "charger_oslo_14".into());
+ down.add_metadata("connector".into(), "2".into());
+ api.events.create(&vec![down]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Report uptime and watch site load
+
+Availability and per-site power are series. Daily uptime per charger gives the
+network-quality number; the site's summed draw, aggregated to the minute, shows how
+close it runs to its grid limit — the input to load-balancing. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The faulted reading in the demo trips the loop, and a `charger_down` event lands:
+
+```python
+for e in client.events.filter(type="charger_down", limit=5):
+ print(e.external_id, e.metadata)
+# charger_down_oslo_14_… {'charger': 'charger_oslo_14', 'connector': '2'}
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — live charger status.
+- [Turn readings into events](/guides/detect-events) — the fault rule.
+- [Query & aggregate](/guides/query-and-aggregate) — uptime and site-load reporting.
diff --git a/versioned_docs/version-1.0/industries/energy-utilities/grid.mdx b/versioned_docs/version-1.0/industries/energy-utilities/grid.mdx
new file mode 100644
index 0000000..7decf19
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/energy-utilities/grid.mdx
@@ -0,0 +1,237 @@
+---
+sidebar_position: 1
+title: Energy — grid telemetry
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Energy — electrical grid telemetry
+
+**The problem.** A distribution operator runs a region of substations, each stepping
+power down through transformers onto feeders that serve neighbourhoods. Operators
+need second-by-second load, voltage and frequency; planners need hourly and daily
+roll-ups to spot trends and size upgrades; and any feeder pushed past its rating must
+raise an overload alarm.
+
+This scenario centres on **aggregation** — turning a firehose of telemetry into the
+roll-ups a SCADA dashboard and a planning report actually consume.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a feeder's load series with a day of
+hourly readings including an overload peak — so the roll-up and overload event below have
+data. The region/substation graph is created by step 1. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="feeder_f12_load_mw", name="Feeder F12 load", unit="mw")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=24, freq="1h")
+load = 8 + 4 * np.sin(np.arange(24) / 24 * 2 * np.pi) + np.random.normal(0, 0.5, 24)
+load[18] = 14.8 # an evening overload past the 12 MW rating
+client.timeseries.insert_from_lists(timestamps=idx, values=load, ts="feeder_f12_load_mw")
+```
+
+## 1. Model the network
+
+A region *contains* substations, a substation *contains* transformers, a transformer
+*feeds* feeders. The same graph pattern as any asset hierarchy.
+
+
+
+
+```java
+Resource region = new Resource();
+region.setExternalId("grid_region_east");
+region.setName("Eastern region");
+region.setLabels(List.of("Region"));
+
+Resource substation = new Resource();
+substation.setExternalId("substation_oslo_1");
+substation.setName("Oslo substation 1");
+substation.setLabels(List.of("Substation"));
+
+RelForm contains = new RelForm();
+contains.setName("contains");
+contains.setFromExternalId("grid_region_east");
+contains.setToExternalId("substation_oslo_1");
+
+client.resources().create(List.of(region, substation), List.of(contains));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id="grid_region_east", name="Eastern region", labels=["Region"]),
+ intellistream_datahub_sdk.Resource(external_id="substation_oslo_1", name="Oslo substation 1", labels=["Substation"])],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("grid_region_east", "substation_oslo_1", "contains")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut region = Resource::new();
+region.external_id = "grid_region_east".into();
+region.name = "Eastern region".into();
+region.labels = Some(vec!["Region".into()]);
+
+let mut substation = Resource::new();
+substation.external_id = "substation_oslo_1".into();
+substation.name = "Oslo substation 1".into();
+substation.labels = Some(vec!["Substation".into()]);
+
+api.resources.create(
+ vec![region, substation],
+ vec![RelForm::by_external_ids("grid_region_east", "substation_oslo_1", "contains")],
+).await?;
+```
+
+
+
+
+Per-feeder series follow the same naming: `feeder_f12_load_mw`, `feeder_f12_voltage_kv`,
+`substation_oslo_1_frequency_hz`.
+
+## 2. Roll telemetry up for the dashboard
+
+Raw load is far too dense to chart directly. Ask for **hourly averages and peaks**
+and the platform returns one value per bucket. The full mechanics are in
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("feeder_f12_load_mw");
+filter.setStart(ZonedDateTime.now().minusDays(1));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg", "max"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> chart(p.getTimestamp(), p.getValue())); // hourly load
+```
+
+
+
+
+```python
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="feeder_f12_load_mw",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg", "max"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ chart(dp.timestamp, dp.average, dp.max)
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+
+let filter = RetrieveFilter {
+ external_id: Some("feeder_f12_load_mw".into()),
+ start: Some(Utc::now() - chrono::Duration::days(1)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into(), "max".into()]),
+ granularity: Some("1h".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { chart(&p.timestamp, p.average, p.max); }
+```
+
+
+
+
+## 3. Overload alarm
+
+When a feeder's hourly peak exceeds its rating, record a `feeder_overload` event so
+the planning team sees a ranked list of strained circuits.
+
+
+
+
+```java
+EventModel event = new EventModel();
+event.setExternalId("feeder_overload_f12_" + System.currentTimeMillis());
+event.setType("feeder_overload");
+event.setStatus("open");
+event.setMetadata(Map.of("feeder", "feeder_f12", "peak_mw", "14.8", "rating_mw", "12.0"));
+event.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(event));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"feeder_overload_f12_{int(pd.Timestamp.now().timestamp())}",
+ type="feeder_overload", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"feeder": "feeder_f12", "peak_mw": "14.8", "rating_mw": "12.0"})])
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::events::Event;
+
+let mut event = Event::new(
+ format!("feeder_overload_f12_{}", Utc::now().timestamp()),
+ "feeder_overload".into(),
+ Utc::now());
+event.status = Some("open".into());
+event.add_metadata("feeder".into(), "feeder_f12".into());
+event.add_metadata("peak_mw".into(), "14.8".into());
+event.add_metadata("rating_mw".into(), "12.0".into());
+api.events.create(&vec![event]).await?;
+```
+
+
+
+
+## See the result
+
+Chart the feeder's hourly peak load — one bar pokes above the 12 MW rating line, which is
+the overload the event flags:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="feeder_f12_load_mw",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["max"], granularity="1h")
+peaks = [dp.max for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.bar(range(len(peaks)), peaks); plt.axhline(12, color="r", ls="--")
+plt.title("Feeder F12 hourly peak load (MW)"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate time-series](/guides/query-and-aggregate) — granularity and aggregate options.
+- [Model assets as a graph](/guides/model-assets-graph) — region/substation/feeder topology.
+- [Turn readings into events](/guides/detect-events) — the overload rule.
diff --git a/versioned_docs/version-1.0/industries/energy-utilities/waste.mdx b/versioned_docs/version-1.0/industries/energy-utilities/waste.mdx
new file mode 100644
index 0000000..db1af23
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/energy-utilities/waste.mdx
@@ -0,0 +1,121 @@
+---
+sidebar_position: 5
+title: Waste management — smart collection
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Waste management — smart collection
+
+**The problem.** Fixed collection schedules get it wrong in both directions: bins in
+busy spots overflow between visits — a health and litter problem — while trucks burn
+fuel and hours emptying half-full bins on quiet streets. The operator needs collection
+**driven by how full bins actually are**, so crews go where the waste is and skip where
+it isn't.
+
+What we solve here is collecting on demand instead of on a calendar — fewer overflows,
+fewer wasted trips.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a bin's fill-level series with a near-full
+reading — so the `collection_due` rule below fires. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="bin_grunerlokka_114_fill_pct", name="Bin 114 fill", unit="pct", value_type="float")])
+client.timeseries.insert_from_lists(timestamps=[pd.Timestamp.now(tz="UTC")], values=[87.0], ts="bin_grunerlokka_114_fill_pct")
+```
+
+## 1. Act on fill level, not the calendar
+
+Each bin reports its fill level. When a bin crosses its collection threshold, raise a
+`collection_due` event that drops it onto the next route. See
+[Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+var series = client.timeseries().retrieve(lastReadingOf("bin_grunerlokka_114_fill_pct"))
+ .getItems().get(0);
+
+double fill = Double.parseDouble(series.getDatapoints().get(0).getValue());
+if (fill >= 80.0) {
+ EventModel due = new EventModel();
+ due.setExternalId("collection_due_114_" + System.currentTimeMillis());
+ due.setType("collection_due");
+ due.setStatus("open");
+ due.setMetadata(Map.of("bin", "bin_grunerlokka_114", "fill_pct", String.valueOf(fill)));
+ due.setEventTime(Long.parseLong(series.getDatapoints().get(0).getTimestamp()));
+ client.events().create(List.of(due));
+}
+```
+
+
+
+
+```python
+points = client.timeseries.retrieve_datapoints(last_reading("bin_grunerlokka_114_fill_pct"))[0]
+fill = float(points.get_datapoints()[0].value)
+if fill >= 80.0:
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"collection_due_114_{int(pd.Timestamp.now().timestamp())}",
+ type="collection_due", status="open",
+ event_time=points.get_datapoints()[0].timestamp,
+ metadata={"bin": "bin_grunerlokka_114", "fill_pct": str(fill)})])
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::events::Event;
+
+let page = api.time_series.retrieve_datapoints(&last_reading("bin_grunerlokka_114_fill_pct")).await?;
+let points = &page.get_items()[0];
+let fill: f64 = points.datapoints[0].value.unwrap_or(0.0);
+if fill >= 80.0 {
+ let mut due = Event::new(
+ format!("collection_due_114_{}", Utc::now().timestamp()),
+ "collection_due".into(),
+ points.datapoints[0].timestamp);
+ due.status = Some("open".into());
+ due.add_metadata("bin".into(), "bin_grunerlokka_114".into());
+ due.add_metadata("fill_pct".into(), fill.to_string());
+ api.events.create(&vec![due]).await?;
+}
+```
+
+
+
+
+## 2. Plan routes from real demand
+
+Fill levels across a district, rolled up over the week, show the fill-rate pattern —
+which areas fill fast and need frequent visits, which can be left longer. That pattern
+sizes routes and frequencies far better than a flat schedule. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+## 3. Flag contamination
+
+A bin reading the wrong weight or material for its stream becomes a `contamination`
+event for the education and enforcement team.
+
+## See the result
+
+The near-full bin in the demo trips the rule:
+
+```text
+collection_due bin_grunerlokka_114 (fill 87%)
+```
+
+## See also
+
+- [Turn readings into events](/guides/detect-events) — the collection-due rule.
+- [Query & aggregate](/guides/query-and-aggregate) — fill-rate patterns for routing.
+- [High-throughput ingestion](/guides/ingest-timeseries) — city-wide sensor volume.
diff --git a/versioned_docs/version-1.0/industries/energy-utilities/water.mdx b/versioned_docs/version-1.0/industries/energy-utilities/water.mdx
new file mode 100644
index 0000000..148bce2
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/energy-utilities/water.mdx
@@ -0,0 +1,164 @@
+---
+sidebar_position: 4
+title: Water utilities — distribution
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Water utilities — distribution & quality
+
+**The problem.** A water network flows from sources through trunk mains into district
+zones and out to meters. When two distant zones report the same quality problem —
+turbidity, low chlorine — at nearly the same time, the operator must find the **common
+upstream source** fast, because that's where to isolate and flush. With a flat sensor
+list it's guesswork. On the flow graph, it's the point where the two zones' upstream
+paths converge.
+
+This scenario introduces the **multi-source common-cause** graph pattern: trace
+upstream from each affected zone and intersect to find where the trouble enters.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the flow-network graph (a source feeding a
+trunk main, which feeds two zones) and a turbidity reading out of range in each zone — so
+the upstream-convergence trace finds their shared main. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[x]) for x in
+ ["source_lake_a", "main_trunk_north", "zone_west_12", "zone_east_07"]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("source_lake_a", "main_trunk_north", "feeds"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("main_trunk_north", "zone_west_12", "feeds"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("main_trunk_north", "zone_east_07", "feeds")])
+
+for s in ["zone_west_12_turbidity_ntu", "zone_east_07_turbidity_ntu"]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=s, name=s, unit="ntu", value_type="float")])
+ client.timeseries.insert_from_lists(timestamps=[pd.Timestamp.now(tz="UTC")], values=[7.5], ts=s)
+```
+
+## 1. Model the flow network
+
+Each `feeds` edge points downstream: source → main → zone. Walking *against* the flow
+(undirected traversal) recovers a zone's upstream supply.
+
+
+
+
+```java
+List flow = List.of(
+ rel("feeds", "source_lake_a", "main_trunk_north"),
+ rel("feeds", "main_trunk_north", "zone_west_12"),
+ rel("feeds", "main_trunk_north", "zone_east_07"));
+
+client.resources().create(assets, flow);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+flow = [
+ intellistream_datahub_sdk.RelForm.by_external_ids("source_lake_a", "main_trunk_north", "feeds"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("main_trunk_north", "zone_west_12", "feeds"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("main_trunk_north", "zone_east_07", "feeds"),
+]
+client.resources.create(assets, flow)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::relations::RelForm;
+
+let flow = vec![
+ RelForm::by_external_ids("source_lake_a", "main_trunk_north", "feeds"),
+ RelForm::by_external_ids("main_trunk_north", "zone_west_12", "feeds"),
+ RelForm::by_external_ids("main_trunk_north", "zone_east_07", "feeds"),
+];
+api.resources.create(assets, flow).await?;
+```
+
+
+
+
+## 2. Two zones report trouble — find where the flows meet
+
+Trace upstream from each affected zone and intersect the two supply paths. The shared
+node — here `main_trunk_north`, or beyond it `source_lake_a` — is where to isolate.
+
+
+
+
+```java
+ResourceNetwork west = client.resources().fetchRelated("zone_west_12", 20);
+ResourceNetwork east = client.resources().fetchRelated("zone_east_07", 20);
+
+Set westUpstream = west.nodes().stream()
+ .map(NodeModel::getExternalId).collect(Collectors.toSet());
+Set common = east.nodes().stream()
+ .map(NodeModel::getExternalId).filter(westUpstream::contains).collect(Collectors.toSet());
+
+// common holds main_trunk_north (and source_lake_a) → isolate there
+```
+
+
+
+
+```python
+west = client.resources.fetch_related(external_id="zone_west_12", depth=20)
+east = client.resources.fetch_related(external_id="zone_east_07", depth=20)
+
+common = ({n.external_id for n in west.nodes} & {n.external_id for n in east.nodes})
+# 'main_trunk_north' in common → isolate there
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+use std::collections::HashSet;
+
+let west = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("zone_west_12").with_depth(20)).await?;
+let east = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("zone_east_07").with_depth(20)).await?;
+
+let w: HashSet<&str> = west.nodes().iter().map(|n| n.external_id()).collect();
+let common: Vec<&str> = east.nodes().iter()
+ .map(|n| n.external_id()).filter(|id| w.contains(id)).collect();
+// common holds "main_trunk_north" → isolate there
+```
+
+
+
+
+This is the same shared-node reasoning as [alarm correlation](/guides/correlate-alarms),
+applied to a flow network to pinpoint a contamination source.
+
+## 3. Detect the problem in the first place
+
+Turbidity and chlorine are series per zone (`zone_west_12_turbidity_ntu`,
+`zone_west_12_chlorine_mg_l`); a reading out of range raises a `water_quality_alert`
+[event](/guides/detect-events) that kicks off the trace above.
+
+## See the result
+
+The two zones' upstream paths intersect at one node — where to isolate and flush:
+
+```text
+main_trunk_north ← the shared supply behind both quality alerts
+```
+
+## See also
+
+- [Correlate alarms with the graph](/guides/correlate-alarms) — upstream-convergence intersection.
+- [Model assets as a graph](/guides/model-assets-graph) — modeling the flow network.
+- [Turn readings into events](/guides/detect-events) — the quality alert.
diff --git a/versioned_docs/version-1.0/industries/energy-utilities/wind.mdx b/versioned_docs/version-1.0/industries/energy-utilities/wind.mdx
new file mode 100644
index 0000000..29319ad
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/energy-utilities/wind.mdx
@@ -0,0 +1,206 @@
+---
+sidebar_position: 2
+title: Renewable energy — wind farm
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Renewable energy — wind farm generation
+
+**The problem.** A wind farm's output swings with the weather minute to minute. The
+operations desk needs to know how much each turbine and the farm as a whole is
+generating against its rated capacity, spot under-performers, and feed a generation
+forecast to the grid operator. Raw per-second power is far too dense — what matters is
+the **rolled-up capacity factor** over useful windows.
+
+This scenario centres on **aggregation** of generation data into the hourly and daily
+figures a forecasting model and a performance dashboard consume.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a turbine's power and wind series with a
+week of hourly readings — so the capacity-factor roll-up has data. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+for s, u in [("turbine_t14_power_kw", "kw"), ("turbine_t14_wind_ms", "m_s")]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(value_type="float", external_id=s, name=s, unit=u)])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=168, freq="1h")
+wind = np.clip(np.random.weibull(2.0, 168) * 6.0, 0, 25)
+power = np.clip(wind**3 * 2.0, 0, 3500) # rated ~3500 kW
+client.timeseries.insert_from_lists(timestamps=idx, values=power, ts="turbine_t14_power_kw")
+client.timeseries.insert_from_lists(timestamps=idx, values=wind, ts="turbine_t14_wind_ms")
+```
+
+## 1. Ingest per-turbine output
+
+Each turbine reports power and wind speed continuously. Hand the readings off in bulk.
+
+
+
+
+```java
+var power = new Timeseries();
+power.setExternalId("turbine_t14_power_kw");
+power.setName("Turbine T14 power");
+power.setUnit("kw");
+var wind = new Timeseries();
+wind.setExternalId("turbine_t14_wind_ms");
+wind.setName("Turbine T14 wind speed");
+wind.setUnit("m_s");
+client.timeseries().create(power, wind);
+
+client.timeseries().ingest(Map.of(
+ "turbine_t14_power_kw", powerReadings, // List
+ "turbine_t14_wind_ms", windReadings));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.timeseries.create([
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="turbine_t14_power_kw",
+ name="Turbine T14 power", unit="kw"),
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="turbine_t14_wind_ms",
+ name="Turbine T14 wind speed", unit="m_s")])
+client.timeseries.insert_from_lists(
+ timestamps=timestamps, values=power_kw, ts="turbine_t14_power_kw")
+client.timeseries.insert_from_lists(
+ timestamps=timestamps, values=wind_ms, ts="turbine_t14_wind_ms")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use intellistream_datahub_sdk::generic::{DataWrapper, DatapointsCollection};
+
+let mut power = TimeSeries::new("turbine_t14_power_kw", "Turbine T14 power");
+power.unit = Some("kw".into());
+api.time_series.create_one(&power).await?;
+
+let mut wind = TimeSeries::new("turbine_t14_wind_ms", "Turbine T14 wind speed");
+wind.unit = Some("m_s".into());
+api.time_series.create_one(&wind).await?;
+
+let mut dw = DataWrapper::new();
+dw.add_item(DatapointsCollection {
+ external_id: Some("turbine_t14_power_kw".into()),
+ datapoints: power_readings, // Vec
+ ..Default::default()
+});
+dw.add_item(DatapointsCollection {
+ external_id: Some("turbine_t14_wind_ms".into()),
+ datapoints: wind_readings,
+ ..Default::default()
+});
+api.time_series.insert_datapoints(&mut dw).await?;
+```
+
+
+
+
+## 2. Roll up to a capacity factor
+
+Capacity factor is average output over rated capacity. Ask for hourly averages, divide
+by the turbine's rating, and you have the curve the forecast and the dashboard need.
+Full mechanics in [Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("turbine_t14_power_kw");
+filter.setStart(ZonedDateTime.now().minusDays(7));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+double rated = 3500.0; // kW
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> recordCapacityFactor(
+ p.getTimestamp(), Double.parseDouble(p.getValue()) / rated));
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="turbine_t14_power_kw",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=7),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+
+rated = 3500.0
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ record_capacity_factor(dp.timestamp, dp.average / rated)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("turbine_t14_power_kw".into()),
+ start: Some(Utc::now() - chrono::Duration::days(7)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into()]),
+ granularity: Some("1h".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+let rated = 3500.0;
+for p in &series.datapoints {
+ if let Some(avg) = p.average { record_capacity_factor(&p.timestamp, avg / rated); }
+}
+```
+
+
+
+
+## 3. Flag an under-performer
+
+When a turbine's capacity factor stays well below the farm average in good wind, raise
+a `turbine_underperformance` [event](/guides/detect-events) for the maintenance team —
+a sign of a pitch or gearbox problem worth a closer look.
+
+## See the result
+
+Print the average capacity factor and chart it over the week:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="turbine_t14_power_kw",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=7), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+cf = [dp.average / 3500 for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+print(f"average capacity factor: {sum(cf)/len(cf):.0%}")
+plt.plot(cf); plt.title("Turbine T14 capacity factor"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — windows and aggregates for the capacity factor.
+- [High-throughput ingestion](/guides/ingest-timeseries) — per-turbine data volume.
+- [Turn readings into events](/guides/detect-events) — the under-performance rule.
+- **[Demand forecasting (advanced)](/advanced/demand-forecasting)** — forecast generation the same way you'd forecast load.
diff --git a/versioned_docs/version-1.0/industries/financial-services/_category_.json b/versioned_docs/version-1.0/industries/financial-services/_category_.json
new file mode 100644
index 0000000..ac1a76f
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/financial-services/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Financial services",
+ "position": 5,
+ "link": { "type": "generated-index", "title": "Financial services" }
+}
diff --git a/versioned_docs/version-1.0/industries/financial-services/aml.mdx b/versioned_docs/version-1.0/industries/financial-services/aml.mdx
new file mode 100644
index 0000000..0c77454
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/financial-services/aml.mdx
@@ -0,0 +1,157 @@
+---
+sidebar_position: 2
+title: Banking — payments & AML
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Banking — payments & anti-money-laundering
+
+**The problem.** Money laundering hides in the *shape* of a transaction network, not
+in any single payment. Funds fan out through mule accounts, loop through shell
+counterparties, and reconverge — a pattern invisible to a per-transaction rule but
+obvious on a graph. A bank's financial-crime team needs to catch a suspicious payment
+as it happens and immediately see the web of accounts around it, in time to hold the
+funds.
+
+The problem here is **finding hidden structure fast enough to act** — react to a live
+payment, then expand its network.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the flagged-payments stream (**subscription
+created before we listen**) and the account network the flagged account sits in — a loop of
+transfers, i.e. a laundering ring. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="flagged_payments_feed", name="Flagged payments", unit="count", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="flagged_payments", name="Flagged payments", timeseries=["flagged_payments_feed"])])
+client.timeseries.insert_from_lists(timestamps=[pd.Timestamp.now(tz="UTC")], values=[1.0], ts="flagged_payments_feed")
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=f"account_{i}", name=f"Account {i}", labels=["Account"]) for i in [77310, 8841, 8842, 8843]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("account_77310", "account_8841", "sent_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("account_8841", "account_8842", "sent_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("account_8842", "account_8843", "sent_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("account_8843", "account_77310", "sent_to")])
+```
+
+## 1. React to a flagged payment live
+
+A rules engine publishes flagged payments; subscribe and pick them up the moment they
+land. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("flagged_payments"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ investigate(accountFrom(msg.payload())); // expand its network (step 2)
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["flagged_payments"]) as listener:
+ for msg in listener:
+ investigate(account_from(msg.payload)) # expand its network (step 2)
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+let mut listener = api.subscriptions.listen(&["flagged_payments"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ investigate(account_from(&msg.payload)).await?; // expand its network (step 2)
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Expand the account into its network
+
+Accounts, transactions and counterparties are a graph. Walk out from the flagged
+account and the returned sub-graph is the cluster — the mules and shells it moves money
+through. Other flagged accounts in the same cluster are the ring.
+
+
+
+
+```java
+ResourceNetwork ring = client.resources().fetchRelated("account_77310", 5);
+
+ring.nodes().stream()
+ .map(NodeModel::getExternalId)
+ .filter(id -> id.startsWith("account_"))
+ .forEach(a -> System.out.println("linked account: " + a));
+```
+
+
+
+
+```python
+ring = client.resources.fetch_related(external_id="account_77310", depth=5)
+
+linked = [n.external_id for n in ring.nodes if n.external_id.startswith("account_")]
+print("ring accounts:", linked)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let ring = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("account_77310").with_depth(5)).await?;
+
+for node in ring.nodes() {
+ if node.external_id().starts_with("account_") {
+ println!("linked account: {}", node.external_id());
+ }
+}
+```
+
+
+
+
+Same shared-node reasoning as [alarm correlation](/guides/correlate-alarms), here
+turning one flagged payment into the whole laundering ring.
+
+## 3. Escalate and watch the trend
+
+Raise a `sar_candidate` [event](/guides/detect-events) on the cluster for the
+investigations queue, and track `account_77310_outflow` as a
+[series](/guides/query-and-aggregate) — a sudden outflow spike is structuring in
+progress.
+
+## See the result
+
+Expanding the flagged account reveals the ring it moves money through:
+
+```text
+ring accounts: ['account_8841', 'account_8842', 'account_8843']
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — reacting to flagged payments.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — expanding the network.
+- [Turn readings into events](/guides/detect-events) — escalating a cluster.
+- **[Fraud classification (advanced)](/advanced/fraud-classification)** — score each cluster with graph features + a model so investigators work the riskiest first.
diff --git a/versioned_docs/version-1.0/industries/financial-services/insurance-fraud.mdx b/versioned_docs/version-1.0/industries/financial-services/insurance-fraud.mdx
new file mode 100644
index 0000000..81a4622
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/financial-services/insurance-fraud.mdx
@@ -0,0 +1,164 @@
+---
+sidebar_position: 3
+title: Insurance — claims & fraud
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Insurance — claims & fraud rings
+
+**The problem.** Most claims are honest and independent. Organised fraud isn't — it
+hides in the *connections* between claims that look unrelated one at a time: the same
+phone number on two policies, the same repair shop across a dozen claims, the same
+address shared by claimants who supposedly don't know each other. A flat claims table
+can't see a ring. A graph can: model claims, policies and the parties they share, and
+a single traversal lights up the cluster.
+
+This scenario centres on **graph entity resolution** — turning a suspicious claim into
+the ring it belongs to.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create two claims that share a repair shop — the
+hidden link a per-claim view misses. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[lbl]) for x, lbl in
+ [("claim_88421", "Claim"), ("claim_88455", "Claim"), ("party_jdoe", "Party"),
+ ("phone_47120099", "Phone"), ("shop_quickfix", "Shop")]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("claim_88421", "party_jdoe", "filed_by"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("claim_88421", "shop_quickfix", "repaired_at"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("party_jdoe", "phone_47120099", "contact_phone"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("claim_88455", "shop_quickfix", "repaired_at")])
+```
+
+## 1. Model claims and the parties they share
+
+Claims, policies and parties (people, phone numbers, addresses, repair shops) are all
+resources; the edges record who is connected to what. The shared parties are what make
+a ring visible.
+
+
+
+
+```java
+List links = List.of(
+ rel("on_policy", "claim_88421", "policy_55righ"),
+ rel("filed_by", "claim_88421", "party_jdoe"),
+ rel("repaired_at", "claim_88421", "shop_quickfix"),
+ rel("contact_phone", "party_jdoe", "phone_47120099"));
+
+client.resources().create(nodes, links);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+links = [
+ intellistream_datahub_sdk.RelForm.by_external_ids("claim_88421", "policy_55righ", "on_policy"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("claim_88421", "party_jdoe", "filed_by"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("claim_88421", "shop_quickfix", "repaired_at"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("party_jdoe", "phone_47120099", "contact_phone"),
+]
+client.resources.create(nodes, links)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::relations::RelForm;
+
+let links = vec![
+ RelForm::by_external_ids("claim_88421", "policy_55righ", "on_policy"),
+ RelForm::by_external_ids("claim_88421", "party_jdoe", "filed_by"),
+ RelForm::by_external_ids("claim_88421", "shop_quickfix", "repaired_at"),
+ RelForm::by_external_ids("party_jdoe", "phone_47120099", "contact_phone"),
+];
+api.resources.create(nodes, links).await?;
+```
+
+
+
+
+## 2. Expand a suspicious claim into its ring
+
+Walk out from the flagged claim. The returned sub-graph is everything connected to it
+— and any *other* claim that shares a party, phone or shop with it is part of the same
+cluster.
+
+
+
+
+```java
+ResourceNetwork ring = client.resources().fetchRelated("claim_88421", 4);
+
+// other claims reachable through shared parties/phones/shops
+ring.nodes().stream()
+ .map(NodeModel::getExternalId)
+ .filter(id -> id.startsWith("claim_") && !id.equals("claim_88421"))
+ .forEach(c -> System.out.println("linked claim: " + c));
+```
+
+
+
+
+```python
+ring = client.resources.fetch_related(external_id="claim_88421", depth=4)
+
+linked = [n.external_id for n in ring.nodes
+ if n.external_id.startswith("claim_") and n.external_id != "claim_88421"]
+print("linked claims:", linked)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let ring = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("claim_88421").with_depth(4)).await?;
+
+for node in ring.nodes() {
+ if node.external_id().starts_with("claim_") && node.external_id() != "claim_88421" {
+ println!("linked claim: {}", node.external_id());
+ }
+}
+```
+
+
+
+
+The more claims a single party, phone or shop connects, the stronger the signal —
+exactly the shared-node reasoning behind
+[alarm correlation](/guides/correlate-alarms), applied to entities instead of assets.
+
+## 3. Flag and watch the trend
+
+Raise a `fraud_review` [event](/guides/detect-events) on the cluster, and track
+`claims_per_shop` or `claims_per_phone` as [series](/guides/query-and-aggregate) — a
+node whose claim count climbs unusually fast is a ring forming in real time.
+
+## See the result
+
+Expanding the flagged claim surfaces the one it's secretly linked to:
+
+```text
+linked claims: ['claim_88455'] ← shares shop_quickfix with claim_88421
+```
+
+## See also
+
+- [Model assets as a graph](/guides/model-assets-graph) — modeling claims, policies and parties.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — the shared-node ring detection.
+- [Turn readings into events](/guides/detect-events) — flagging a cluster for review.
+- **[Fraud classification (advanced)](/advanced/fraud-classification)** — turn the ring into graph features and score it with a model.
diff --git a/versioned_docs/version-1.0/industries/financial-services/portfolio-risk.mdx b/versioned_docs/version-1.0/industries/financial-services/portfolio-risk.mdx
new file mode 100644
index 0000000..42f5003
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/financial-services/portfolio-risk.mdx
@@ -0,0 +1,216 @@
+---
+sidebar_position: 1
+title: Finance — portfolio & risk
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Finance — portfolio metrics & risk limits
+
+**The problem.** A trading desk runs several portfolios, each holding positions in
+many instruments. Risk wants a tidy, queryable home for per-portfolio time-series —
+mark-to-market value, daily P&L, one-day Value-at-Risk — segmented so each desk sees
+only its own books, plus an event whenever a portfolio breaches a risk limit.
+
+This scenario leans on **datasets** to partition data by desk, and on events as a
+durable record of limit breaches.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the desk's dataset and a portfolio
+mark-to-market series with a month of daily values. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+desk_id = client.datasets.create([intellistream_datahub_sdk.Dataset(external_id="desk_equities", name="Equities desk")])[0].id
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="portfolio_growth_mtm_usd",
+ name="Growth portfolio — MtM (USD)", unit="usd", value_type="float", data_set_id=desk_id)])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=30, freq="1d")
+client.timeseries.insert_from_lists(timestamps=idx,
+ values=1e7 + np.cumsum(np.random.normal(0, 5e4, 30)), ts="portfolio_growth_mtm_usd")
+```
+
+## 1. Partition by desk with a dataset
+
+A dataset is the unit of grouping and access. Give each desk its own; everything the
+desk owns — series, positions — carries its `data_set_id`.
+
+
+
+
+```java
+DataSetModel desk = new DataSetModel();
+desk.setExternalId("desk_equities");
+desk.setName("Equities desk");
+var created = client.datasets().create(List.of(desk));
+long deskId = created.getItems().get(0).getId();
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+created = client.datasets.create([
+ intellistream_datahub_sdk.Dataset(external_id="desk_equities", name="Equities desk")])
+desk_id = created[0].id
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::datasets::Dataset;
+
+let mut ds = Dataset::new("Equities desk".into());
+ds.external_id = "desk_equities".into();
+let created = api.datasets.create(&vec![ds]).await?;
+let desk_id = created.get_items()[0].id;
+```
+
+
+
+
+## 2. Record per-portfolio metrics
+
+Create a series per portfolio metric, tagged with the desk's `data_set_id`, then
+append a point at each mark. Values are exact decimals — pick a `NUMERIC` value type
+so currency amounts store faithfully.
+
+
+
+
+```java
+var mtm = new Timeseries();
+mtm.setExternalId("portfolio_growth_mtm_usd");
+mtm.setName("Growth portfolio — mark-to-market (USD)");
+mtm.setValueType("numeric");
+mtm.setDataSetId(deskId);
+mtm.setUnit("usd");
+client.timeseries().create(mtm);
+
+client.timeseries().ingest(Map.of(
+ "portfolio_growth_mtm_usd", List.of(
+ Datapoint.of(Instant.now(), "10428155.42")))); // exact decimal as text
+```
+
+
+
+
+```python
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="portfolio_growth_mtm_usd",
+ name="Growth portfolio — mark-to-market (USD)",
+ unit="usd", value_type="float", data_set_id=desk_id)])
+
+client.timeseries.insert_from_lists(
+ timestamps=[pd.Timestamp.now(tz="UTC")], values=[10428155.42],
+ ts="portfolio_growth_mtm_usd")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use chrono::Utc;
+
+let mut ts = TimeSeries::new("portfolio_growth_mtm_usd", "Growth portfolio — mark-to-market (USD)");
+ts.unit = Some("usd".into());
+ts.value_type = Some("numeric".into());
+ts.data_set_id = desk_id; // desk_id is already Option
+api.time_series.create_one(&ts).await?;
+
+api.time_series
+ .insert_datapoint(None, Some("portfolio_growth_mtm_usd".into()), Utc::now(),
+ "10428155.42".into())
+ .await?;
+```
+
+
+
+
+## 3. Flag a risk-limit breach
+
+When a portfolio's one-day VaR exceeds its mandate, record a `var_limit_breach`
+event. Because the event carries structured metadata, risk can reconstruct exactly
+which book breached, by how much, and when.
+
+
+
+
+```java
+EventModel breach = new EventModel();
+breach.setExternalId("var_limit_breach_growth_" + System.currentTimeMillis());
+breach.setType("var_limit_breach");
+breach.setStatus("open");
+breach.setDataSetId(deskId);
+breach.setMetadata(Map.of(
+ "portfolio", "portfolio_growth", "var_1d_usd", "512340", "limit_usd", "500000"));
+breach.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(breach));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"var_limit_breach_growth_{int(pd.Timestamp.now().timestamp())}",
+ type="var_limit_breach", status="open", data_set_id=desk_id,
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"portfolio": "portfolio_growth",
+ "var_1d_usd": "512340", "limit_usd": "500000"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut breach = Event::new(
+ format!("var_limit_breach_growth_{}", Utc::now().timestamp()),
+ "var_limit_breach".into(),
+ Utc::now());
+breach.status = Some("open".into());
+breach.data_set_id = desk_id;
+breach.add_metadata("portfolio".into(), "portfolio_growth".into());
+breach.add_metadata("var_1d_usd".into(), "512340".into());
+breach.add_metadata("limit_usd".into(), "500000".into());
+api.events.create(&vec![breach]).await?;
+```
+
+
+
+
+:::note Exact money
+Use the `NUMERIC` value type for monetary series — it stores exact decimals, unlike
+the floating-point types. See the [Time-series reference](/reference/timeseries) for
+the full list of value types.
+:::
+
+## See the result
+
+Chart the portfolio's mark-to-market over the month:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="portfolio_growth_mtm_usd",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=30), end=pd.Timestamp.now(tz="UTC"))
+s = pd.Series({pd.to_datetime(p.timestamp): float(p.value)
+ for p in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()}).sort_index()
+s.plot(title="Growth portfolio mark-to-market (USD)"); plt.show()
+```
+
+## See also
+
+- [Datasets reference](/reference/datasets) — grouping and access by desk.
+- [Turn readings into events](/guides/detect-events) — the limit-breach rule.
+- [Query & aggregate](/guides/query-and-aggregate) — daily P&L roll-ups.
diff --git a/versioned_docs/version-1.0/industries/healthcare/_category_.json b/versioned_docs/version-1.0/industries/healthcare/_category_.json
new file mode 100644
index 0000000..90a959b
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/healthcare/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Healthcare",
+ "position": 6,
+ "link": { "type": "generated-index", "title": "Healthcare" }
+}
diff --git a/versioned_docs/version-1.0/industries/healthcare/cold-chain.mdx b/versioned_docs/version-1.0/industries/healthcare/cold-chain.mdx
new file mode 100644
index 0000000..52f7b07
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/healthcare/cold-chain.mdx
@@ -0,0 +1,124 @@
+---
+sidebar_position: 4
+title: Healthcare — pharmacy cold chain
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Healthcare — pharmacy cold chain
+
+**The problem.** Vaccines, blood products and many medications are only safe within a
+narrow temperature band. A pharmacy or ward fridge that drifts out of range — a door
+left ajar, a failing compressor overnight — can spoil tens of thousands of euros of
+stock and, worse, leave a patient given a product that quietly lost its potency. Staff
+can't watch every fridge; they need an alarm the moment one strays, and a clean
+temperature record to prove the chain held.
+
+What we solve here is protecting temperature-sensitive stock with live monitoring and an
+auditable record.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a fridge's temperature series, **create the
+subscription before we listen**, and push it above the 2–8 °C band. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="fridge_pharmacy_12_temp_c", name="Pharmacy fridge 12 temp", unit="deg_c", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="pharmacy_fridges", name="Pharmacy fridges", timeseries=["fridge_pharmacy_12_temp_c"])])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=60, freq="1min")
+t = np.full(60, 4.5); t[-10:] = 9.3 # rises above the 8 °C ceiling
+client.timeseries.insert_from_lists(timestamps=idx, values=t, ts="fridge_pharmacy_12_temp_c")
+```
+
+## 1. Watch every fridge live
+
+Each fridge streams its temperature; a subscription brings them into one feed. When one
+leaves its band, raise a `cold_chain_excursion` event so it's attended before stock is
+lost. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("pharmacy_fridges"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (outOfBand(msg.payload(), 2.0, 8.0)) { // °C — vaccine fridge band
+ EventModel excursion = new EventModel();
+ excursion.setExternalId("cold_chain_excursion_f12_" + System.currentTimeMillis());
+ excursion.setType("cold_chain_excursion");
+ excursion.setStatus("critical");
+ excursion.setMetadata(Map.of("fridge", "fridge_pharmacy_12", "temp_c", "9.3"));
+ excursion.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(excursion));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["pharmacy_fridges"]) as listener:
+ for msg in listener:
+ if out_of_band(msg.payload, 2.0, 8.0): # °C — vaccine fridge band
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"cold_chain_excursion_f12_{int(pd.Timestamp.now().timestamp())}",
+ type="cold_chain_excursion", status="critical",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"fridge": "fridge_pharmacy_12", "temp_c": "9.3"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["pharmacy_fridges"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if out_of_band(&msg.payload, 2.0, 8.0) { // °C — vaccine fridge band
+ let mut excursion = Event::new(
+ format!("cold_chain_excursion_f12_{}", Utc::now().timestamp()),
+ "cold_chain_excursion".into(),
+ Utc::now());
+ excursion.status = Some("critical".into());
+ excursion.add_metadata("fridge".into(), "fridge_pharmacy_12".into());
+ excursion.add_metadata("temp_c".into(), "9.3".into());
+ api.events.create(&vec![excursion]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Keep the compliance record
+
+For an audit, retrieve a fridge's temperature history over any window to produce the
+time-stamped record showing the chain held. See [Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The warming fridge trips the loop while there's still time to act:
+
+```text
+cold_chain_excursion_f12_… → critical (9.3 °C, above the 2–8 °C band)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the fridge-monitoring loop.
+- [Turn readings into events](/guides/detect-events) — the excursion rule.
+- [Maritime — reefer cold chain](/industries/transport-logistics/maritime) — the same pattern at sea.
diff --git a/versioned_docs/version-1.0/industries/healthcare/hospital-operations.mdx b/versioned_docs/version-1.0/industries/healthcare/hospital-operations.mdx
new file mode 100644
index 0000000..03bfa79
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/healthcare/hospital-operations.mdx
@@ -0,0 +1,197 @@
+---
+sidebar_position: 1
+title: Healthcare — hospital operations
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Healthcare — hospital operations & patient flow
+
+**The problem.** A hospital runs at the edge of capacity. Beds free up and fill
+minute by minute, the emergency department backs up, and a ward that quietly hits
+100% occupancy turns into a corridor of waiting patients. The operations team needs a
+single live picture of capacity across every ward — and an alert the moment a unit is
+about to overflow, while there's still time to act.
+
+This scenario is about **live operational monitoring**: model the hospital, stream
+occupancy in, and drive a capacity wall-board off a push connection. It uses only
+operational signals — bed counts, wait times, census — never patient records.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a ward's occupancy series, **create the
+subscription before we listen**, and push it over the 95% line. The hospital graph is
+created by step 1. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="ward_icu_occupancy_pct", name="ICU occupancy %", unit="pct", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="bed_capacity", name="Bed capacity", timeseries=["ward_icu_occupancy_pct"])])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=60, freq="1min")
+occ = np.full(60, 88.0); occ[-10:] = 96.0 # crosses the 95% capacity threshold
+client.timeseries.insert_from_lists(timestamps=idx, values=occ, ts="ward_icu_occupancy_pct")
+```
+
+## 1. Model the hospital
+
+A hospital *contains* wards, a ward *contains* beds. The same hierarchy you'd model
+for any physical estate, which lets a ward-level alert roll up to the whole site.
+
+
+
+
+```java
+Resource hospital = new Resource();
+hospital.setExternalId("hospital_central");
+hospital.setName("Central Hospital");
+hospital.setLabels(List.of("Hospital"));
+
+Resource ward = new Resource();
+ward.setExternalId("ward_icu");
+ward.setName("Intensive care");
+ward.setLabels(List.of("Ward"));
+
+RelForm contains = new RelForm();
+contains.setName("contains");
+contains.setFromExternalId("hospital_central");
+contains.setToExternalId("ward_icu");
+
+client.resources().create(List.of(hospital, ward), List.of(contains));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id="hospital_central", name="Central Hospital", labels=["Hospital"]),
+ intellistream_datahub_sdk.Resource(external_id="ward_icu", name="Intensive care", labels=["Ward"])],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("hospital_central", "ward_icu", "contains")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut hospital = Resource::new();
+hospital.external_id = "hospital_central".into();
+hospital.name = "Central Hospital".into();
+hospital.labels = Some(vec!["Hospital".into()]);
+
+let mut ward = Resource::new();
+ward.external_id = "ward_icu".into();
+ward.name = "Intensive care".into();
+ward.labels = Some(vec!["Ward".into()]);
+
+api.resources.create(
+ vec![hospital, ward],
+ vec![RelForm::by_external_ids("hospital_central", "ward_icu", "contains")],
+).await?;
+```
+
+
+
+
+Capacity signals become series: `ward_icu_occupancy_pct`, `ward_icu_free_beds`,
+`ed_wait_minutes`.
+
+## 2. Drive the capacity wall-board live
+
+A wall-board polling every minute is already stale. Subscribe to the occupancy series
+and update the moment a reading lands — and raise a `capacity_warning` event when a
+ward crosses its threshold, so charge nurses are paged before it's full. See
+[Consume live data](/guides/realtime-subscriptions) for the listener lifecycle.
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("bed_capacity"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ updateWallboard(msg.payload());
+ if (overThreshold(msg.payload())) { // e.g. occupancy > 95%
+ EventModel warning = new EventModel();
+ warning.setExternalId("capacity_warning_icu_" + System.currentTimeMillis());
+ warning.setType("capacity_warning");
+ warning.setStatus("open");
+ warning.setMetadata(Map.of("ward", "ward_icu", "occupancy_pct", "96"));
+ warning.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(warning));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["bed_capacity"]) as listener:
+ for msg in listener:
+ update_wallboard(msg.payload)
+ if over_threshold(msg.payload): # e.g. occupancy > 95%
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"capacity_warning_icu_{int(pd.Timestamp.now().timestamp())}",
+ type="capacity_warning", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"ward": "ward_icu", "occupancy_pct": "96"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["bed_capacity"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ update_wallboard(&msg.payload);
+ if over_threshold(&msg.payload) { // e.g. occupancy > 95%
+ let mut warning = Event::new(
+ format!("capacity_warning_icu_{}", Utc::now().timestamp()),
+ "capacity_warning".into(),
+ Utc::now());
+ warning.status = Some("open".into());
+ warning.add_metadata("ward".into(), "ward_icu".into());
+ warning.add_metadata("occupancy_pct".into(), "96".into());
+ api.events.create(&vec![warning]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 3. Review the day's pressure points
+
+Hourly-average occupancy per ward shows planners *when* the squeeze happens, so they
+can shift staff and elective schedules. See [Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The ICU crossing 95% trips the loop:
+
+```text
+capacity_warning_icu_… → open (occupancy 96% — page the charge nurse)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the capacity wall-board's push loop.
+- [Turn readings into events](/guides/detect-events) — the threshold rule.
+- [Query & aggregate](/guides/query-and-aggregate) — daily occupancy roll-ups.
diff --git a/versioned_docs/version-1.0/industries/healthcare/medical-devices.mdx b/versioned_docs/version-1.0/industries/healthcare/medical-devices.mdx
new file mode 100644
index 0000000..d1d1468
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/healthcare/medical-devices.mdx
@@ -0,0 +1,170 @@
+---
+sidebar_position: 2
+title: Healthcare — medical-device fleet
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Healthcare — medical-device fleet
+
+**The problem.** A hospital runs thousands of connected devices — infusion pumps,
+ventilators, monitors — and two things must never slip: a critical device must be
+available when a clinician reaches for it, and when a **recall or fault pattern** hits a
+device model, every affected unit must be found and pulled *today*, before it reaches a
+patient. Tracking that across wards by spreadsheet is how units get missed.
+
+What we solve here is device availability and fast, complete recall tracing.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the model, the ward and two more pumps of
+the same model in it. Step 1 registers a third, so the recall walk in step 2 returns all three.
+
+```python
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+nodes = [intellistream_datahub_sdk.Resource(external_id="model_acme_x200", name="Acme X200 infusion pump", labels=["PumpModel"]),
+ intellistream_datahub_sdk.Resource(external_id="ward_icu", name="Intensive care", labels=["Ward"])]
+edges = []
+for u in ["3392", "3401"]:
+ nodes.append(intellistream_datahub_sdk.Resource(external_id=f"pump_icu_{u}", name=f"Infusion pump {u}", labels=["Pump"]))
+ edges += [intellistream_datahub_sdk.RelForm.by_external_ids(f"pump_icu_{u}", "ward_icu", "located_in"),
+ intellistream_datahub_sdk.RelForm.by_external_ids(f"pump_icu_{u}", "model_acme_x200", "is_model")]
+client.resources.create(nodes, edges)
+```
+
+## 1. Model the fleet and watch availability
+
+A hospital *contains* wards, a ward *contains* devices, and each device *is a* model.
+Device status is a series; a unit that faults or drops offline raises a `device_down`
+event for biomedical engineering.
+
+
+
+
+```java
+Resource pump = new Resource();
+pump.setExternalId("pump_icu_3391");
+pump.setName("Infusion pump 3391");
+pump.setLabels(List.of("Pump"));
+
+RelForm inWard = new RelForm();
+inWard.setName("located_in");
+inWard.setFromExternalId("pump_icu_3391");
+inWard.setToExternalId("ward_icu");
+
+RelForm isModel = new RelForm();
+isModel.setName("is_model");
+isModel.setFromExternalId("pump_icu_3391");
+isModel.setToExternalId("model_acme_x200");
+
+client.resources().create(List.of(pump), List.of(inWard, isModel));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id="pump_icu_3391", name="Infusion pump 3391", labels=["Pump"])],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("pump_icu_3391", "ward_icu", "located_in"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("pump_icu_3391", "model_acme_x200", "is_model")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut pump = Resource::new();
+pump.external_id = "pump_icu_3391".into();
+pump.name = "Infusion pump 3391".into();
+pump.labels = Some(vec!["Pump".into()]);
+
+api.resources.create(
+ vec![pump],
+ vec![
+ RelForm::by_external_ids("pump_icu_3391", "ward_icu", "located_in"),
+ RelForm::by_external_ids("pump_icu_3391", "model_acme_x200", "is_model"),
+ ],
+).await?;
+```
+
+
+
+
+## 2. A recall lands — find every affected unit
+
+When `model_acme_x200` is recalled, walk the graph out from the model node: every device
+that `is_model` of it, and the ward each sits in, is the exact pull list. This is the
+[blast-radius traversal](/guides/correlate-alarms) — the same shape as an aerospace part
+recall, applied to hospital equipment.
+
+
+
+
+```java
+ResourceNetwork affected = client.resources().fetchRelated("model_acme_x200", 3);
+
+affected.nodes().stream()
+ .filter(n -> n.getExternalId().startsWith("pump_"))
+ .forEach(d -> System.out.println("pull from service: " + d.getExternalId()));
+```
+
+
+
+
+```python
+affected = client.resources.fetch_related(external_id="model_acme_x200", depth=3)
+
+for node in affected.nodes:
+ if node.external_id.startswith("pump_"):
+ print("pull from service:", node.external_id)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let affected = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("model_acme_x200").with_depth(3)).await?;
+
+for node in affected.nodes() {
+ if node.external_id().starts_with("pump_") {
+ println!("pull from service: {}", node.external_id());
+ }
+}
+```
+
+
+
+
+## 3. Predict failures before they strand a clinician
+
+Pump and battery health are series; rolling them up surfaces units trending toward
+failure for planned swap-out — and the [XGBoost failure
+predictor](/advanced/xgboost-failure-prediction) does it properly.
+
+## See the result
+
+The recall walk returns every unit of the model to pull:
+
+```text
+pull from service: pump_icu_3391
+pull from service: pump_icu_3392
+pull from service: pump_icu_3401
+```
+
+## See also
+
+- [Model assets as a graph](/guides/model-assets-graph) — the hospital/ward/device hierarchy.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — recall blast-radius tracing.
+- [Aerospace fleet](/industries/transport-logistics/aerospace) — the same recall pattern.
diff --git a/versioned_docs/version-1.0/industries/healthcare/patient-flow.mdx b/versioned_docs/version-1.0/industries/healthcare/patient-flow.mdx
new file mode 100644
index 0000000..1be56f9
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/healthcare/patient-flow.mdx
@@ -0,0 +1,172 @@
+---
+sidebar_position: 3
+title: Healthcare — patient flow
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Healthcare — patient flow
+
+**The problem.** Patients move through a hospital in a chain — emergency department →
+assessment → ward → discharge — and the whole thing jams if any link backs up. A slow
+discharge process leaves no free beds, which backs up the ward, which strands patients
+in the ED corridor. Operations needs to see **where the flow is actually stuck** —
+which stage's wait is growing — so they fix the real bottleneck, not the loudest one.
+
+What we solve here is finding the bottleneck stage in the patient journey, using the
+flow data the hospital already has.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the ED time-to-bed series with a day of
+readings whose wait grows through the afternoon. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="ed_time_to_bed_minutes", name="ED time to bed (min)", unit="min", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=24, freq="1h")
+ttb = 90 + np.linspace(0, 120, 24) + np.random.normal(0, 10, 24)
+client.timeseries.insert_from_lists(timestamps=idx, values=ttb, ts="ed_time_to_bed_minutes")
+```
+
+## 1. Roll up the wait at each stage
+
+ED wait, time-to-bed, and length-of-stay are series. Hourly and daily roll-ups show
+which stage is slow *right now* versus its normal — the bottleneck. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("ed_time_to_bed_minutes");
+filter.setStart(ZonedDateTime.now().minusDays(1));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg", "max"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> trackStage("time_to_bed", p.getTimestamp(), p.getValue()));
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="ed_time_to_bed_minutes",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg", "max"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ track_stage("time_to_bed", dp.timestamp, dp.average)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("ed_time_to_bed_minutes".into()),
+ aggregates: Some(vec!["avg".into(), "max".into()]),
+ granularity: Some("1h".into()),
+ start: Some(Utc::now() - chrono::Duration::days(1)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { track_stage("time_to_bed", &p.timestamp, p.average); }
+```
+
+
+
+
+## 2. Alert when the ED breaches its target
+
+When the ED's four-hour-target breach rate climbs past a threshold, raise a
+`flow_breach` event so the patient-flow coordinator can pull the right lever —
+expedite discharges, open an escalation area. See [Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+EventModel breach = new EventModel();
+breach.setExternalId("flow_breach_ed_" + System.currentTimeMillis());
+breach.setType("flow_breach");
+breach.setStatus("open");
+breach.setMetadata(Map.of("stage", "emergency_department", "breach_rate_pct", "22"));
+breach.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(breach));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"flow_breach_ed_{int(pd.Timestamp.now().timestamp())}",
+ type="flow_breach", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"stage": "emergency_department", "breach_rate_pct": "22"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut breach = Event::new(
+ format!("flow_breach_ed_{}", Utc::now().timestamp()),
+ "flow_breach".into(),
+ Utc::now());
+breach.status = Some("open".into());
+breach.add_metadata("stage".into(), "emergency_department".into());
+breach.add_metadata("breach_rate_pct".into(), "22".into());
+api.events.create(&vec![breach]).await?;
+```
+
+
+
+
+## 3. Forecast the surge
+
+Admissions follow strong daily and seasonal patterns — [forecast
+them](/advanced/demand-forecasting) to staff ahead of the surge instead of reacting to it.
+
+## See the result
+
+Chart the hourly wait — the climb is the bottleneck the coordinator acts on:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="ed_time_to_bed_minutes",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+v = [dp.average for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.plot(v); plt.title("ED time-to-bed (min), hourly average"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — per-stage wait roll-ups.
+- [Turn readings into events](/guides/detect-events) — the breach rule.
+- [Demand forecasting](/advanced/demand-forecasting) — predicting admissions.
diff --git a/versioned_docs/version-1.0/industries/manufacturing-process/_category_.json b/versioned_docs/version-1.0/industries/manufacturing-process/_category_.json
new file mode 100644
index 0000000..42a5525
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/manufacturing-process/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Manufacturing & process",
+ "position": 3,
+ "link": { "type": "generated-index", "title": "Manufacturing & process" }
+}
diff --git a/versioned_docs/version-1.0/industries/manufacturing-process/discrete.mdx b/versioned_docs/version-1.0/industries/manufacturing-process/discrete.mdx
new file mode 100644
index 0000000..4dd01bd
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/manufacturing-process/discrete.mdx
@@ -0,0 +1,232 @@
+---
+sidebar_position: 1
+title: Manufacturing — quality & OEE
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Manufacturing — production quality & OEE
+
+**The problem.** A factory floor is dozens of stations, each emitting a cycle time, a
+unit count, and the occasional scrap event — thousands of readings a minute. Plant
+managers don't want the raw firehose; they want **Overall Equipment Effectiveness**
+(availability × performance × quality), rolled up per station, line and shift, plus an
+alert when a station's scrap rate spikes.
+
+This scenario centres on **ingesting at volume and aggregating** the result into the
+hourly and shift-level numbers a production dashboard actually shows.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the station's series with an 8-hour shift
+of readings, so the OEE roll-up has data. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+for s, u in [("station_07_cycle_time_s", "s"), ("station_07_scrap", "count")]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=s, name=s, unit=u, value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=8 * 60, freq="1min")
+client.timeseries.insert_from_lists(timestamps=idx, values=11 + np.random.normal(0, 0.6, len(idx)), ts="station_07_cycle_time_s")
+client.timeseries.insert_from_lists(timestamps=idx, values=np.random.poisson(0.3, len(idx)).astype(float), ts="station_07_scrap")
+```
+
+## 1. Model the line and stream its output
+
+A plant *contains* lines, a line *contains* stations. Each station's signals are
+series: `station_07_cycle_time_s`, `station_07_units`, `station_07_scrap`.
+
+
+
+
+```java
+var cycleTime = new Timeseries();
+cycleTime.setExternalId("station_07_cycle_time_s");
+cycleTime.setName("Station 07 cycle time");
+cycleTime.setUnit("s");
+var scrap = new Timeseries();
+scrap.setExternalId("station_07_scrap");
+scrap.setName("Station 07 scrap count");
+scrap.setUnit("count");
+client.timeseries().create(cycleTime, scrap);
+
+// readings arrive continuously across every station — hand them off in bulk
+client.timeseries().ingest(Map.of(
+ "station_07_cycle_time_s", cycleTimes, // List
+ "station_07_scrap", scrapCounts));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.timeseries.create([
+ intellistream_datahub_sdk.TimeSeries(value_type="float", external_id="station_07_cycle_time_s",
+ name="Station 07 cycle time", unit="s"),
+ intellistream_datahub_sdk.TimeSeries(external_id="station_07_scrap", name="Station 07 scrap count", unit="count")])
+
+client.timeseries.insert_from_lists(
+ timestamps=timestamps, values=cycle_times, ts="station_07_cycle_time_s")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+let mut cycle = TimeSeries::new("station_07_cycle_time_s", "Station 07 cycle time");
+cycle.unit = Some("s".into());
+api.time_series.create_one(&cycle).await?;
+let mut scrap = TimeSeries::new("station_07_scrap", "Station 07 scrap count");
+scrap.unit = Some("count".into());
+api.time_series.create_one(&scrap).await?;
+api.time_series
+ .insert_datapoint(None, Some("station_07_cycle_time_s".into()), ts, "11.4".into())
+ .await?;
+```
+
+
+
+
+See [high-throughput ingestion](/guides/ingest-timeseries) for sustaining floor-wide volume.
+
+## 2. Roll up to hourly OEE inputs
+
+The dashboard needs per-hour averages and totals, not raw cycles. Ask for the
+aggregates and granularity and the platform returns one value per bucket — the
+performance inputs to OEE. (There's no `count` aggregate; a per-hour count comes
+from `sum`-ming a per-event count series.) Full mechanics in
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("station_07_cycle_time_s");
+filter.setStart(ZonedDateTime.now().minusHours(8)); // one shift
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg", "sum"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> recordHourlyPerformance(p.getTimestamp(), p.getValue()));
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="station_07_cycle_time_s",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=8),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg", "sum"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ record_hourly_performance(dp.timestamp, dp.average, dp.sum)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("station_07_cycle_time_s".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(8)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["avg".into(), "sum".into()]),
+ granularity: Some("1h".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { record_hourly_performance(&p.timestamp, p.average); }
+```
+
+
+
+
+## 3. Flag a quality excursion
+
+When a station's hourly scrap rate jumps past its limit, record a `scrap_excursion`
+event so the quality team sees a ranked list of stations needing attention.
+
+
+
+
+```java
+EventModel excursion = new EventModel();
+excursion.setExternalId("scrap_excursion_st07_" + System.currentTimeMillis());
+excursion.setType("scrap_excursion");
+excursion.setStatus("open");
+excursion.setMetadata(Map.of("station", "station_07", "scrap_rate_pct", "4.8", "limit_pct", "2.0"));
+excursion.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(excursion));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"scrap_excursion_st07_{int(pd.Timestamp.now().timestamp())}",
+ type="scrap_excursion", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"station": "station_07", "scrap_rate_pct": "4.8", "limit_pct": "2.0"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut excursion = Event::new(
+ format!("scrap_excursion_st07_{}", Utc::now().timestamp()),
+ "scrap_excursion".into(),
+ Utc::now());
+excursion.status = Some("open".into());
+excursion.add_metadata("station".into(), "station_07".into());
+excursion.add_metadata("scrap_rate_pct".into(), "4.8".into());
+excursion.add_metadata("limit_pct".into(), "2.0".into());
+api.events.create(&vec![excursion]).await?;
+```
+
+
+
+
+## See the result
+
+Chart the hourly average cycle time — the line the production dashboard watches:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="station_07_cycle_time_s",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=8), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+vals = [dp.average for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.plot(vals); plt.title("Station 07 hourly avg cycle time (s)"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — the OEE roll-up mechanics.
+- [High-throughput ingestion](/guides/ingest-timeseries) — floor-wide write volume.
+- [Turn readings into events](/guides/detect-events) — the scrap-rate rule.
diff --git a/versioned_docs/version-1.0/industries/manufacturing-process/pharma.mdx b/versioned_docs/version-1.0/industries/manufacturing-process/pharma.mdx
new file mode 100644
index 0000000..46ed2e5
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/manufacturing-process/pharma.mdx
@@ -0,0 +1,176 @@
+---
+sidebar_position: 3
+title: Pharma — batch genealogy
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Pharma — batch manufacturing & genealogy
+
+**The problem.** A finished drug lot fails a quality check. Before anything ships,
+the manufacturer must reconstruct its **genealogy**: which intermediate lots, raw
+material lots, and equipment produced it, and whether any deviation in the process
+record explains the failure. The process data has to be *exact* — a pH or temperature
+stored with floating-point drift isn't acceptable in a regulated batch record — and
+the lineage has to be walkable in seconds, not days.
+
+This scenario combines **exact-value time-series** with **upstream graph genealogy**.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the batch genealogy — a final lot derived
+from an intermediate, which came from two raw-material lots, produced on a bioreactor — so
+the upstream walk in step 2 returns a real lineage. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[lbl]) for x, lbl in
+ [("lot_22f_final", "Lot"), ("lot_int_88", "Lot"), ("lot_raw_acme_41", "Lot"),
+ ("lot_raw_acme_42", "Lot"), ("equipment_bioreactor_3", "Equipment")]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("lot_22f_final", "lot_int_88", "derived_from"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("lot_int_88", "lot_raw_acme_41", "derived_from"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("lot_int_88", "lot_raw_acme_42", "derived_from"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("lot_22f_final", "equipment_bioreactor_3", "produced_on")])
+```
+
+## 1. Record process data exactly
+
+A bioreactor's pH, temperature and dissolved oxygen are the batch record. Use the
+`NUMERIC` value type so values store as exact decimals, not floats.
+
+
+
+
+```java
+var ph = new Timeseries();
+ph.setExternalId("batch_22f_ph");
+ph.setName("Batch 22F — pH");
+ph.setValueType("numeric");
+ph.setUnit("ph");
+client.timeseries().create(ph);
+
+client.timeseries().ingest(Map.of(
+ "batch_22f_ph", List.of(Datapoint.of(Instant.now(), "7.0421")))); // exact
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="batch_22f_ph", name="Batch 22F — pH", unit="ph", value_type="float")])
+
+client.timeseries.insert_from_lists(
+ timestamps=[pd.Timestamp.now(tz="UTC")], values=[7.0421], ts="batch_22f_ph")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use chrono::Utc;
+
+let mut ph = TimeSeries::new("batch_22f_ph", "Batch 22F — pH");
+ph.unit = Some("ph".into());
+ph.value_type = Some("numeric".into());
+api.time_series.create_one(&ph).await?;
+
+api.time_series
+ .insert_datapoint(None, Some("batch_22f_ph".into()), Utc::now(), "7.0421".into())
+ .await?;
+```
+
+
+
+
+:::note Exact decimals
+`NUMERIC` stores values without floating-point rounding — see the
+[Time-series reference](/reference/timeseries) for the value types. For a regulated
+batch record, prefer it over the float types.
+:::
+
+## 2. Walk the genealogy upstream
+
+The failed lot `derived_from` intermediate lots, which `derived_from` raw material
+lots; each step `produced_on` a piece of equipment. Walk *up* from the failed lot and
+the returned sub-graph is its complete genealogy — every input and every machine that
+touched it.
+
+
+
+
+```java
+ResourceNetwork lineage = client.resources().fetchRelated("lot_22f_final", 10);
+
+lineage.nodes().forEach(n -> {
+ if (n.getExternalId().startsWith("lot_raw_"))
+ System.out.println("raw material: " + n.getExternalId());
+ if (n.getExternalId().startsWith("equipment_"))
+ System.out.println("equipment: " + n.getExternalId());
+});
+```
+
+
+
+
+```python
+lineage = client.resources.fetch_related(external_id="lot_22f_final", depth=10)
+
+for node in lineage.nodes:
+ if node.external_id.startswith("lot_raw_"):
+ print("raw material:", node.external_id)
+ elif node.external_id.startswith("equipment_"):
+ print("equipment:", node.external_id)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let lineage = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("lot_22f_final").with_depth(10)).await?;
+
+for node in lineage.nodes() {
+ if node.external_id().starts_with("lot_raw_") {
+ println!("raw material: {}", node.external_id());
+ }
+}
+```
+
+
+
+
+If a raw material lot turns up in the genealogy of *other* failed batches too, you've
+found a shared root cause — the same intersection trick as
+[alarm correlation](/guides/correlate-alarms).
+
+## 3. Capture deviations as events
+
+Every out-of-band reading in the batch record is a `process_deviation` event tied to
+the lot, so the review team reconstructs an exact, ordered deviation timeline. See
+[Turn readings into events](/guides/detect-events).
+
+## See the result
+
+Walking up from the failed lot returns its complete genealogy:
+
+```text
+raw material: lot_raw_acme_41
+raw material: lot_raw_acme_42
+equipment: equipment_bioreactor_3
+```
+
+## See also
+
+- [Time-series reference](/reference/timeseries) — exact `NUMERIC` values.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — the genealogy traversal pattern.
+- [Turn readings into events](/guides/detect-events) — the deviation rule.
diff --git a/versioned_docs/version-1.0/industries/manufacturing-process/semiconductor.mdx b/versioned_docs/version-1.0/industries/manufacturing-process/semiconductor.mdx
new file mode 100644
index 0000000..0694f4a
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/manufacturing-process/semiconductor.mdx
@@ -0,0 +1,170 @@
+---
+sidebar_position: 2
+title: Semiconductor — yield & defect tracing
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Semiconductor — yield & defect tracing
+
+**The problem.** In a fab, yield is everything — and a yield drop is a detective story.
+A wafer lot passes through hundreds of process steps on dozens of tools; when lots start
+failing final test, the cause could be any one tool drifting out of spec. The killer
+question is: **what do the failing lots have in common?** Find the one tool or chamber
+that every bad lot passed through and you've found the culprit; miss it and you scrap
+wafers for weeks.
+
+What we solve here is tracing a yield excursion back to its root tool before it costs a
+fortune in scrapped silicon.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create two failing lots that both ran on
+`tool_etch_07` (plus other tools) — so the intersection in step 2 finds the shared
+culprit. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[lbl]) for x, lbl in
+ [("lot_22841", "Lot"), ("lot_22863", "Lot"), ("tool_etch_07", "Tool"),
+ ("tool_litho_03", "Tool"), ("tool_cmp_05", "Tool")]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("lot_22841", "tool_etch_07", "processed_on"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("lot_22841", "tool_litho_03", "processed_on"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("lot_22863", "tool_etch_07", "processed_on"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("lot_22863", "tool_cmp_05", "processed_on")])
+```
+
+## 1. Record process data exactly
+
+Chamber pressure, temperature and gas flow are the process record, and they must be
+exact — use the `NUMERIC` value type so values store without floating-point drift.
+
+
+
+
+```java
+var pressure = new Timeseries();
+pressure.setExternalId("tool_etch_07_chamber_pressure_mtorr");
+pressure.setName("Etch-07 chamber pressure");
+pressure.setValueType("numeric");
+pressure.setUnit("mtorr");
+client.timeseries().create(pressure);
+
+client.timeseries().ingest(Map.of(
+ "tool_etch_07_chamber_pressure_mtorr",
+ List.of(Datapoint.of(Instant.now(), "12.0436"))));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="tool_etch_07_chamber_pressure_mtorr",
+ name="Etch-07 chamber pressure", unit="mtorr", value_type="float")])
+client.timeseries.insert_from_lists(
+ timestamps=[pd.Timestamp.now(tz="UTC")], values=[12.0436],
+ ts="tool_etch_07_chamber_pressure_mtorr")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use chrono::Utc;
+
+let mut p = TimeSeries::new("tool_etch_07_chamber_pressure_mtorr", "Etch-07 chamber pressure");
+p.unit = Some("mtorr".into());
+p.value_type = Some("numeric".into());
+api.time_series.create_one(&p).await?;
+
+api.time_series
+ .insert_datapoint(None, Some("tool_etch_07_chamber_pressure_mtorr".into()), Utc::now(), "12.0436".into())
+ .await?;
+```
+
+
+
+
+## 2. Find the tool every failing lot shares
+
+Each lot is connected to the tools it ran on (`processed_on`). Walk the graph from each
+failing lot and intersect: the tool common to all of them is the prime suspect — the
+same shared-node reasoning as [alarm correlation](/guides/correlate-alarms), applied to
+process genealogy.
+
+
+
+
+```java
+ResourceNetwork lotA = client.resources().fetchRelated("lot_22841", 6);
+ResourceNetwork lotB = client.resources().fetchRelated("lot_22863", 6);
+
+Set toolsA = lotA.nodes().stream()
+ .map(NodeModel::getExternalId).filter(id -> id.startsWith("tool_")).collect(Collectors.toSet());
+Set suspect = lotB.nodes().stream()
+ .map(NodeModel::getExternalId).filter(toolsA::contains).collect(Collectors.toSet());
+
+// suspect contains "tool_etch_07" → the chamber to pull and check
+```
+
+
+
+
+```python
+lot_a = client.resources.fetch_related(external_id="lot_22841", depth=6)
+lot_b = client.resources.fetch_related(external_id="lot_22863", depth=6)
+
+tools_a = {n.external_id for n in lot_a.nodes if n.external_id.startswith("tool_")}
+suspect = tools_a & {n.external_id for n in lot_b.nodes}
+# 'tool_etch_07' in suspect → pull and check that chamber
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+use std::collections::HashSet;
+
+let lot_a = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("lot_22841").with_depth(6)).await?;
+let lot_b = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("lot_22863").with_depth(6)).await?;
+
+let tools_a: HashSet<&str> = lot_a.nodes().iter()
+ .map(|n| n.external_id()).filter(|id| id.starts_with("tool_")).collect();
+let suspect: Vec<&str> = lot_b.nodes().iter()
+ .map(|n| n.external_id()).filter(|id| tools_a.contains(id)).collect();
+// suspect contains "tool_etch_07"
+```
+
+
+
+
+## 3. Confirm against the tool's process record
+
+With the suspect tool identified, pull its exact process series around the failing lots'
+run times and compare to a known-good window — the drift that explains the excursion
+should be right there. See [Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The tools common to both failing lots:
+
+```text
+suspect contains 'tool_etch_07' ← the chamber to pull and check
+```
+
+## See also
+
+- [Time-series reference](/reference/timeseries) — exact `NUMERIC` process values.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — the shared-tool intersection.
+- [Query & aggregate](/guides/query-and-aggregate) — comparing tool records.
diff --git a/versioned_docs/version-1.0/industries/mining-metals/_category_.json b/versioned_docs/version-1.0/industries/mining-metals/_category_.json
new file mode 100644
index 0000000..0bdf038
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/mining-metals/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Mining & metals",
+ "position": 7,
+ "link": { "type": "generated-index", "title": "Mining & metals" }
+}
diff --git a/versioned_docs/version-1.0/industries/mining-metals/operations.mdx b/versioned_docs/version-1.0/industries/mining-metals/operations.mdx
new file mode 100644
index 0000000..5259f78
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/mining-metals/operations.mdx
@@ -0,0 +1,185 @@
+---
+sidebar_position: 1
+title: Mining — equipment & throughput
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Mining — equipment uptime & throughput
+
+**The problem.** A mine's economics live and die on its haul fleet and processing
+plant. A haul truck stopped on the ramp is tonnes-per-hour evaporating; a crusher that
+fails unexpectedly idles everything behind it. The site needs to **catch equipment
+trending toward failure before it stops the line**, and to see true throughput against
+plan so it can act on the bottleneck rather than the loudest radio call.
+
+What we solve here is unplanned downtime and hidden bottlenecks across a fleet of
+expensive, hard-worked machines.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a haul truck's oil-pressure series (sagging
+below its limit) and a crusher throughput series. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="truck_785_oil_pressure_kpa", name="Truck 785 oil pressure", unit="kpa", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=60, freq="1min")
+op = np.full(60, 320.0); op[-15:] -= np.linspace(0, 90, 15) # sags below 250 kPa
+client.timeseries.insert_from_lists(timestamps=idx, values=op, ts="truck_785_oil_pressure_kpa")
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="crusher_01_throughput_tph", name="Crusher 01 throughput", unit="tph", value_type="float")])
+idx2 = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=12, freq="1h")
+client.timeseries.insert_from_lists(timestamps=idx2, values=np.random.uniform(1800, 2200, 12), ts="crusher_01_throughput_tph")
+```
+
+## 1. Spot a machine trending to failure
+
+Haul-truck engine temperature, oil pressure and payload stream in. A drifting reading
+raises an `equipment_warning` before the breakdown, so maintenance is scheduled, not
+scrambled. See [Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+var series = client.timeseries().retrieve(lastHourOf("truck_785_oil_pressure_kpa"))
+ .getItems().get(0);
+
+if (series.getDatapoints().stream().anyMatch(p -> Double.parseDouble(p.getValue()) < 250.0)) {
+ EventModel warning = new EventModel();
+ warning.setExternalId("equipment_warning_785_" + System.currentTimeMillis());
+ warning.setType("equipment_warning");
+ warning.setStatus("open");
+ warning.setMetadata(Map.of("unit", "truck_785", "signal", "oil_pressure"));
+ warning.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(warning));
+}
+```
+
+
+
+
+```python
+points = client.timeseries.retrieve_datapoints(last_hour("truck_785_oil_pressure_kpa"))[0]
+if any(float(dp.value) < 250.0 for dp in points.get_datapoints()):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"equipment_warning_785_{int(pd.Timestamp.now().timestamp())}",
+ type="equipment_warning", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"unit": "truck_785", "signal": "oil_pressure"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let page = api.time_series.retrieve_datapoints(&last_hour("truck_785_oil_pressure_kpa")).await?;
+let points = &page.get_items()[0];
+let low = points.datapoints.iter()
+ .any(|p| p.value.unwrap_or(999.0) < 250.0);
+if low {
+ let mut warning = Event::new(
+ format!("equipment_warning_785_{}", Utc::now().timestamp()),
+ "equipment_warning".into(),
+ Utc::now());
+ warning.status = Some("open".into());
+ warning.add_metadata("unit".into(), "truck_785".into());
+ warning.add_metadata("signal".into(), "oil_pressure".into());
+ api.events.create(&vec![warning]).await?;
+}
+```
+
+
+
+
+## 2. See throughput against plan
+
+Payload-per-cycle and cycle counts are series per truck; roll them up to tonnes-per-
+hour by hour and shift to find where the real bottleneck is — the loader, the haul, or
+the crusher. See [Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("crusher_01_throughput_tph");
+filter.setStart(ZonedDateTime.now().minusHours(12));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> compareToPlan(p.getTimestamp(), p.getValue()));
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="crusher_01_throughput_tph",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=12),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ compare_to_plan(dp.timestamp, dp.average)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("crusher_01_throughput_tph".into()),
+ aggregates: Some(vec!["avg".into()]),
+ granularity: Some("1h".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(12)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { compare_to_plan(&p.timestamp, p.average); }
+```
+
+
+
+
+## 3. Tie equipment to its place in the flow
+
+Model pit → loader → truck → crusher as a [graph](/guides/model-assets-graph) so a
+crusher fault's [blast radius](/guides/correlate-alarms) — every truck that feeds it —
+is one traversal away when you plan the workaround.
+
+## See the result
+
+The sagging oil pressure trips the predictive rule before the breakdown:
+
+```text
+equipment_warning_785_… → open (oil pressure fell below 250 kPa)
+```
+
+## See also
+
+- [Turn readings into events](/guides/detect-events) — the predictive-maintenance rule.
+- [Query & aggregate](/guides/query-and-aggregate) — tonnes-per-hour roll-ups.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — a fault's downstream impact.
+- **[Predictive maintenance (advanced)](/advanced/predictive-maintenance)** — anomaly detection on haul-truck vibration.
diff --git a/versioned_docs/version-1.0/industries/mining-metals/processing-plant.mdx b/versioned_docs/version-1.0/industries/mining-metals/processing-plant.mdx
new file mode 100644
index 0000000..3fd2840
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/mining-metals/processing-plant.mdx
@@ -0,0 +1,175 @@
+---
+sidebar_position: 2
+title: Mining — processing plant
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Mining — processing plant recovery
+
+**The problem.** After the rock is hauled, the money is made (or lost) in the
+concentrator: how much of the valuable metal the mill actually **recovers** from the
+ore. A recovery that quietly slips a couple of percent — from a grind that's too coarse,
+a reagent dose that's off, a feed grade that changed — is metal going out with the
+tailings, worth a fortune over a shift. Metallurgists need recovery and throughput as
+live, rolled-up numbers so they can act on a dip while the ore is still in the circuit.
+
+What we solve here is protecting metal recovery by turning mill telemetry into the KPIs
+the control room steers on.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the mill's recovery series with a dip below
+target. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="mill_1_recovery_pct", name="Mill 1 recovery %", unit="pct", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=12, freq="1h")
+rec = np.full(12, 91.0); rec[-4:] -= np.linspace(0, 3, 4) # dips below the 91% target
+client.timeseries.insert_from_lists(timestamps=idx, values=rec, ts="mill_1_recovery_pct")
+```
+
+## 1. Track recovery and throughput
+
+Feed grade, concentrate grade, tailings grade and tonnage are series; recovery is
+computed from them. Roll them up hourly to see recovery against target. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("mill_1_recovery_pct");
+filter.setStart(ZonedDateTime.now().minusHours(12));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("avg"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> compareToTarget(p.getTimestamp(), p.getValue())); // % recovery
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="mill_1_recovery_pct",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=12),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ compare_to_target(dp.timestamp, dp.average)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("mill_1_recovery_pct".into()),
+ aggregates: Some(vec!["avg".into()]),
+ granularity: Some("1h".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(12)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { compare_to_target(&p.timestamp, p.average); }
+```
+
+
+
+
+## 2. Flag a recovery dip
+
+When recovery drops below target for a sustained period, raise a `recovery_loss` event so
+the metallurgist investigates grind, reagents or feed before more metal is lost. See
+[Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+EventModel loss = new EventModel();
+loss.setExternalId("recovery_loss_mill1_" + System.currentTimeMillis());
+loss.setType("recovery_loss");
+loss.setStatus("open");
+loss.setMetadata(Map.of("mill", "mill_1", "recovery_pct", "88.4", "target_pct", "91.0"));
+loss.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(loss));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"recovery_loss_mill1_{int(pd.Timestamp.now().timestamp())}",
+ type="recovery_loss", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"mill": "mill_1", "recovery_pct": "88.4", "target_pct": "91.0"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut loss = Event::new(
+ format!("recovery_loss_mill1_{}", Utc::now().timestamp()),
+ "recovery_loss".into(),
+ Utc::now());
+loss.status = Some("open".into());
+loss.add_metadata("mill".into(), "mill_1".into());
+loss.add_metadata("recovery_pct".into(), "88.4".into());
+loss.add_metadata("target_pct".into(), "91.0".into());
+api.events.create(&vec![loss]).await?;
+```
+
+
+
+
+## 3. Find the best operating point
+
+Recovery, grind and reagents interact in complex ways — [cluster the operating
+states](/advanced/kmeans-clustering) to discover which regimes recover best, or build a
+[soft sensor](/advanced/random-forest-soft-sensor) to estimate recovery between assays.
+
+## See the result
+
+Chart recovery against target — the dip is metal walking out with the tailings:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="mill_1_recovery_pct",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=12), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["avg"], granularity="1h")
+v = [dp.average for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.plot(v); plt.axhline(91, color="r", ls="--"); plt.title("Mill 1 recovery (%)"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — recovery and throughput KPIs.
+- [Turn readings into events](/guides/detect-events) — the recovery-dip rule.
+- [Mining operations](/industries/mining-metals/operations) — the haul fleet that feeds the mill.
diff --git a/versioned_docs/version-1.0/industries/mining-metals/tailings-safety.mdx b/versioned_docs/version-1.0/industries/mining-metals/tailings-safety.mdx
new file mode 100644
index 0000000..2f1cdaa
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/mining-metals/tailings-safety.mdx
@@ -0,0 +1,134 @@
+---
+sidebar_position: 3
+title: Mining — tailings dam safety
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Mining — tailings dam safety
+
+**The problem.** A tailings storage facility — the dam holding the mine's waste slurry —
+is one of the highest-consequence structures a company operates. A failure is
+catastrophic: lives, communities and the licence to operate. The warning signs are
+**slow and subtle**: pore-water pressure creeping up inside the embankment, the phreatic
+(water) surface rising, millimetres of movement accumulating. They're invisible day to
+day and only obvious in the trend. Engineers need those instruments watched continuously
+against trigger levels, with an alarm the instant one is crossed.
+
+What we solve here is the continuous, trigger-based monitoring a tailings dam's safety
+case depends on.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a piezometer's pore-pressure series,
+**create the subscription before we listen**, and feed in a pressure that climbs past its
+trigger. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="piezometer_p14_pore_pressure_kpa", name="Piezometer P14 pore pressure", unit="kpa", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="tsf_instruments", name="Tailings instruments", timeseries=["piezometer_p14_pore_pressure_kpa"])])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=90, freq="1h")
+client.timeseries.insert_from_lists(timestamps=idx, values=150 + np.linspace(0, 40, 90), # rises past ~180 trigger
+ ts="piezometer_p14_pore_pressure_kpa")
+```
+
+## 1. Watch the instruments against trigger levels
+
+Piezometers (pore pressure), phreatic levels and movement prisms stream in. A reading
+past its **trigger level** raises a `tailings_alarm` — graded by severity — straight to
+the engineer of record. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("tsf_instruments"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ String level = triggerLevel(msg.payload()); // null | "amber" | "red"
+ if (level != null) {
+ EventModel alarm = new EventModel();
+ alarm.setExternalId("tailings_alarm_p14_" + System.currentTimeMillis());
+ alarm.setType("tailings_alarm");
+ alarm.setStatus(level.equals("red") ? "critical" : "warning");
+ alarm.setMetadata(Map.of("instrument", "piezometer_p14", "level", level,
+ "pore_pressure_kpa", "186"));
+ alarm.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(alarm));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["tsf_instruments"]) as listener:
+ for msg in listener:
+ level = trigger_level(msg.payload) # None | "amber" | "red"
+ if level:
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"tailings_alarm_p14_{int(pd.Timestamp.now().timestamp())}",
+ type="tailings_alarm",
+ status="critical" if level == "red" else "warning",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"instrument": "piezometer_p14", "level": level,
+ "pore_pressure_kpa": "186"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["tsf_instruments"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if let Some(level) = trigger_level(&msg.payload) { // "amber" | "red"
+ let mut alarm = Event::new(
+ format!("tailings_alarm_p14_{}", Utc::now().timestamp()),
+ "tailings_alarm".into(),
+ Utc::now());
+ alarm.status = Some(if level == "red" { "critical" } else { "warning" }.into());
+ alarm.add_metadata("instrument".into(), "piezometer_p14".into());
+ alarm.add_metadata("level".into(), level.into());
+ alarm.add_metadata("pore_pressure_kpa".into(), "186".into());
+ api.events.create(&vec![alarm]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Watch the trend, not just the level
+
+A single reading rarely fails a dam; a rising *trend* is the real signal. Roll pore
+pressure up to daily values and watch the slope — a steady climb toward the trigger is
+the early warning the safety case is built on. See [Query & aggregate](/guides/query-and-aggregate),
+and for an ahead-of-time call, the [early-warning model](/advanced/oxygen-crash-early-warning).
+
+## See the result
+
+The climbing pore pressure crosses its trigger and raises a graded alarm:
+
+```text
+tailings_alarm_p14_… → warning (pore pressure past its amber trigger level)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — continuous instrument monitoring.
+- [Turn readings into events](/guides/detect-events) — trigger-level alarms.
+- [Water utilities](/industries/energy-utilities/water) — related dam/flow-network monitoring.
diff --git a/versioned_docs/version-1.0/industries/oil-and-gas/_category_.json b/versioned_docs/version-1.0/industries/oil-and-gas/_category_.json
new file mode 100644
index 0000000..5e61981
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/oil-and-gas/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Oil & gas",
+ "position": 1,
+ "link": { "type": "generated-index", "title": "Oil & gas" }
+}
diff --git a/versioned_docs/version-1.0/industries/oil-and-gas/drilling.mdx b/versioned_docs/version-1.0/industries/oil-and-gas/drilling.mdx
new file mode 100644
index 0000000..045f45f
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/oil-and-gas/drilling.mdx
@@ -0,0 +1,151 @@
+---
+sidebar_position: 1
+title: Oil & gas — drilling operations
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Oil & gas — drilling operations
+
+**The problem.** Drilling a well is the most dangerous and expensive phase of upstream
+oil & gas. A **kick** — formation fluid pushing into the wellbore — can escalate to a
+blowout in minutes if it isn't caught; a **stuck pipe** can cost millions and lose the
+well. The signature is in the data the rig already streams: when mud flowing *out*
+starts to exceed mud pumped *in*, and the pit volume creeps up, a kick is developing.
+The driller needs that flagged the instant it starts, not after the trip tank confirms
+it.
+
+What we solve here is catching a well-control event in its first seconds, while it can
+still be shut in safely.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the rig's series, **create the
+subscription before we listen**, and feed in five minutes of flow that ends in a
+developing kick — so the loop below has something real to catch. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+channels = ["rig_dw1_flow_in_gpm", "rig_dw1_flow_out_gpm", "rig_dw1_pit_volume_bbl"]
+units = {"rig_dw1_flow_in_gpm": "gpm", "rig_dw1_flow_out_gpm": "gpm", "rig_dw1_pit_volume_bbl": "bbl"}
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=c, name=c, unit=units[c], value_type="float")
+ for c in channels])
+
+# the subscription — created up front, before the listen loop in step 1
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="rig_deepwater_1", name="Rig Deepwater 1", timeseries=channels)])
+
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=300, freq="1s")
+flow_out = np.full(300, 600.0); flow_out[-60:] += np.linspace(0, 25, 60) # kick: flow-out climbs
+pit = np.full(300, 120.0); pit[-60:] += np.linspace(0, 4, 60) # pit volume rises
+client.timeseries.insert_from_lists(timestamps=idx, values=np.full(300, 600.0), ts="rig_dw1_flow_in_gpm")
+client.timeseries.insert_from_lists(timestamps=idx, values=flow_out, ts="rig_dw1_flow_out_gpm")
+client.timeseries.insert_from_lists(timestamps=idx, values=pit, ts="rig_dw1_pit_volume_bbl")
+```
+
+## 1. Stream the drilling signals live
+
+Weight-on-bit, standpipe pressure, and mud flow in/out stream from the rig into a
+subscription. The moment flow-out runs ahead of flow-in while pit volume rises, raise a
+`kick_detected` alarm. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("rig_deepwater_1"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (influxDetected(msg.payload())) { // flow_out > flow_in & pit rising
+ EventModel kick = new EventModel();
+ kick.setExternalId("kick_detected_a12_" + System.currentTimeMillis());
+ kick.setType("kick_detected");
+ kick.setStatus("critical");
+ kick.setMetadata(Map.of("well", "well_a12", "rig", "rig_deepwater_1",
+ "flow_delta_gpm", "18"));
+ kick.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(kick));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["rig_deepwater_1"]) as listener:
+ for msg in listener:
+ if influx_detected(msg.payload): # flow_out > flow_in & pit rising
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"kick_detected_a12_{int(pd.Timestamp.now().timestamp())}",
+ type="kick_detected", status="critical",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"well": "well_a12", "rig": "rig_deepwater_1", "flow_delta_gpm": "18"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["rig_deepwater_1"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if influx_detected(&msg.payload) { // flow_out > flow_in & pit rising
+ let mut kick = Event::new(
+ format!("kick_detected_a12_{}", Utc::now().timestamp()),
+ "kick_detected".into(),
+ Utc::now());
+ kick.status = Some("critical".into());
+ kick.add_metadata("well".into(), "well_a12".into());
+ kick.add_metadata("rig".into(), "rig_deepwater_1".into());
+ kick.add_metadata("flow_delta_gpm".into(), "18".into());
+ api.events.create(&vec![kick]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Learn the drilling state, not just one signal
+
+A kick rarely shows in any single channel — it's the *pattern* across flow, pressure
+and pit volume over a few seconds that gives it away. A model that watches all the
+drilling signals together as a sequence catches it earlier and with fewer false alarms
+than a per-channel threshold. That's exactly what the
+[LSTM anomaly detector](/advanced/lstm-anomaly-detection) does.
+
+## 3. Cut non-productive time
+
+Rate-of-penetration, torque and connection times are series per well section; daily
+roll-ups surface the formations and crews where non-productive time accumulates, the
+single biggest controllable cost in a drilling programme. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+Run the setup, then the loop above. The demo kick (flow-out climbing past flow-in) trips
+the detector and a `kick_detected` event lands — confirm it:
+
+```python
+hits = client.events.filter(type="kick_detected", limit=5)
+for e in hits:
+ print(e.external_id, "→", e.status)
+# kick_detected_a12_1751… → critical
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the well-control monitoring loop.
+- **[LSTM anomaly detection (advanced)](/advanced/lstm-anomaly-detection)** — catch a kick from the multivariate signal pattern.
+- [Oil & gas — production monitoring](/industries/oil-and-gas/production) — once the well is producing.
diff --git a/versioned_docs/version-1.0/industries/oil-and-gas/emissions.mdx b/versioned_docs/version-1.0/industries/oil-and-gas/emissions.mdx
new file mode 100644
index 0000000..820f5be
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/oil-and-gas/emissions.mdx
@@ -0,0 +1,172 @@
+---
+sidebar_position: 6
+title: Oil & gas — emissions & flaring
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Oil & gas — emissions & flaring (ESG)
+
+**The problem.** Methane and routine flaring are now under hard scrutiny — from
+regulators with reporting mandates and penalties, and from investors tracking ESG
+performance. An operator has to **measure flare volumes and methane, report them
+accurately, and catch excess flaring** before it becomes a violation or a headline.
+The data exists across every platform; the job is turning it into auditable monthly
+numbers and timely alerts.
+
+What we solve here is verifiable emissions reporting and early detection of excess
+flaring.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the platform's flare-volume series with
+a month of daily readings, so the monthly roll-up in step 1 has data. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="platform_north_flare_volume_m3", name="North platform flare volume", unit="m3", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=30, freq="1d")
+client.timeseries.insert_from_lists(
+ timestamps=idx, values=np.random.uniform(3000, 5000, 30), ts="platform_north_flare_volume_m3")
+```
+
+## 1. Roll emissions up for reporting
+
+Flare volume and methane rate are series per site. Monthly totals — and a CO₂-equivalent
+from them — are the numbers the regulatory submission and the ESG dashboard are built on.
+See [Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("platform_north_flare_volume_m3");
+filter.setStart(ZonedDateTime.now().minusDays(30));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("sum"));
+filter.setGranularity("1d");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+double monthly = client.timeseries().retrieve(request).getItems().get(0)
+ .getDatapoints().stream().mapToDouble(p -> Double.parseDouble(p.getValue())).sum();
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="platform_north_flare_volume_m3",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=30),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1d")
+
+monthly = sum(dp.sum for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints())
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("platform_north_flare_volume_m3".into()),
+ start: Some(Utc::now() - chrono::Duration::days(30)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["sum".into()]),
+ granularity: Some("1d".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+let monthly: f64 = series.datapoints.iter()
+ .filter_map(|p| p.sum).sum();
+```
+
+
+
+
+## 2. Catch excess flaring early
+
+When a site's daily flaring crosses its permitted rate, raise a `flaring_exceedance`
+event — with the volume and limit in metadata — so the operator corrects and the
+compliance team has a dated record. See [Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+EventModel exceedance = new EventModel();
+exceedance.setExternalId("flaring_exceedance_north_" + System.currentTimeMillis());
+exceedance.setType("flaring_exceedance");
+exceedance.setStatus("open");
+exceedance.setMetadata(Map.of("site", "platform_north", "flared_m3", "142000", "permit_m3", "120000"));
+exceedance.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(exceedance));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"flaring_exceedance_north_{int(pd.Timestamp.now().timestamp())}",
+ type="flaring_exceedance", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"site": "platform_north", "flared_m3": "142000", "permit_m3": "120000"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut exceedance = Event::new(
+ format!("flaring_exceedance_north_{}", Utc::now().timestamp()),
+ "flaring_exceedance".into(),
+ Utc::now());
+exceedance.status = Some("open".into());
+exceedance.add_metadata("site".into(), "platform_north".into());
+exceedance.add_metadata("flared_m3".into(), "142000".into());
+exceedance.add_metadata("permit_m3".into(), "120000".into());
+api.events.create(&vec![exceedance]).await?;
+```
+
+
+
+
+## See the result
+
+Print the month's total and chart the daily flaring — the bar chart is what the ESG
+dashboard shows:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="platform_north_flare_volume_m3",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=30), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1d")
+daily = [dp.sum for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+print(f"monthly flaring: {sum(daily):,.0f} m³") # ~120,000 m³
+plt.bar(range(len(daily)), daily); plt.title("North platform — daily flaring (m³)"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — monthly emissions totals.
+- [Turn readings into events](/guides/detect-events) — the exceedance rule.
+- [Environmental — air quality](/industries/energy-utilities/air-quality) — the same monitoring-and-report pattern.
diff --git a/versioned_docs/version-1.0/industries/oil-and-gas/pipeline.mdx b/versioned_docs/version-1.0/industries/oil-and-gas/pipeline.mdx
new file mode 100644
index 0000000..bbb946d
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/oil-and-gas/pipeline.mdx
@@ -0,0 +1,182 @@
+---
+sidebar_position: 3
+title: Oil & gas — pipeline integrity
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Oil & gas — pipeline integrity
+
+**The problem.** A pipeline leak is an environmental and financial emergency, and the
+two questions that follow are both urgent: *is* there a leak, and *where*. The signal
+for the first is a mass imbalance — what flows into a segment should equal what flows
+out, and a sustained gap with a pressure drop means product is escaping. The second is
+a graph question: given the leaking segment, which valves isolate it and what's
+downstream.
+
+What we solve here is detecting a leak from the flow balance and locating it on the
+network so the right valves close fast.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the segment's flow series (with an
+inflow/outflow gap — a leak) and the pipeline graph the locate step walks. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+# flow series: inflow 1000 vs outflow 940 m³/h → a 60 m³/h imbalance (leak)
+for s, v in [("segment_pn_07_flow_in_m3h", 1000.0), ("segment_pn_07_flow_out_m3h", 940.0)]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=s, name=s, unit="m3h", value_type="float")])
+ client.timeseries.insert_from_lists(timestamps=[pd.Timestamp.now(tz="UTC")], values=[v], ts=s)
+
+# the network: stations, the segment, and the valves that isolate it
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[x]) for x in
+ ["station_kollsnes", "segment_pn_07", "station_mongstad", "valve_v18", "valve_v19"]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("station_kollsnes", "segment_pn_07", "connects_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("segment_pn_07", "station_mongstad", "connects_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("segment_pn_07", "valve_v18", "isolated_by"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("segment_pn_07", "valve_v19", "isolated_by")])
+```
+
+You'll also need small `latest(id)` and `tolerance(flow)` helpers — `latest` reads the
+last value of a series, `tolerance` returns an acceptable gap (say 1% of inflow).
+
+## 1. Detect the imbalance
+
+Each segment reports inlet and outlet flow and pressure. A sustained inlet-minus-outlet
+gap raises a `leak_suspected` event scoped to the segment. See
+[Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+double inflow = latest("segment_pn_07_flow_in_m3h");
+double outflow = latest("segment_pn_07_flow_out_m3h");
+
+if (inflow - outflow > tolerance(inflow)) {
+ EventModel leak = new EventModel();
+ leak.setExternalId("leak_suspected_pn07_" + System.currentTimeMillis());
+ leak.setType("leak_suspected");
+ leak.setStatus("critical");
+ leak.setMetadata(Map.of("segment", "segment_pn_07",
+ "imbalance_m3h", String.valueOf(inflow - outflow)));
+ leak.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(leak));
+}
+```
+
+
+
+
+```python
+inflow = latest("segment_pn_07_flow_in_m3h")
+outflow = latest("segment_pn_07_flow_out_m3h")
+
+if inflow - outflow > tolerance(inflow):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"leak_suspected_pn07_{int(pd.Timestamp.now().timestamp())}",
+ type="leak_suspected", status="critical",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"segment": "segment_pn_07", "imbalance_m3h": str(inflow - outflow)})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let inflow: f64 = latest("segment_pn_07_flow_in_m3h").await?;
+let outflow: f64 = latest("segment_pn_07_flow_out_m3h").await?;
+
+if inflow - outflow > tolerance(inflow) {
+ let mut leak = Event::new(
+ format!("leak_suspected_pn07_{}", Utc::now().timestamp()),
+ "leak_suspected".into(),
+ Utc::now());
+ leak.status = Some("critical".into());
+ leak.add_metadata("segment".into(), "segment_pn_07".into());
+ leak.add_metadata("imbalance_m3h".into(), (inflow - outflow).to_string());
+ api.events.create(&vec![leak]).await?;
+}
+```
+
+
+
+
+## 2. Locate it and find the isolation valves
+
+Model the pipeline as a graph — stations joined by segments, valves on the segments.
+Walk out from the leaking segment to find the valves that bound it and everything
+downstream that must be isolated. This is the
+[connectivity traversal](/guides/correlate-alarms) applied to a flow network.
+
+
+
+
+```java
+RelatedResourcesForm form = new RelatedResourcesForm();
+form.setExternalId("segment_pn_07");
+form.setDepth(3);
+form.setRelationshipTypes(List.of("connects_to", "isolated_by"));
+
+ResourceNetwork around = client.resources().fetchRelated(form);
+around.nodes().stream()
+ .filter(n -> n.getExternalId().startsWith("valve_"))
+ .forEach(v -> System.out.println("close: " + v.getExternalId()));
+```
+
+
+
+
+```python
+around = client.resources.fetch_related(
+ external_id="segment_pn_07", depth=3, relationship_types=["connects_to", "isolated_by"])
+
+for node in around.nodes:
+ if node.external_id.startswith("valve_"):
+ print("close:", node.external_id)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let around = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("segment_pn_07")
+ .with_depth(3)
+ .with_relationship_types(vec!["connects_to".into(), "isolated_by".into()])).await?;
+
+for node in around.nodes() {
+ if node.external_id().starts_with("valve_") {
+ println!("close: {}", node.external_id());
+ }
+}
+```
+
+
+
+
+## See the result
+
+With the demo leak in place (inflow 1000 vs outflow 940), step 1 raises a
+`leak_suspected` event and step 2 prints the valves that bound the segment:
+
+```text
+close: valve_v18
+close: valve_v19
+```
+
+## See also
+
+- [Turn readings into events](/guides/detect-events) — the mass-balance rule.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — locating on the network.
+- [Water utilities](/industries/energy-utilities/water) — the same flow-graph reasoning.
diff --git a/versioned_docs/version-1.0/industries/oil-and-gas/production.mdx b/versioned_docs/version-1.0/industries/oil-and-gas/production.mdx
new file mode 100644
index 0000000..228f451
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/oil-and-gas/production.mdx
@@ -0,0 +1,334 @@
+---
+sidebar_position: 2
+title: Oil & gas — production monitoring
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Oil & gas — upstream production monitoring
+
+**The problem.** An offshore field has dozens of wells, each with an electric
+submersible pump (ESP) and a cluster of sensors — wellhead pressure, casing
+temperature, flow rate. Engineers in an onshore control room need a live picture of
+every well, fast roll-ups for daily production reports, and an alarm the moment a
+pump trends toward failure.
+
+This scenario wires that together: model the field as a graph, stream sensor data in
+at volume, and raise an event when a reading goes out of bounds.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the well's sensor series and feed in a
+pump-intake pressure that sags toward the anomaly threshold — so the alarm in step 3
+actually fires. The field/well/pump graph is created by step 1 below; this also adds the
+two temperature sensors and the shared `cooling_system` that step 4 walks between, so the
+correlation there returns something. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+sensors = {"wellhead_pressure_bar": "bar", "casing_temperature_c": "deg_c",
+ "flow_rate_bpd": "bpd", "pump_intake_pressure_bar": "bar"}
+for s, u in sensors.items():
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=s, name=s, unit=u, value_type="float")])
+
+# intake pressure: healthy ~95 bar, then sagging below the 80-bar ESP-anomaly line
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=120, freq="1min")
+intake = np.full(120, 95.0); intake[-30:] -= np.linspace(0, 20, 30)
+client.timeseries.insert_from_lists(timestamps=idx, values=intake, ts="pump_intake_pressure_bar")
+
+# step 4 walks out from two alarmed temperature sensors and looks for what they share
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=n, labels=[lbl]) for x, n, lbl in
+ [("cooling_system", "Platform cooling system", "System"),
+ ("wellhead_temp_a12", "Wellhead temperature A-12", "Sensor"),
+ ("pump_temp_a12", "ESP motor temperature A-12", "Sensor")]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("wellhead_temp_a12", "cooling_system", "PART_OF"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("pump_temp_a12", "cooling_system", "PART_OF")])
+```
+
+## 1. Model the field
+
+A field *contains* wells; a well *contains* a pump and *produces* sensor series.
+Modeling it as a graph lets you walk from a field down to any individual sensor.
+
+
+
+
+```java
+Resource field = new Resource();
+field.setExternalId("field_north_sea");
+field.setName("North Sea field");
+field.setLabels(List.of("Field"));
+
+Resource well = new Resource();
+well.setExternalId("well_a12");
+well.setName("Well A-12");
+well.setLabels(List.of("Well"));
+
+Resource pump = new Resource();
+pump.setExternalId("pump_esp_a12");
+pump.setName("ESP — Well A-12");
+pump.setLabels(List.of("Pump"));
+
+RelForm fieldWell = new RelForm();
+fieldWell.setName("contains");
+fieldWell.setFromExternalId("field_north_sea");
+fieldWell.setToExternalId("well_a12");
+
+RelForm wellPump = new RelForm();
+wellPump.setName("contains");
+wellPump.setFromExternalId("well_a12");
+wellPump.setToExternalId("pump_esp_a12");
+
+client.resources().create(List.of(field, well, pump), List.of(fieldWell, wellPump));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+nodes = [
+ intellistream_datahub_sdk.Resource(external_id="field_north_sea", name="North Sea field", labels=["Field"]),
+ intellistream_datahub_sdk.Resource(external_id="well_a12", name="Well A-12", labels=["Well"]),
+ intellistream_datahub_sdk.Resource(external_id="pump_esp_a12", name="ESP — Well A-12", labels=["Pump"]),
+]
+edges = [
+ intellistream_datahub_sdk.RelForm.by_external_ids("field_north_sea", "well_a12", "contains"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("well_a12", "pump_esp_a12", "contains"),
+]
+client.resources.create(nodes, edges)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut field = Resource::new();
+field.external_id = "field_north_sea".into();
+field.name = "North Sea field".into();
+field.labels = Some(vec!["Field".into()]);
+
+let mut well = Resource::new();
+well.external_id = "well_a12".into();
+well.name = "Well A-12".into();
+well.labels = Some(vec!["Well".into()]);
+
+let mut pump = Resource::new();
+pump.external_id = "pump_esp_a12".into();
+pump.name = "ESP — Well A-12".into();
+pump.labels = Some(vec!["Pump".into()]);
+
+api.resources.create(
+ vec![field, well, pump],
+ vec![
+ RelForm::by_external_ids("field_north_sea", "well_a12", "contains"),
+ RelForm::by_external_ids("well_a12", "pump_esp_a12", "contains"),
+ ],
+).await?;
+```
+
+
+
+
+Create a series per sensor the same way — `wellhead_pressure_bar`,
+`casing_temperature_c`, `flow_rate_bpd` — and relate each to `well_a12`.
+
+## 2. Stream sensor data at volume
+
+Each well emits readings every second across many sensors — easily millions of
+points an hour at field scale. Hand them to the ingester grouped by series; it
+chunks and parallelises the writes. See [high-throughput ingestion](/guides/ingest-timeseries)
+for the full mechanics.
+
+
+
+
+```java
+client.timeseries().ingest(Map.of(
+ "wellhead_pressure_bar", pressureReadings, // List
+ "casing_temperature_c", temperatureReadings,
+ "flow_rate_bpd", flowReadings));
+```
+
+
+
+
+```python
+client.timeseries.insert_from_lists(
+ timestamps=timestamps, values=pressure_bar, ts="wellhead_pressure_bar")
+client.timeseries.insert_from_lists(
+ timestamps=timestamps, values=temperature_c, ts="casing_temperature_c")
+client.timeseries.insert_from_lists(
+ timestamps=timestamps, values=flow_bpd, ts="flow_rate_bpd")
+```
+
+
+
+
+```rust
+api.time_series
+ .insert_datapoint(None, Some("wellhead_pressure_bar".into()), ts, "248.6".into())
+ .await?;
+api.time_series
+ .insert_datapoint(None, Some("casing_temperature_c".into()), ts, "92.0".into())
+ .await?;
+api.time_series
+ .insert_datapoint(None, Some("flow_rate_bpd".into()), ts, "1850.0".into())
+ .await?;
+```
+
+
+
+
+## 3. Alarm on an out-of-bounds reading
+
+A creeping intake pressure is an early sign of an ESP problem. Read the recent
+window and raise an `esp_anomaly` event when it crosses the line — the control room
+queries open events for the field, and the event references the well it concerns.
+
+
+
+
+```java
+var series = client.timeseries().retrieve(lastHourOf("pump_intake_pressure_bar"))
+ .getItems().get(0);
+
+if (series.getDatapoints().stream().anyMatch(p -> Double.parseDouble(p.getValue()) < 80.0)) {
+ EventModel event = new EventModel();
+ event.setExternalId("esp_anomaly_a12_" + System.currentTimeMillis());
+ event.setType("esp_anomaly");
+ event.setStatus("open");
+ event.setMetadata(Map.of("well", "well_a12", "pump", "pump_esp_a12"));
+ event.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(event));
+}
+```
+
+
+
+
+```python
+points = client.timeseries.retrieve_datapoints(last_hour("pump_intake_pressure_bar"))[0]
+if any(float(dp.value) < 80.0 for dp in points.get_datapoints()):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"esp_anomaly_a12_{int(pd.Timestamp.now().timestamp())}",
+ type="esp_anomaly", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"well": "well_a12", "pump": "pump_esp_a12"})])
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::events::Event;
+
+let page = api.time_series.retrieve_datapoints(&last_hour("pump_intake_pressure_bar")).await?;
+let points = &page.get_items()[0];
+let low = points.datapoints.iter()
+ .any(|p| p.value.unwrap_or(99.0) < 80.0);
+if low {
+ let mut event = Event::new(
+ format!("esp_anomaly_a12_{}", Utc::now().timestamp()),
+ "esp_anomaly".into(),
+ Utc::now());
+ event.status = Some("open".into());
+ event.add_metadata("well".into(), "well_a12".into());
+ event.add_metadata("pump".into(), "pump_esp_a12".into());
+ api.events.create(&vec![event]).await?;
+}
+```
+
+
+
+
+## 4. Are two alarms one fault? Ask the graph
+
+On a busy platform, alarms rarely arrive alone. When a wellhead temperature alarm and
+a separate pump alarm fire together, the control room needs to know fast: two
+independent problems, or one upstream cause? The graph answers it. Walk out from each
+alarmed sensor and look for a subsystem they share — if both are `PART_OF` the
+platform `cooling_system`, a single cooling failure is driving both.
+
+
+
+
+```java
+ResourceNetwork na = client.resources().fetchRelated("wellhead_temp_a12", 5);
+ResourceNetwork nb = client.resources().fetchRelated("pump_temp_a12", 5);
+
+Set aNodes = na.nodes().stream()
+ .map(NodeModel::getExternalId).collect(Collectors.toSet());
+Set shared = nb.nodes().stream()
+ .map(NodeModel::getExternalId).filter(aNodes::contains).collect(Collectors.toSet());
+
+// shared contains "cooling_system" → one root cause, not two
+```
+
+
+
+
+```python
+na = client.resources.fetch_related(external_id="wellhead_temp_a12", depth=5)
+nb = client.resources.fetch_related(external_id="pump_temp_a12", depth=5)
+
+shared = {n.external_id for n in na.nodes} & {n.external_id for n in nb.nodes}
+# 'cooling_system' in shared → one root cause, not two
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+use std::collections::HashSet;
+
+let na = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("wellhead_temp_a12").with_depth(5)).await?;
+let nb = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("pump_temp_a12").with_depth(5)).await?;
+
+let a: HashSet<&str> = na.nodes().iter().map(|n| n.external_id()).collect();
+let shared: Vec<&str> = nb.nodes().iter()
+ .map(|n| n.external_id()).filter(|id| a.contains(id)).collect();
+// shared contains "cooling_system" → one root cause, not two
+```
+
+
+
+
+That single query turns two alarms into one incident — see
+[Correlate alarms with the graph](/guides/correlate-alarms) for the general pattern.
+
+## See the result
+
+Chart the intake pressure the alarm watches — it sags below the 80-bar line in the last
+half hour, which is exactly what trips the `esp_anomaly` event:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="pump_intake_pressure_bar",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=2), end=pd.Timestamp.now(tz="UTC"))
+s = pd.Series({pd.to_datetime(p.timestamp): float(p.value)
+ for p in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()}).sort_index()
+s.plot(title="Pump intake pressure (bar)"); plt.axhline(80, color="r", ls="--"); plt.show()
+# the line drops from ~95 to ~75 bar — below the threshold, so the alarm fires
+```
+
+## See also
+
+- [Model assets as a graph](/guides/model-assets-graph) — the field/well/pump modeling pattern.
+- [High-throughput ingestion](/guides/ingest-timeseries) — sustaining field-scale write volume.
+- [Turn readings into events](/guides/detect-events) — the detection rule in detail.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — find the shared subsystem behind two alarms.
+- [Consume live data](/guides/realtime-subscriptions) — drive the control-room view from a live stream.
diff --git a/versioned_docs/version-1.0/industries/oil-and-gas/refining.mdx b/versioned_docs/version-1.0/industries/oil-and-gas/refining.mdx
new file mode 100644
index 0000000..83a6390
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/oil-and-gas/refining.mdx
@@ -0,0 +1,170 @@
+---
+sidebar_position: 4
+title: Oil & gas — refinery operations
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Oil & gas — refinery operations
+
+**The problem.** A refinery turns crude into products through tightly-coupled units —
+distillation columns, crackers, reformers — each running a delicate balance of
+temperature, pressure and flow. Drift off that balance and the unit makes off-spec
+product (reprocessing cost), burns excess energy (the refinery's second-biggest
+expense after feed), or trips entirely. Operators need the **process KPIs that reveal
+drift** and an alert when a unit heads off-spec, early enough to nudge it back.
+
+What we solve here is keeping units on-spec and energy-efficient by turning raw process
+tags into the few numbers that actually run the plant.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the crude unit's energy series with a
+day of hourly readings, so the roll-up in step 1 has data. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="cdu_1_energy_gj", name="CDU-1 energy", unit="gj", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=24, freq="1h")
+client.timeseries.insert_from_lists(timestamps=idx, values=np.random.uniform(40, 60, 24), ts="cdu_1_energy_gj")
+```
+
+## 1. Track the process KPIs
+
+Column temperatures, reflux and feed are series; the numbers operators steer by —
+energy per barrel, reflux ratio, separation quality — come from rolling these up.
+See [Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("cdu_1_energy_gj");
+filter.setStart(ZonedDateTime.now().minusDays(1));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("sum"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+// divide hourly energy by hourly throughput for energy-per-barrel
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> recordEnergyIntensity(p.getTimestamp(), p.getValue()));
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="cdu_1_energy_gj",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ record_energy_intensity(dp.timestamp, dp.sum)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("cdu_1_energy_gj".into()),
+ start: Some(Utc::now() - chrono::Duration::days(1)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["sum".into()]),
+ granularity: Some("1h".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { record_energy_intensity(&p.timestamp, p.sum); }
+```
+
+
+
+
+## 2. Catch a process upset early
+
+When a column's top temperature drifts out of its control band, raise a `process_upset`
+event so the board operator corrects before product goes off-spec or the unit trips.
+See [Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+EventModel upset = new EventModel();
+upset.setExternalId("process_upset_cdu1_" + System.currentTimeMillis());
+upset.setType("process_upset");
+upset.setStatus("open");
+upset.setMetadata(Map.of("unit", "crude_unit_1", "tag", "column_top_temp", "value_c", "168"));
+upset.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(upset));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"process_upset_cdu1_{int(pd.Timestamp.now().timestamp())}",
+ type="process_upset", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"unit": "crude_unit_1", "tag": "column_top_temp", "value_c": "168"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut upset = Event::new(
+ format!("process_upset_cdu1_{}", Utc::now().timestamp()),
+ "process_upset".into(),
+ Utc::now());
+upset.status = Some("open".into());
+upset.add_metadata("unit".into(), "crude_unit_1".into());
+upset.add_metadata("tag".into(), "column_top_temp".into());
+upset.add_metadata("value_c".into(), "168".into());
+api.events.create(&vec![upset]).await?;
+```
+
+
+
+
+## See the result
+
+Chart the hourly energy you rolled up — the bar chart is the shape the board operator
+watches for drift:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="cdu_1_energy_gj",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=1), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1h")
+vals = [dp.sum for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.bar(range(len(vals)), vals); plt.title("CDU-1 hourly energy (GJ)"); plt.xlabel("hour"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — energy-intensity and yield KPIs.
+- [Turn readings into events](/guides/detect-events) — the upset rule.
+- [Manufacturing — OEE](/industries/manufacturing-process/discrete) — the same roll-up pattern for discrete plants.
diff --git a/versioned_docs/version-1.0/industries/oil-and-gas/storage.mdx b/versioned_docs/version-1.0/industries/oil-and-gas/storage.mdx
new file mode 100644
index 0000000..cb80727
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/oil-and-gas/storage.mdx
@@ -0,0 +1,139 @@
+---
+sidebar_position: 5
+title: Oil & gas — tank storage & custody
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Oil & gas — tank storage & custody transfer
+
+**The problem.** A tank terminal holds millions of barrels whose value turns on one
+thing: that the books match the tanks. Every custody transfer in or out is metered, and
+the measured volume in the tank should move by exactly that amount. When it doesn't —
+when there's an unexplained loss — it's an evaporation issue, a measurement error, a
+leak, or theft, and each needs investigating fast. The challenge is **reconciling
+metered movements against measured inventory** continuously, not at month-end.
+
+What we solve here is keeping book and physical inventory in agreement, and flagging the
+gap the moment it opens.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the tank's volume and metered-movement
+series with a built-in discrepancy — the measured drop is larger than the metered
+deliveries, so the reconciliation flags an unexplained loss. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=12, freq="1h")
+# measured volume falls 3000 bbl over the shift
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="tank_t_12_volume_bbl", name="Tank 12 volume", unit="bbl", value_type="float")])
+client.timeseries.insert_from_lists(timestamps=idx, values=np.linspace(50000, 47000, 12), ts="tank_t_12_volume_bbl")
+# but only 2500 bbl was metered out → 500 bbl unexplained
+for s, total in [("tank_t_12_receipts_bbl", 0.0), ("tank_t_12_deliveries_bbl", 2500.0)]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=s, name=s, unit="bbl", value_type="float")])
+ client.timeseries.insert_from_lists(timestamps=idx, values=np.full(12, total / 12), ts=s)
+```
+
+## 1. Reconcile movement against measurement
+
+Compare the tank's measured volume change over a period against the net of its metered
+transfers. A gap beyond tolerance is an unexplained loss.
+
+
+
+
+```java
+// measured volume now vs the start of the shift
+double opening = firstValue("tank_t_12_volume_bbl", shiftStart);
+double closing = latest("tank_t_12_volume_bbl");
+double measuredChange = closing - opening;
+
+// net metered movement over the same period (receipts − deliveries)
+double metered = sum("tank_t_12_receipts_bbl", shiftStart)
+ - sum("tank_t_12_deliveries_bbl", shiftStart);
+
+double unexplained = measuredChange - metered;
+if (Math.abs(unexplained) > tolerance(closing)) {
+ EventModel loss = new EventModel();
+ loss.setExternalId("inventory_discrepancy_t12_" + System.currentTimeMillis());
+ loss.setType("inventory_discrepancy");
+ loss.setStatus("open");
+ loss.setMetadata(Map.of("tank", "tank_t_12", "unexplained_bbl",
+ String.format("%.1f", unexplained)));
+ loss.setEventTime(shiftStart);
+ client.events().create(List.of(loss));
+}
+```
+
+
+
+
+```python
+opening = first_value("tank_t_12_volume_bbl", shift_start)
+closing = latest("tank_t_12_volume_bbl")
+measured_change = closing - opening
+
+metered = (sum_over("tank_t_12_receipts_bbl", shift_start)
+ - sum_over("tank_t_12_deliveries_bbl", shift_start))
+
+unexplained = measured_change - metered
+if abs(unexplained) > tolerance(closing):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"inventory_discrepancy_t12_{int(pd.Timestamp.now().timestamp())}",
+ type="inventory_discrepancy", status="open",
+ event_time=shift_start,
+ metadata={"tank": "tank_t_12", "unexplained_bbl": f"{unexplained:.1f}"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let measured_change: f64 = latest("tank_t_12_volume_bbl").await?
+ - first_value("tank_t_12_volume_bbl", shift_start).await?;
+let metered: f64 = sum_over("tank_t_12_receipts_bbl", shift_start).await?
+ - sum_over("tank_t_12_deliveries_bbl", shift_start).await?;
+
+let unexplained = measured_change - metered;
+if unexplained.abs() > tolerance(latest("tank_t_12_volume_bbl").await?) {
+ let mut loss = Event::new(
+ format!("inventory_discrepancy_t12_{}", Utc::now().timestamp()),
+ "inventory_discrepancy".into(),
+ shift_start);
+ loss.status = Some("open".into());
+ loss.add_metadata("tank".into(), "tank_t_12".into());
+ loss.add_metadata("unexplained_bbl".into(), format!("{:.1}", unexplained));
+ api.events.create(&vec![loss]).await?;
+}
+```
+
+
+
+
+## 2. Watch tank conditions
+
+Level, temperature and pressure are series per tank; trends catch a tank breathing
+abnormally or a level moving with no scheduled transfer — an early sign of a leak.
+See [Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The reconciliation compares the 3000 bbl the tank actually lost against the 2500 bbl that
+was metered out — and flags the gap:
+
+```text
+unexplained_bbl ≈ 500.0 → an inventory_discrepancy event is raised for tank_t_12
+```
+
+## See also
+
+- [Turn readings into events](/guides/detect-events) — the reconciliation rule.
+- [Finance — exact values](/industries/financial-services/portfolio-risk) — the same need for exact, reconciled numbers.
+- [Query & aggregate](/guides/query-and-aggregate) — tank-condition trends.
diff --git a/versioned_docs/version-1.0/industries/overview.mdx b/versioned_docs/version-1.0/industries/overview.mdx
new file mode 100644
index 0000000..d71cf77
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/overview.mdx
@@ -0,0 +1,127 @@
+---
+title: Browse scenarios by sector
+---
+
+# Industry scenarios
+
+Worked examples, each starting from a real problem an industry needs solved and showing
+how to solve it with the SDK. They're organised by **sector** — pick yours and work
+through its scenarios, which build on each other across the value chain.
+
+:::tip How long each takes
+Every scenario is self-contained: a **Set up demo data** block (run once) → the
+solution → **See the result**. Plan on **~5 minutes to read** one, or **~10–15 minutes
+to run it end to end** from an empty workspace (a couple of minutes more the very first
+time, to install the SDK and set `BASE_URL`/`TOKEN`). When you're ready to go past CRUD
+into prediction, the **[Advanced scenarios (AI)](/advanced/machine-learning-gently)** — no
+data-science background needed — run ~20 minutes to 2 hours each.
+:::
+
+## Oil & gas
+
+Upstream to downstream — the full value chain.
+
+- [Drilling operations](/industries/oil-and-gas/drilling) — catch a well-control kick in its first seconds
+- [Production monitoring](/industries/oil-and-gas/production) — ESP anomaly detection on a field graph
+- [Pipeline integrity](/industries/oil-and-gas/pipeline) — detect and locate a leak on the network
+- [Refinery operations](/industries/oil-and-gas/refining) — keep units on-spec and energy-efficient
+- [Tank storage & custody](/industries/oil-and-gas/storage) — reconcile inventory, catch unexplained loss
+- [Emissions & flaring](/industries/oil-and-gas/emissions) — methane & flaring compliance reporting
+
+## Energy & utilities
+
+- [Grid telemetry](/industries/energy-utilities/grid) — SCADA roll-ups and overload events
+- [Wind generation](/industries/energy-utilities/wind) — capacity factor for a wind farm
+- [EV charging network](/industries/energy-utilities/ev-charging) — uptime and grid load
+- [Water distribution](/industries/energy-utilities/water) — trace a contamination source upstream
+- [Waste collection](/industries/energy-utilities/waste) — collect on fill level, not a calendar
+- [Air quality](/industries/energy-utilities/air-quality) — heterogeneous sensors, correct units
+
+## Manufacturing & process
+
+- [Discrete manufacturing](/industries/manufacturing-process/discrete) — OEE from station telemetry
+- [Semiconductor](/industries/manufacturing-process/semiconductor) — trace a yield excursion to one tool
+- [Pharma](/industries/manufacturing-process/pharma) — exact batch records and upstream genealogy
+
+## Transport & logistics
+
+- [Rail](/industries/transport-logistics/rail) — is the line still connected after a closure?
+- [Maritime](/industries/transport-logistics/maritime) — a reefer cold-chain alarm mid-ocean
+- [Airports](/industries/transport-logistics/airports) — catch a turnaround slipping before the slot is lost
+- [Ports](/industries/transport-logistics/ports) — crane productivity and yard congestion
+- [Last-mile delivery](/industries/transport-logistics/last-mile) — hit delivery windows
+- [Retail replenishment](/industries/transport-logistics/retail) — per-region demand, segmented by dataset
+- [Aerospace fleet](/industries/transport-logistics/aerospace) — a suspect part's blast radius across the fleet
+- [Supply chain](/industries/transport-logistics/supply-chain) — trace a recall downstream from a bad lot
+
+## Financial services
+
+- [Portfolio & risk](/industries/financial-services/portfolio-risk) — exact-decimal metrics and risk-limit events
+- [AML & payments](/industries/financial-services/aml) — expand a flagged payment into its ring
+- [Insurance fraud](/industries/financial-services/insurance-fraud) — expand a claim into its fraud ring
+
+## Healthcare
+
+- [Hospital operations](/industries/healthcare/hospital-operations) — a live capacity wall-board before a ward overflows
+- [Medical-device fleet](/industries/healthcare/medical-devices) — device availability and fast recall tracing
+- [Patient flow](/industries/healthcare/patient-flow) — find the bottleneck stage in the patient journey
+- [Pharmacy cold chain](/industries/healthcare/cold-chain) — protect temperature-sensitive stock
+
+## Mining & metals
+
+- [Mining operations](/industries/mining-metals/operations) — equipment uptime and true throughput against plan
+- [Processing plant](/industries/mining-metals/processing-plant) — protect metal recovery at the mill
+- [Tailings dam safety](/industries/mining-metals/tailings-safety) — trigger-based monitoring of a high-consequence dam
+
+## Agriculture & food
+
+- [Precision farming](/industries/agriculture-food/precision-farming) — drone imagery alongside soil sensors
+- [Salmon farming](/industries/agriculture-food/salmon-farming) — catch an oxygen crash in time to aerate
+
+## Technology & operations
+
+- [Observability (IT ops)](/industries/technology-operations/observability) — alert on an SLO breach the instant it happens
+- [Network performance](/industries/technology-operations/network) — localize degradation to shared backhaul
+- [Data centers](/industries/technology-operations/data-centers) — catch a thermal problem before racks throttle
+
+## Built environment
+
+- [Construction](/industries/built-environment/construction) — drawings and permits tied to the site
+- [Smart buildings](/industries/built-environment/smart-buildings) — flag a stuffy zone before the complaints
+
+## Going deeper
+
+The [Advanced scenarios (AI)](/advanced/machine-learning-gently) take these further into real
+algorithms — forecasting, classification, clustering and anomaly detection. **No
+data-science background needed:** start with
+[Machine learning, gently](/advanced/machine-learning-gently) for the ideas in plain
+language, then [Generate sample data](/advanced/generate-sample-data) for a sandbox to
+run them against. To clean and feature-engineer data while keeping every step traceable,
+see [Data pipeline & lineage](/advanced/data-cleaning-lineage). The models below are
+grouped by what they *do*.
+
+### Detect & monitor condition
+
+Learn what healthy looks like, then surface the drift.
+
+- [Predictive maintenance](/advanced/predictive-maintenance) — flag a machine drifting from its healthy vibration signature *(~1–2 hr)*
+- [Asset health scoring](/advanced/asset-health-score) — one composite health index from many signals *(~20–30 min)*
+- [LSTM anomaly detection](/advanced/lstm-anomaly-detection) — catch a multivariate event from the joint pattern *(~2 hr)*
+- [Process monitoring with PCA](/advanced/pca-process-monitoring) — spot a break in many sensors' normal relationship *(~1–1.5 hr)*
+
+### Forecast & predict ahead
+
+Predict what's coming while there's still time to act.
+
+- [Demand forecasting](/advanced/demand-forecasting) — forecast load or sales with gradient boosting *(~1 hr)*
+- [LSTM forecasting](/advanced/lstm-forecasting) — forecast a decline curve with a neural network *(~2 hr)*
+- [Early warning](/advanced/oxygen-crash-early-warning) — predict a salmon-pen oxygen crash before it happens *(~1 hr)*
+- [Failure prediction with XGBoost](/advanced/xgboost-failure-prediction) — predict an ESP failure a week out, with feature importance *(~1–2 hr)*
+
+### Classify, cluster & infer
+
+Label, group or estimate from features.
+
+- [Fraud classification](/advanced/fraud-classification) — graph features + a model to rank fraud/AML alerts *(~1–2 hr)*
+- [K-Means clustering](/advanced/kmeans-clustering) — discover operating regimes, asset cohorts and network communities *(~1–1.5 hr)*
+- [Soft sensor with Random Forest](/advanced/random-forest-soft-sensor) — infer a lab quality from cheap sensors *(~1 hr)*
diff --git a/versioned_docs/version-1.0/industries/technology-operations/_category_.json b/versioned_docs/version-1.0/industries/technology-operations/_category_.json
new file mode 100644
index 0000000..c79453c
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/technology-operations/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Technology & operations",
+ "position": 9,
+ "link": { "type": "generated-index", "title": "Technology & operations" }
+}
diff --git a/versioned_docs/version-1.0/industries/technology-operations/data-centers.mdx b/versioned_docs/version-1.0/industries/technology-operations/data-centers.mdx
new file mode 100644
index 0000000..392338c
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/technology-operations/data-centers.mdx
@@ -0,0 +1,182 @@
+---
+sidebar_position: 3
+title: Data centers — power & cooling
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Data centers — power & cooling
+
+**The problem.** A data center's two biggest risks are heat and the power bill. A
+cooling unit that quietly fails lets a row of racks climb toward thermal shutdown;
+meanwhile cooling and power overhead — everything not spent on actual compute — is the
+largest controllable cost. Facilities teams need to **catch a thermal problem before
+racks throttle**, and to drive overhead down by seeing exactly where the watts go.
+
+What we solve here is preventing heat-related outages and cutting the energy that never
+reaches a server.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a rack's inlet-temperature series,
+**create the subscription before we listen** (with a hot spell), and the rack→cooling
+graph — so step 3 finds the shared unit. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="rack_r14_inlet_temp_c", name="Rack R14 inlet temp", unit="deg_c", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="hall_2_thermal", name="Hall 2 thermal", timeseries=["rack_r14_inlet_temp_c"])])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=60, freq="1min")
+t = np.full(60, 24.0); t[-10:] = 29.4 # above the 27 °C band
+client.timeseries.insert_from_lists(timestamps=idx, values=t, ts="rack_r14_inlet_temp_c")
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[lbl]) for x, lbl in
+ [("rack_r14", "Rack"), ("rack_r15", "Rack"), ("crac_unit_2", "Crac")]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("rack_r14", "crac_unit_2", "cooled_by"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("rack_r15", "crac_unit_2", "cooled_by")])
+```
+
+## 1. Catch a thermal problem early
+
+Rack inlet temperatures and cooling-unit status stream live. When inlet temperature
+climbs past the safe band, raise a `thermal_warning` so the team intervenes before
+hardware protects itself by shutting down. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("hall_2_thermal"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (inletAbove(msg.payload(), 27.0)) { // °C — above ASHRAE band
+ EventModel warning = new EventModel();
+ warning.setExternalId("thermal_warning_r14_" + System.currentTimeMillis());
+ warning.setType("thermal_warning");
+ warning.setStatus("open");
+ warning.setMetadata(Map.of("rack", "rack_r14", "inlet_c", "29.4"));
+ warning.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(warning));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["hall_2_thermal"]) as listener:
+ for msg in listener:
+ if inlet_above(msg.payload, 27.0): # °C — above ASHRAE band
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"thermal_warning_r14_{int(pd.Timestamp.now().timestamp())}",
+ type="thermal_warning", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"rack": "rack_r14", "inlet_c": "29.4"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["hall_2_thermal"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if inlet_above(&msg.payload, 27.0) { // °C — above ASHRAE band
+ let mut warning = Event::new(
+ format!("thermal_warning_r14_{}", Utc::now().timestamp()),
+ "thermal_warning".into(),
+ Utc::now());
+ warning.status = Some("open".into());
+ warning.add_metadata("rack".into(), "rack_r14".into());
+ warning.add_metadata("inlet_c".into(), "29.4".into());
+ api.events.create(&vec![warning]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Drive down overhead (PUE)
+
+PUE is total facility power over IT power. Roll both meters up to hourly sums, divide,
+and the trend shows whether a cooling tweak actually helped. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+## 3. When racks overheat together, find the cooling unit
+
+Several racks climbing at once usually means one shared cooling unit, not a coincidence.
+Model racks → CRAC units as a [graph](/guides/model-assets-graph) and walk from each hot
+rack to the unit they share — the exact
+[two-alarms-one-cooling-system](/guides/correlate-alarms) pattern.
+
+
+
+
+```java
+ResourceNetwork a = client.resources().fetchRelated("rack_r14", 3);
+ResourceNetwork b = client.resources().fetchRelated("rack_r15", 3);
+
+Set aNodes = a.nodes().stream().map(NodeModel::getExternalId).collect(Collectors.toSet());
+Set shared = b.nodes().stream().map(NodeModel::getExternalId)
+ .filter(aNodes::contains).collect(Collectors.toSet());
+// shared contains "crac_unit_2" → one failing unit, not two hot racks
+```
+
+
+
+
+```python
+a = client.resources.fetch_related(external_id="rack_r14", depth=3)
+b = client.resources.fetch_related(external_id="rack_r15", depth=3)
+shared = {n.external_id for n in a.nodes} & {n.external_id for n in b.nodes}
+# 'crac_unit_2' in shared → one failing unit
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+use std::collections::HashSet;
+
+let a = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("rack_r14").with_depth(3)).await?;
+let b = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("rack_r15").with_depth(3)).await?;
+let an: HashSet<&str> = a.nodes().iter().map(|n| n.external_id()).collect();
+let shared: Vec<&str> = b.nodes().iter()
+ .map(|n| n.external_id()).filter(|id| an.contains(id)).collect();
+// shared contains "crac_unit_2"
+```
+
+
+
+
+## See the result
+
+The hot rack trips the loop, and the two warming racks share one cooling unit:
+
+```text
+thermal_warning raised for rack_r14
+shared unit: crac_unit_2 ← one failing CRAC, not two hot racks
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the thermal-monitoring loop.
+- [Query & aggregate](/guides/query-and-aggregate) — the PUE trend.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — find the shared cooling unit.
diff --git a/versioned_docs/version-1.0/industries/technology-operations/network.mdx b/versioned_docs/version-1.0/industries/technology-operations/network.mdx
new file mode 100644
index 0000000..c9d923a
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/technology-operations/network.mdx
@@ -0,0 +1,173 @@
+---
+sidebar_position: 2
+title: Telecom — network performance
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Telecom — network performance & capacity
+
+**The problem.** Subscribers notice a degraded cell long before a threshold report
+does — dropped calls, stalling video, dead zones at rush hour. The network operations
+centre needs to **spot a degrading or congesting cell and localize the cause**, then
+see far enough ahead to add capacity before the busy hour turns into an outage. The
+catch: a cell's problem is often not in the cell at all, but in a backhaul link it
+shares with others.
+
+What we solve here is finding and localizing degradation fast, and planning capacity
+before congestion bites.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create two cells' utilisation series (with
+busy-hour peaks) and the topology where they share a backhaul link — so the localisation
+in step 2 points at the link. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+util = 50 + 40 * np.clip(np.sin(np.arange(168) / 24 * 2 * np.pi), 0, 1) # ~90% at busy hour
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=168, freq="1h")
+for c in ["cell_oslo_4412", "cell_oslo_4418"]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=f"{c}_prb_util", name=f"{c} PRB util", unit="pct", value_type="float")])
+ client.timeseries.insert_from_lists(timestamps=idx, values=util, ts=f"{c}_prb_util")
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[lbl]) for x, lbl in
+ [("cell_oslo_4412", "Cell"), ("cell_oslo_4418", "Cell"),
+ ("backhaul_link_88", "BackhaulLink"), ("controller_rnc_3", "Controller")]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("cell_oslo_4412", "backhaul_link_88", "backhauled_by"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("cell_oslo_4418", "backhaul_link_88", "backhauled_by"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("backhaul_link_88", "controller_rnc_3", "connects_to")])
+```
+
+## 1. Find the busy-hour congestion
+
+Per-cell throughput, drop rate and active users are series. Aggregating to hourly peaks
+reveals which cells run hot at the busy hour — the capacity-planning shortlist. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("cell_oslo_4412_prb_util");
+filter.setStart(ZonedDateTime.now().minusDays(7));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("max"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> flagIfSaturated(p.getTimestamp(), p.getValue())); // PRB utilisation %
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="cell_oslo_4412_prb_util",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=7),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["max"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ flag_if_saturated(dp.timestamp, dp.max)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("cell_oslo_4412_prb_util".into()),
+ aggregates: Some(vec!["max".into()]),
+ granularity: Some("1h".into()),
+ start: Some(Utc::now() - chrono::Duration::days(7)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { flag_if_saturated(&p.timestamp, p.max); }
+```
+
+
+
+
+## 2. Localize a fault to shared backhaul
+
+When several cells degrade together, the common cause is usually a backhaul link or
+controller they share. Model cells → backhaul → controller as a
+[graph](/guides/model-assets-graph) and walk from each degraded cell to the element they
+have in common — the [shared-dependency](/guides/correlate-alarms) pattern, pointing the
+field team at the one link to fix instead of a dozen cells to chase.
+
+
+
+
+```java
+ResourceNetwork c1 = client.resources().fetchRelated("cell_oslo_4412", 4);
+ResourceNetwork c2 = client.resources().fetchRelated("cell_oslo_4418", 4);
+
+Set n1 = c1.nodes().stream().map(NodeModel::getExternalId).collect(Collectors.toSet());
+Set shared = c2.nodes().stream().map(NodeModel::getExternalId)
+ .filter(n1::contains).collect(Collectors.toSet());
+// shared contains "backhaul_link_88" → fix the link, not the cells
+```
+
+
+
+
+```python
+c1 = client.resources.fetch_related(external_id="cell_oslo_4412", depth=4)
+c2 = client.resources.fetch_related(external_id="cell_oslo_4418", depth=4)
+shared = {n.external_id for n in c1.nodes} & {n.external_id for n in c2.nodes}
+# 'backhaul_link_88' in shared
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+use std::collections::HashSet;
+
+let c1 = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("cell_oslo_4412").with_depth(4)).await?;
+let c2 = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("cell_oslo_4418").with_depth(4)).await?;
+let n1: HashSet<&str> = c1.nodes().iter().map(|n| n.external_id()).collect();
+let shared: Vec<&str> = c2.nodes().iter()
+ .map(|n| n.external_id()).filter(|id| n1.contains(id)).collect();
+// shared contains "backhaul_link_88"
+```
+
+
+
+
+## See the result
+
+Busy-hour utilisation peaks near 90%, and the two degraded cells share one element:
+
+```text
+backhaul_link_88 ← fix the link, not the dozen cells behind it
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — busy-hour capacity planning.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — localize to shared backhaul.
+- [Turn readings into events](/guides/detect-events) — degradation alerts.
diff --git a/versioned_docs/version-1.0/industries/technology-operations/observability.mdx b/versioned_docs/version-1.0/industries/technology-operations/observability.mdx
new file mode 100644
index 0000000..c8203f7
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/technology-operations/observability.mdx
@@ -0,0 +1,296 @@
+---
+sidebar_position: 1
+title: IT operations — observability
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# IT operations — service observability
+
+**The problem.** A platform team runs dozens of services across a fleet of hosts.
+They need every service's golden signals — request latency, error rate, CPU, memory
+— in one place, a live alerting worker that reacts the instant an SLO is breached,
+and an event trail for post-incident review.
+
+This scenario centres on **live consumption**: a long-running worker that tails the
+metrics stream and reacts in real time, rather than polling on a timer.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the checkout latency series, **create the
+subscription before we listen** (with a breach), and build the dependency graph where two
+services share a database — so step 4's correlation finds the shared cause. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="service_checkout_request_latency_ms", name="Checkout p99 latency", unit="ms", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="checkout_slo", name="Checkout SLO", timeseries=["service_checkout_request_latency_ms"])])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=60, freq="1min")
+lat = np.full(60, 180.0); lat[-10:] = 360.0 # p99 breaches the 300 ms SLO
+client.timeseries.insert_from_lists(timestamps=idx, values=lat, ts="service_checkout_request_latency_ms")
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[lbl]) for x, lbl in
+ [("service_checkout", "Service"), ("service_payments", "Service"),
+ ("order_db", "Database"), ("host_web_01", "Host")]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("service_checkout", "order_db", "depends_on"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("service_payments", "order_db", "depends_on"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("service_checkout", "host_web_01", "runs_on")])
+```
+
+## 1. Model services and hosts
+
+A service *runs on* hosts; both are resources. Modeling the topology lets an alert
+on a host fan out to the services it affects.
+
+
+
+
+```java
+Resource service = new Resource();
+service.setExternalId("service_checkout");
+service.setName("Checkout service");
+service.setLabels(List.of("Service"));
+
+Resource host = new Resource();
+host.setExternalId("host_web_01");
+host.setName("web-01");
+host.setLabels(List.of("Host"));
+
+RelForm runsOn = new RelForm();
+runsOn.setName("runs_on");
+runsOn.setFromExternalId("service_checkout");
+runsOn.setToExternalId("host_web_01");
+
+client.resources().create(List.of(service, host), List.of(runsOn));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id="service_checkout", name="Checkout service", labels=["Service"]),
+ intellistream_datahub_sdk.Resource(external_id="host_web_01", name="web-01", labels=["Host"])],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("service_checkout", "host_web_01", "runs_on")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut service = Resource::new();
+service.external_id = "service_checkout".into();
+service.name = "Checkout service".into();
+service.labels = Some(vec!["Service".into()]);
+
+let mut host = Resource::new();
+host.external_id = "host_web_01".into();
+host.name = "web-01".into();
+host.labels = Some(vec!["Host".into()]);
+
+api.resources.create(
+ vec![service, host],
+ vec![RelForm::by_external_ids("service_checkout", "host_web_01", "runs_on")],
+).await?;
+```
+
+
+
+
+Metrics become series: `service_checkout_request_latency_ms`,
+`service_checkout_error_rate`, `host_web_01_cpu_utilization`.
+
+## 2. Alert in real time on an SLO breach
+
+Subscribe to the service's golden-signal series and run the SLO check inside the
+delivery loop — the worker reacts the moment a bad datapoint lands, and records an
+event the on-call dashboard is watching. See [Consume live data](/guides/realtime-subscriptions)
+for the listener lifecycle.
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("checkout_slo"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (breachesSlo(msg.payload())) { // e.g. p99 latency > 300 ms
+ EventModel event = new EventModel();
+ event.setExternalId("slo_breach_checkout_" + System.currentTimeMillis());
+ event.setType("slo_breach");
+ event.setStatus("firing");
+ event.setMetadata(Map.of("service", "service_checkout", "slo", "latency_p99"));
+ event.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(event));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["checkout_slo"]) as listener:
+ for msg in listener:
+ if breaches_slo(msg.payload): # e.g. p99 latency > 300 ms
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"slo_breach_checkout_{int(pd.Timestamp.now().timestamp())}",
+ type="slo_breach", status="firing",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"service": "service_checkout", "slo": "latency_p99"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["checkout_slo"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if breaches_slo(&msg.payload) { // e.g. p99 latency > 300 ms
+ let mut event = Event::new(
+ format!("slo_breach_checkout_{}", Utc::now().timestamp()),
+ "slo_breach".into(),
+ Utc::now());
+ event.status = Some("firing".into());
+ event.add_metadata("service".into(), "service_checkout".into());
+ event.add_metadata("slo".into(), "latency_p99".into());
+ api.events.create(&vec![event]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 3. Review the incident afterward
+
+Every breach is a durable event. Filter by type and time to reconstruct an incident
+timeline, or by the `service` metadata to see one service's history.
+
+
+
+
+```java
+EventRetreiver retriever = new EventRetreiver();
+retriever.getFilter().setType(List.of("slo_breach"));
+retriever.setLimit(200);
+DataWrapper breaches = client.events().filter(retriever);
+```
+
+
+
+
+```python
+breaches = client.events.filter(type="slo_breach", limit=200)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::{EventFilter, EventFilterForm};
+
+let filter = EventFilterForm::default()
+ .set_filter(EventFilter {
+ r#type: Some(vec!["slo_breach".into()]), ..Default::default() })
+ .set_limit(200)
+ .build();
+let breaches = api.events.filter(&filter).await?;
+```
+
+
+
+
+## 4. Which services share a failing dependency?
+
+When `service_checkout` and `service_payments` breach their SLOs at the same moment,
+the on-call question is whether they're independent incidents or both downstream of
+one failing component. Model dependencies as edges — a service `depends_on` the
+datastores and caches it calls — and the graph answers it: walk out from each
+breaching service and intersect to find the component they have in common.
+
+
+
+
+```java
+ResourceNetwork checkout = client.resources().fetchRelated("service_checkout", 4);
+ResourceNetwork payments = client.resources().fetchRelated("service_payments", 4);
+
+Set checkoutDeps = checkout.nodes().stream()
+ .map(NodeModel::getExternalId).collect(Collectors.toSet());
+Set shared = payments.nodes().stream()
+ .map(NodeModel::getExternalId).filter(checkoutDeps::contains).collect(Collectors.toSet());
+
+// shared contains "order_db" → one failing dependency, not two outages
+```
+
+
+
+
+```python
+checkout = client.resources.fetch_related(external_id="service_checkout", depth=4)
+payments = client.resources.fetch_related(external_id="service_payments", depth=4)
+
+shared = ({n.external_id for n in checkout.nodes}
+ & {n.external_id for n in payments.nodes})
+# 'order_db' in shared → one failing dependency, not two outages
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+use std::collections::HashSet;
+
+let checkout = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("service_checkout").with_depth(4)).await?;
+let payments = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("service_payments").with_depth(4)).await?;
+
+let deps: HashSet<&str> = checkout.nodes().iter().map(|n| n.external_id()).collect();
+let shared: Vec<&str> = payments.nodes().iter()
+ .map(|n| n.external_id()).filter(|id| deps.contains(id)).collect();
+// shared contains "order_db" → one failing dependency, not two outages
+```
+
+
+
+
+The reverse is just as useful for change management: `fetchRelated` from a component
+you're about to restart returns its **blast radius** — every service that depends on
+it. See [Correlate alarms with the graph](/guides/correlate-alarms) for the pattern.
+
+## See the result
+
+The breach trips the loop, and expanding the two affected services finds their common cause:
+
+```text
+slo_breach event raised for service_checkout
+shared dependency: order_db ← fix the database, not each service
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the subscription + ack lifecycle.
+- [Turn readings into events](/guides/detect-events) — detection rules, on a timer or live.
+- [Model assets as a graph](/guides/model-assets-graph) — service/host topology.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — find the shared dependency behind two alerts.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/_category_.json b/versioned_docs/version-1.0/industries/transport-logistics/_category_.json
new file mode 100644
index 0000000..c4cbd71
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/_category_.json
@@ -0,0 +1,5 @@
+{
+ "label": "Transport & logistics",
+ "position": 4,
+ "link": { "type": "generated-index", "title": "Transport & logistics" }
+}
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/aerospace.mdx b/versioned_docs/version-1.0/industries/transport-logistics/aerospace.mdx
new file mode 100644
index 0000000..b70bacf
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/aerospace.mdx
@@ -0,0 +1,193 @@
+---
+sidebar_position: 7
+title: Aerospace — fleet maintenance
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Aerospace — fleet predictive maintenance
+
+**The problem.** An airline operates a fleet of aircraft, each a deep hierarchy of
+systems and components, each component streaming sensor data every second of every
+flight. When one part starts trending toward failure, two questions follow
+immediately: is this part about to fail, and **which other aircraft carry the same
+part** and are therefore at the same risk? The second question is a graph question —
+and answering it fast is the difference between scheduled maintenance and a grounding.
+
+This scenario leans on the **knowledge graph**: model the fleet down to the component,
+and when one component looks suspect, walk the graph to its whole blast radius.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the part type and two aircraft whose pumps
+are of that type. Step 1 adds a third, so the blast-radius walk in step 2 returns the whole
+affected fleet.
+
+```python
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+nodes = [intellistream_datahub_sdk.Resource(external_id="part_type_hp_47", name="Hydraulic pump type HP-47", labels=["part_type_hp_47"])]
+edges = []
+for ac in ["ln_318", "ln_401"]:
+ nodes += [intellistream_datahub_sdk.Resource(external_id=f"aircraft_{ac}", name=f"Aircraft {ac}", labels=[f"aircraft_{ac}"]),
+ intellistream_datahub_sdk.Resource(external_id=f"hyd_pump_{ac}_1", name=f"Pump {ac}", labels=[f"hyd_pump_{ac}_1"])]
+ edges += [intellistream_datahub_sdk.RelForm.by_external_ids(f"aircraft_{ac}", f"hyd_pump_{ac}_1", "contains"),
+ intellistream_datahub_sdk.RelForm.by_external_ids(f"hyd_pump_{ac}_1", "part_type_hp_47", "is_part_type")]
+client.resources.create(nodes, edges)
+```
+
+## 1. Model the fleet to the component
+
+A fleet *contains* aircraft, an aircraft *contains* systems, a system *contains*
+components. Components of the same part type also link to a shared `part_type` node —
+that shared node is what makes fleet-wide correlation possible.
+
+
+
+
+```java
+Resource aircraft = new Resource();
+aircraft.setExternalId("aircraft_ln_312");
+aircraft.setName("Aircraft LN-312");
+aircraft.setLabels(List.of("aircraft_ln_312"));
+
+Resource pump = new Resource();
+pump.setExternalId("hyd_pump_ln_312_1");
+pump.setName("Hydraulic pump #1");
+pump.setLabels(List.of("hyd_pump_ln_312_1"));
+
+// the component is an instance of a shared part type
+RelForm installed = new RelForm();
+installed.setName("contains");
+installed.setFromExternalId("aircraft_ln_312");
+installed.setToExternalId("hyd_pump_ln_312_1");
+
+RelForm isType = new RelForm();
+isType.setName("is_part_type");
+isType.setFromExternalId("hyd_pump_ln_312_1");
+isType.setToExternalId("part_type_hp_47");
+
+client.resources().create(List.of(aircraft, pump), List.of(installed, isType));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id="aircraft_ln_312", name="Aircraft LN-312", labels=["aircraft_ln_312"]),
+ intellistream_datahub_sdk.Resource(external_id="hyd_pump_ln_312_1", name="Hydraulic pump #1", labels=["hyd_pump_ln_312_1"])],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("aircraft_ln_312", "hyd_pump_ln_312_1", "contains"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("hyd_pump_ln_312_1", "part_type_hp_47", "is_part_type")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut aircraft = Resource::new();
+aircraft.external_id = "aircraft_ln_312".into();
+aircraft.name = "Aircraft LN-312".into();
+aircraft.labels = Some(vec!["aircraft_ln_312".into()]);
+
+let mut pump = Resource::new();
+pump.external_id = "hyd_pump_ln_312_1".into();
+pump.name = "Hydraulic pump #1".into();
+pump.labels = Some(vec!["hyd_pump_ln_312_1".into()]);
+
+api.resources.create(
+ vec![aircraft, pump],
+ vec![
+ RelForm::by_external_ids("aircraft_ln_312", "hyd_pump_ln_312_1", "contains"),
+ RelForm::by_external_ids("hyd_pump_ln_312_1", "part_type_hp_47", "is_part_type"),
+ ],
+).await?;
+```
+
+
+
+
+Each component streams series — `hyd_pump_ln_312_1_vibration_mm_s`,
+`hyd_pump_ln_312_1_temp_c` — [ingested at fleet scale](/guides/ingest-timeseries).
+
+## 2. A part looks suspect — find the blast radius
+
+Rising vibration on one pump suggests a part-type defect. Walk the graph out from the
+shared `part_type_hp_47` node: every component `is_part_type` of it, and every
+aircraft that `contains` one, is the blast radius — the list maintenance needs to
+inspect.
+
+
+
+
+```java
+ResourceNetwork affected = client.resources().fetchRelated("part_type_hp_47", 3);
+
+// every aircraft in the returned sub-graph carries this part type
+affected.nodes().stream()
+ .filter(n -> n.getExternalId().startsWith("aircraft_"))
+ .forEach(a -> System.out.println("inspect: " + a.getExternalId()));
+```
+
+
+
+
+```python
+affected = client.resources.fetch_related(external_id="part_type_hp_47", depth=3)
+
+for node in affected.nodes:
+ if node.external_id.startswith("aircraft_"):
+ print("inspect:", node.external_id)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let affected = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("part_type_hp_47").with_depth(3)).await?;
+
+for node in affected.nodes() {
+ if node.external_id().starts_with("aircraft_") {
+ println!("inspect: {}", node.external_id());
+ }
+}
+```
+
+
+
+
+One traversal turns a single suspect reading into a precise, fleet-wide inspection
+list. See [Correlate alarms with the graph](/guides/correlate-alarms) for the pattern.
+
+## 3. Record the finding
+
+Raise a `component_anomaly` event per affected component so the maintenance system
+schedules the work and keeps an auditable trail.
+
+## See the result
+
+Walking out from the suspect part type returns every aircraft to inspect:
+
+```text
+inspect: aircraft_ln_312
+inspect: aircraft_ln_318
+inspect: aircraft_ln_401
+```
+
+## See also
+
+- [Model assets as a graph](/guides/model-assets-graph) — the fleet/aircraft/component hierarchy.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — blast-radius traversal in detail.
+- [High-throughput ingestion](/guides/ingest-timeseries) — per-flight sensor volume.
+- [Turn readings into events](/guides/detect-events) — the anomaly rule.
+- **[Predictive maintenance (advanced)](/advanced/predictive-maintenance)** — learn a component's healthy vibration signature and flag the drift.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/airports.mdx b/versioned_docs/version-1.0/industries/transport-logistics/airports.mdx
new file mode 100644
index 0000000..8cbd0ca
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/airports.mdx
@@ -0,0 +1,132 @@
+---
+sidebar_position: 3
+title: Airports — turnaround & on-time
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Airports — turnaround & on-time operations
+
+**The problem.** An aircraft on the ground earns nothing, and every minute past its
+slot ripples outward — a late turnaround misses the departure wave, the next rotation
+slips, and by evening the delay has spread across the network. A turnaround is a relay
+of independent teams — deboarding, cleaning, fuelling, catering, loading, boarding — and
+if any leg runs long, the whole thing does. Operations needs to **see a turnaround
+falling behind while it can still be recovered**, not after pushback is missed.
+
+What we solve here is delay propagation: catching the slow leg of a turnaround in time
+to throw resources at it.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the stand's turnaround feed, **create the
+subscription before we listen**, and push a leg-overrun reading. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="stand_b12_fuelling_progress", name="Stand B12 fuelling", unit="pct", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="stand_b12_turnaround", name="Stand B12 turnaround", timeseries=["stand_b12_fuelling_progress"])])
+client.timeseries.insert_from_lists(timestamps=[pd.Timestamp.now(tz="UTC")], values=[1.0], ts="stand_b12_fuelling_progress")
+```
+
+## 1. Track each turnaround milestone live
+
+Milestone timestamps and progress stream in per stand. When a leg overruns its planned
+duration, raise a `turnaround_risk` event so the duty manager reallocates before the
+slot is lost. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("stand_b12_turnaround"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (legOverrunning(msg.payload())) { // e.g. fuelling past plan
+ EventModel risk = new EventModel();
+ risk.setExternalId("turnaround_risk_su204_" + System.currentTimeMillis());
+ risk.setType("turnaround_risk");
+ risk.setStatus("open");
+ risk.setMetadata(Map.of("flight", "flight_su204", "leg", "fuelling", "stand", "stand_b12"));
+ risk.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(risk));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["stand_b12_turnaround"]) as listener:
+ for msg in listener:
+ if leg_overrunning(msg.payload): # e.g. fuelling past plan
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"turnaround_risk_su204_{int(pd.Timestamp.now().timestamp())}",
+ type="turnaround_risk", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"flight": "flight_su204", "leg": "fuelling", "stand": "stand_b12"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["stand_b12_turnaround"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if leg_overrunning(&msg.payload) { // e.g. fuelling past plan
+ let mut risk = Event::new(
+ format!("turnaround_risk_su204_{}", Utc::now().timestamp()),
+ "turnaround_risk".into(),
+ Utc::now());
+ risk.status = Some("open".into());
+ risk.add_metadata("flight".into(), "flight_su204".into());
+ risk.add_metadata("leg".into(), "fuelling".into());
+ risk.add_metadata("stand".into(), "stand_b12".into());
+ api.events.create(&vec![risk]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Trace the knock-on through shared aircraft
+
+One late arrival delays the *next* flight on the same airframe, and the one after that.
+Model flights and the aircraft and crews they share as a [graph](/guides/model-assets-graph);
+walking out from the delayed flight reveals the downstream rotations at risk — the
+[blast radius](/guides/correlate-alarms) of the delay — so re-timing decisions are made
+with the whole chain in view.
+
+## 3. Learn from the day
+
+On-time performance and per-leg durations are series; daily roll-ups by stand and
+ground-handler surface the legs that chronically run long. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The overrunning leg trips the loop:
+
+```text
+turnaround_risk_su204_… → open (fuelling running past plan, slot at risk)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — live milestone tracking.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — delay knock-on across rotations.
+- [Query & aggregate](/guides/query-and-aggregate) — on-time-performance reporting.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/last-mile.mdx b/versioned_docs/version-1.0/industries/transport-logistics/last-mile.mdx
new file mode 100644
index 0000000..b3df15a
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/last-mile.mdx
@@ -0,0 +1,123 @@
+---
+sidebar_position: 5
+title: Logistics — last-mile delivery
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Logistics — last-mile delivery
+
+**The problem.** The last mile is where delivery promises are kept or broken. A van
+running behind quietly turns a two-hour window into a missed slot; a failed delivery
+means a costly re-attempt and a customer who stops trusting the estimate. Dispatch
+needs to **see a route slipping while there's still time to re-sequence or reassign**,
+not when the angry message arrives.
+
+What we solve here is hitting delivery windows and cutting failed first attempts.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a van's ETA-slack feed, **create the
+subscription before we listen**, and push a behind-schedule reading. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="van_22_eta_slack_min", name="Van 22 ETA slack", unit="min", value_type="float")])
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="fleet_eta", name="Fleet ETA", timeseries=["van_22_eta_slack_min"])])
+client.timeseries.insert_from_lists(timestamps=[pd.Timestamp.now(tz="UTC")], values=[-12.0], ts="van_22_eta_slack_min")
+```
+
+## 1. Catch a slipping route live
+
+Each van streams progress against its plan — stops completed, running ETA. When the
+projected arrival for the next stop slips past its window, raise a `delivery_at_risk`
+event so dispatch can act. See [Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("fleet_eta"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (etaPastWindow(msg.payload())) {
+ EventModel risk = new EventModel();
+ risk.setExternalId("delivery_at_risk_v22_" + System.currentTimeMillis());
+ risk.setType("delivery_at_risk");
+ risk.setStatus("open");
+ risk.setMetadata(Map.of("van", "van_22", "route", "route_oslo_e", "stops_left", "9"));
+ risk.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(risk));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["fleet_eta"]) as listener:
+ for msg in listener:
+ if eta_past_window(msg.payload):
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"delivery_at_risk_v22_{int(pd.Timestamp.now().timestamp())}",
+ type="delivery_at_risk", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"van": "van_22", "route": "route_oslo_e", "stops_left": "9"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["fleet_eta"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if eta_past_window(&msg.payload) {
+ let mut risk = Event::new(
+ format!("delivery_at_risk_v22_{}", Utc::now().timestamp()),
+ "delivery_at_risk".into(),
+ Utc::now());
+ risk.status = Some("open".into());
+ risk.add_metadata("van".into(), "van_22".into());
+ risk.add_metadata("route".into(), "route_oslo_e".into());
+ risk.add_metadata("stops_left".into(), "9".into());
+ api.events.create(&vec![risk]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 2. Improve tomorrow's routes
+
+On-time rate, stops-per-hour and failed-delivery counts are series per route and
+driver. Daily roll-ups show which routes chronically run long and where first-attempt
+failures cluster — the input to better planning. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The slipping route trips the loop:
+
+```text
+delivery_at_risk_v22_… → open (projected arrival past the delivery window)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — live route tracking.
+- [Turn readings into events](/guides/detect-events) — the at-risk rule.
+- [Query & aggregate](/guides/query-and-aggregate) — on-time and failure reporting.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/maritime.mdx b/versioned_docs/version-1.0/industries/transport-logistics/maritime.mdx
new file mode 100644
index 0000000..8f677d9
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/maritime.mdx
@@ -0,0 +1,171 @@
+---
+sidebar_position: 2
+title: Maritime — fleet & cold chain
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Maritime — fleet tracking & reefer cold chain
+
+**The problem.** A shipping line moves refrigerated ("reefer") containers across
+oceans. A reefer that drifts out of its temperature band for too long can spoil an
+entire container of cargo — and on a ship mid-Atlantic, the only way to catch it is a
+live feed, not a daily report. The fleet desk needs each vessel's reefers streaming in
+real time and an alarm the instant one breaches its band.
+
+This scenario is about **live consumption at sea**: a long-running monitor that tails
+every reefer's temperature and reacts the moment it strays.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a reefer's supply/set-point series and feed
+in air that drifts warm out of band. Step 1 below creates the subscription (before the
+listen loop) over these series. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+for s in ["reefer_msc_1182_supply_c", "reefer_msc_1182_setpoint_c"]:
+ client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id=s, name=s, unit="deg_c", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=120, freq="1min")
+client.timeseries.insert_from_lists(timestamps=idx, values=np.full(120, -18.0), ts="reefer_msc_1182_setpoint_c")
+supply = np.full(120, -18.0); supply[-30:] += np.linspace(0, 6, 30) # drifting warm
+client.timeseries.insert_from_lists(timestamps=idx, values=supply, ts="reefer_msc_1182_supply_c")
+```
+
+## 1. Subscribe to the reefer fleet
+
+Each reefer reports its set-point and supply-air temperature as series; a subscription
+fans them all into one live feed.
+
+
+
+
+```java
+Subscription sub = new Subscription();
+sub.setExternalId("reefer_fleet");
+sub.setName("Reefer fleet");
+sub.setTimeseries(List.of(
+ IdCollection.createFromExternalId("reefer_msc_1182_supply_c"),
+ IdCollection.createFromExternalId("reefer_msc_1182_setpoint_c")));
+client.subscriptions().create(List.of(sub));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+client.subscriptions.create([intellistream_datahub_sdk.Subscription(
+ external_id="reefer_fleet", name="Reefer fleet",
+ timeseries=["reefer_msc_1182_supply_c", "reefer_msc_1182_setpoint_c"])])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::subscriptions::Subscription;
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let sub = Subscription::new("reefer_fleet".into(), "Reefer fleet".into(),
+ vec![IdAndExtId::from_external_id("reefer_msc_1182_supply_c"),
+ IdAndExtId::from_external_id("reefer_msc_1182_setpoint_c")]);
+api.subscriptions.create(&sub).await?;
+```
+
+
+
+
+## 2. Watch for a temperature excursion live
+
+Drive the listener in a loop; when a reefer's supply air strays from its set-point
+beyond tolerance, raise a `reefer_excursion` event so the fleet desk — and the ship —
+are alerted while there's still time to act. See
+[Consume live data](/guides/realtime-subscriptions).
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("reefer_fleet"))
+ .stream((SubscriptionMessage msg) -> { // auto-acks after each message
+ if (outOfBand(msg.payload())) { // |supply - setpoint| > tolerance
+ EventModel excursion = new EventModel();
+ excursion.setExternalId("reefer_excursion_1182_" + System.currentTimeMillis());
+ excursion.setType("reefer_excursion");
+ excursion.setStatus("open");
+ excursion.setMetadata(Map.of("reefer", "reefer_msc_1182", "vessel", "vessel_nordic_star"));
+ excursion.setEventTime(ZonedDateTime.now());
+ client.events().create(List.of(excursion));
+ }
+ })) {
+ awaitShutdown(); // your app lifecycle; closing the stream ends delivery
+}
+```
+
+
+
+
+```python
+with client.subscriptions.listen(["reefer_fleet"]) as listener:
+ for msg in listener:
+ if out_of_band(msg.payload): # |supply - setpoint| > tolerance
+ client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"reefer_excursion_1182_{int(pd.Timestamp.now().timestamp())}",
+ type="reefer_excursion", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"reefer": "reefer_msc_1182", "vessel": "vessel_nordic_star"})])
+ listener.ack([msg.message_id])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut listener = api.subscriptions.listen(&["reefer_fleet"]).await?;
+while let Some(Ok(msg)) = listener.next().await {
+ if out_of_band(&msg.payload) { // |supply - setpoint| > tolerance
+ let mut excursion = Event::new(
+ format!("reefer_excursion_1182_{}", Utc::now().timestamp()),
+ "reefer_excursion".into(),
+ Utc::now());
+ excursion.status = Some("open".into());
+ excursion.add_metadata("reefer".into(), "reefer_msc_1182".into());
+ excursion.add_metadata("vessel".into(), "vessel_nordic_star".into());
+ api.events.create(&vec![excursion]).await?;
+ }
+ listener.ack(&[msg.message_id.as_str()]).await?;
+}
+```
+
+
+
+
+## 3. Prove the cold chain held
+
+For each delivered container, retrieve the voyage's temperature series to produce the
+compliance record the receiver requires — a complete, time-stamped trace. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+## See the result
+
+The warming reefer trips the loop:
+
+```text
+reefer_excursion_1182_… → open (supply air strayed from set-point beyond tolerance)
+```
+
+## See also
+
+- [Consume live data](/guides/realtime-subscriptions) — the reefer-monitoring loop.
+- [Turn readings into events](/guides/detect-events) — the excursion rule.
+- [Query & aggregate](/guides/query-and-aggregate) — the voyage compliance record.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/ports.mdx b/versioned_docs/version-1.0/industries/transport-logistics/ports.mdx
new file mode 100644
index 0000000..dcfe99b
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/ports.mdx
@@ -0,0 +1,168 @@
+---
+sidebar_position: 4
+title: Ports — terminal throughput
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Ports — container terminal throughput
+
+**The problem.** A container terminal is judged on two numbers: how fast cranes work a
+ship, and how long boxes sit in the yard. A crane running below its move-rate keeps a
+vessel — and its berthing fees — alongside longer than planned; a container that dwells
+too long clogs stacks and blocks the next move. Terminal operations needs to **see a
+crane falling behind and a yard filling up in time to rebalance**, not at the end-of-
+shift report.
+
+What we solve here is berth productivity and yard congestion — the throughput the whole
+terminal is measured on.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create a crane's move series with six hours of
+readings. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="crane_qc_07_moves", name="Crane QC-07 moves", unit="count", value_type="float")])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=6 * 60, freq="1min")
+client.timeseries.insert_from_lists(timestamps=idx, values=np.random.poisson(0.45, len(idx)).astype(float), ts="crane_qc_07_moves")
+```
+
+## 1. Watch crane productivity by the hour
+
+Crane moves are series per crane. Roll them up to moves-per-hour and compare against the
+target to find an under-performing crane mid-vessel. See
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("crane_qc_07_moves");
+filter.setStart(ZonedDateTime.now().minusHours(6));
+filter.setEnd(ZonedDateTime.now());
+filter.setAggregates(List.of("sum"));
+filter.setGranularity("1h");
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems().get(0).getDatapoints()
+ .forEach(p -> compareToTarget(p.getTimestamp(), p.getValue())); // moves/hour
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="crane_qc_07_moves",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=6),
+ end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1h")
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ compare_to_target(dp.timestamp, dp.sum)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+use chrono::Utc;
+
+let filter = RetrieveFilter {
+ external_id: Some("crane_qc_07_moves".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(6)),
+ end: Some(Utc::now()),
+ aggregates: Some(vec!["sum".into()]),
+ granularity: Some("1h".into()),
+ ..Default::default()
+};
+let page = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+let series = &page.get_items()[0];
+for p in &series.datapoints { compare_to_target(&p.timestamp, p.sum); }
+```
+
+
+
+
+## 2. Flag yard congestion before it blocks moves
+
+Stack occupancy is a series per yard block. When a block crosses its safe fill, raise a
+`yard_congestion` event so planners divert incoming boxes to another block. See
+[Turn readings into events](/guides/detect-events).
+
+
+
+
+```java
+EventModel congestion = new EventModel();
+congestion.setExternalId("yard_congestion_b07_" + System.currentTimeMillis());
+congestion.setType("yard_congestion");
+congestion.setStatus("open");
+congestion.setMetadata(Map.of("block", "yard_block_b07", "occupancy_pct", "94"));
+congestion.setEventTime(ZonedDateTime.now());
+client.events().create(List.of(congestion));
+```
+
+
+
+
+```python
+client.events.create([intellistream_datahub_sdk.Event(
+ external_id=f"yard_congestion_b07_{int(pd.Timestamp.now().timestamp())}",
+ type="yard_congestion", status="open",
+ event_time=pd.Timestamp.now(tz="UTC"),
+ metadata={"block": "yard_block_b07", "occupancy_pct": "94"})])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::Event;
+use chrono::Utc;
+
+let mut congestion = Event::new(
+ format!("yard_congestion_b07_{}", Utc::now().timestamp()),
+ "yard_congestion".into(),
+ Utc::now());
+congestion.status = Some("open".into());
+congestion.add_metadata("block".into(), "yard_block_b07".into());
+congestion.add_metadata("occupancy_pct".into(), "94".into());
+api.events.create(&vec![congestion]).await?;
+```
+
+
+
+
+## See the result
+
+Chart moves-per-hour against the target the terminal works to:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="crane_qc_07_moves",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=6), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1h")
+moves = [dp.sum for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.bar(range(len(moves)), moves); plt.axhline(25, color="r", ls="--")
+plt.title("Crane QC-07 moves per hour"); plt.show()
+```
+
+## See also
+
+- [Query & aggregate](/guides/query-and-aggregate) — crane moves-per-hour.
+- [Turn readings into events](/guides/detect-events) — the congestion rule.
+- [High-throughput ingestion](/guides/ingest-timeseries) — equipment telemetry volume.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/rail.mdx b/versioned_docs/version-1.0/industries/transport-logistics/rail.mdx
new file mode 100644
index 0000000..fbb9cde
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/rail.mdx
@@ -0,0 +1,161 @@
+---
+sidebar_position: 1
+title: Rail — network operations
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Rail — network operations & connectivity
+
+**The problem.** A rail network is a web of stations joined by track segments. When a
+segment is closed — a fault, flooding, maintenance — the controller's first question
+is sharp and time-critical: **is the line between two points still connected, or is it
+severed?** If severed, trains must reroute or terminate now. That's a reachability
+question, and on a graph it's a single traversal.
+
+This scenario introduces a graph pattern the others don't: **connectivity** — walk the
+network to see what's still reachable after a link goes down.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the rail network (stations joined by
+`connects_to` track) so the reachability check has a graph to walk. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[x]) for x in
+ ["station_oslo_s", "junction_lierstranda", "station_drammen", "station_bergen_n"]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("station_oslo_s", "junction_lierstranda", "connects_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("junction_lierstranda", "station_drammen", "connects_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("station_drammen", "station_bergen_n", "connects_to")])
+```
+
+## 1. Model the network
+
+Stations are resources; track segments are `connects_to` edges between them.
+
+
+
+
+```java
+List track = List.of(
+ rel("connects_to", "station_oslo_s", "junction_lierstranda"),
+ rel("connects_to", "junction_lierstranda", "station_drammen"),
+ rel("connects_to", "station_drammen", "station_bergen_n"));
+
+client.resources().create(stations, track);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+track = [
+ intellistream_datahub_sdk.RelForm.by_external_ids("station_oslo_s", "junction_lierstranda", "connects_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("junction_lierstranda", "station_drammen", "connects_to"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("station_drammen", "station_bergen_n", "connects_to"),
+]
+client.resources.create(stations, track)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::relations::RelForm;
+
+let track = vec![
+ RelForm::by_external_ids("station_oslo_s", "junction_lierstranda", "connects_to"),
+ RelForm::by_external_ids("junction_lierstranda", "station_drammen", "connects_to"),
+ RelForm::by_external_ids("station_drammen", "station_bergen_n", "connects_to"),
+];
+api.resources.create(stations, track).await?;
+```
+
+
+
+
+## 2. Is the line still connected?
+
+A segment is closed — remove it (or exclude it), then ask: starting from one terminal,
+is the other terminal still in the reachable set? Walk `connects_to` from the origin
+and check whether the destination is among the returned nodes.
+
+
+
+
+```java
+RelatedResourcesForm reach = new RelatedResourcesForm();
+reach.setExternalId("station_oslo_s");
+reach.setDepth(100);
+reach.setRelationshipTypes(List.of("connects_to"));
+
+ResourceNetwork reachable = client.resources().fetchRelated(reach);
+
+boolean stillConnected = reachable.nodes().stream()
+ .anyMatch(n -> n.getExternalId().equals("station_bergen_n"));
+
+System.out.println(stillConnected ? "route open" : "SEVERED — reroute required");
+```
+
+
+
+
+```python
+reachable = client.resources.fetch_related(
+ external_id="station_oslo_s", depth=100, relationship_types=["connects_to"])
+
+still_connected = any(n.external_id == "station_bergen_n" for n in reachable.nodes)
+print("route open" if still_connected else "SEVERED — reroute required")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let reachable = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("station_oslo_s")
+ .with_depth(100)
+ .with_relationship_types(vec!["connects_to".into()])).await?;
+
+let still_connected = reachable.nodes().iter()
+ .any(|n| n.external_id() == "station_bergen_n");
+println!("{}", if still_connected { "route open" } else { "SEVERED — reroute required" });
+```
+
+
+
+
+The returned node set *is* the connected component reachable from the origin. Membership
+answers connectivity directly — see [Correlate alarms with the graph](/guides/correlate-alarms)
+for the traversal mechanics.
+
+## 3. Track punctuality
+
+Dwell times and delays are series (`station_drammen_dwell_s`, `service_r10_delay_min`);
+roll them up for the punctuality report and raise a `delay_threshold` event when a
+service slips past its allowance.
+
+## See the result
+
+With the network intact, the destination is reachable:
+
+```text
+route open ← station_bergen_n is in the set reachable from station_oslo_s
+```
+
+Delete a `connects_to` segment and re-run the check and it prints `SEVERED — reroute required`.
+
+## See also
+
+- [Model assets as a graph](/guides/model-assets-graph) — modeling the network topology.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — reachability via traversal.
+- [Turn readings into events](/guides/detect-events) — punctuality alerts.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/retail.mdx b/versioned_docs/version-1.0/industries/transport-logistics/retail.mdx
new file mode 100644
index 0000000..20ae55b
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/retail.mdx
@@ -0,0 +1,156 @@
+---
+sidebar_position: 6
+title: Retail — demand & replenishment
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Retail — demand & replenishment
+
+**The problem.** A retail chain spans regions, each with stores, each tracking sales
+and stock per SKU. Regional teams must see *their* numbers and only their numbers,
+while head office rolls everything up. The data is one big stream; the requirement is
+clean separation by region — for access, for ownership, for reporting.
+
+This scenario centres on **datasets** as the unit of segmentation: give each region its
+own, tag everything with it, and the partitioning falls out for free.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the region's dataset and a store's SKU
+sales series with a month of daily units. Safe to re-run.
+
+```python
+import intellistream_datahub_sdk, numpy as np, pandas as pd
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+region_id = client.datasets.create([intellistream_datahub_sdk.Dataset(external_id="region_nordics", name="Nordics region")])[0].id
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(external_id="store_oslo_01_sku_4471_sales",
+ name="Oslo-01 SKU 4471 units sold", unit="units", value_type="float", data_set_id=region_id)])
+idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=30, freq="1d")
+client.timeseries.insert_from_lists(timestamps=idx, values=np.random.poisson(40, 30).astype(float),
+ ts="store_oslo_01_sku_4471_sales")
+```
+
+## 1. A dataset per region
+
+Create a dataset for each region; every series the region owns carries its
+`data_set_id`.
+
+
+
+
+```java
+DataSetModel region = new DataSetModel();
+region.setExternalId("region_nordics");
+region.setName("Nordics region");
+long regionId = client.datasets().create(List.of(region))
+ .getItems().get(0).getId();
+
+var salesSeries = new Timeseries();
+salesSeries.setExternalId("store_oslo_01_sku_4471_sales");
+salesSeries.setName("Oslo 01 — SKU 4471 units sold");
+salesSeries.setDataSetId(regionId);
+salesSeries.setUnit("units");
+client.timeseries().create(salesSeries);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+region_id = client.datasets.create([
+ intellistream_datahub_sdk.Dataset(external_id="region_nordics", name="Nordics region")])[0].id
+
+client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
+ external_id="store_oslo_01_sku_4471_sales",
+ name="Oslo 01 — SKU 4471 units sold", unit="units", value_type="float", data_set_id=region_id)])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::datasets::Dataset;
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+// Dataset::new derives external_id as snake_case → set it explicitly to match
+let mut ds = Dataset::new("Nordics region".into());
+ds.external_id = "region_nordics".into();
+let created = api.datasets.create(&vec![ds]).await?;
+let region_id = created.get_items()[0].id;
+
+let mut ts = TimeSeries::new("store_oslo_01_sku_4471_sales", "Oslo 01 — SKU 4471 units sold");
+ts.unit = Some("units".into());
+ts.data_set_id = region_id;
+api.time_series.create_one(&ts).await?;
+```
+
+
+
+
+## 2. Roll up demand per region
+
+Each region lists or filters by its own dataset, then aggregates daily sales to feed
+replenishment. See [Datasets reference](/reference/datasets) and
+[Query & aggregate](/guides/query-and-aggregate).
+
+
+
+
+```java
+import ai.intellistream.datahub.models.IdCollection;
+
+// scope to the region's dataset
+DataWrapper regional = client.datasets().byIds(
+ List.of(IdCollection.createFromExternalId("region_nordics")));
+```
+
+
+
+
+```python
+regional = client.datasets.by_ids(["region_nordics"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let regional = api.datasets.by_ids(&vec![IdAndExtId::from_external_id("region_nordics")]).await?;
+```
+
+
+
+
+## 3. Trigger replenishment
+
+When a store's on-hand for a SKU falls below its reorder point, raise a `reorder`
+[event](/guides/detect-events) carrying the store, SKU and suggested quantity — the
+replenishment system acts on it, scoped to the region's dataset.
+
+## See the result
+
+Chart the SKU's daily sales — the demand signal replenishment runs on:
+
+```python
+import matplotlib.pyplot as plt
+
+rf = intellistream_datahub_sdk.RetrieveFilter(ts="store_oslo_01_sku_4471_sales",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(days=30), end=pd.Timestamp.now(tz="UTC"),
+ aggregates=["sum"], granularity="1d")
+sales = [dp.sum for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()]
+plt.bar(range(len(sales)), sales); plt.title("Oslo-01 SKU 4471 daily units sold"); plt.show()
+```
+
+## See also
+
+- [Datasets reference](/reference/datasets) — segmentation and access by region.
+- [Query & aggregate](/guides/query-and-aggregate) — demand roll-ups.
+- [Turn readings into events](/guides/detect-events) — the reorder rule.
+- **[Demand forecasting (advanced)](/advanced/demand-forecasting)** — forecast demand to drive replenishment ahead of the shelf going empty.
diff --git a/versioned_docs/version-1.0/industries/transport-logistics/supply-chain.mdx b/versioned_docs/version-1.0/industries/transport-logistics/supply-chain.mdx
new file mode 100644
index 0000000..2f83a62
--- /dev/null
+++ b/versioned_docs/version-1.0/industries/transport-logistics/supply-chain.mdx
@@ -0,0 +1,176 @@
+---
+sidebar_position: 8
+title: Supply chain — traceability
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Supply chain — traceability & recall
+
+**The problem.** A defect is found in a component from one supplier lot. The clock
+starts immediately: which finished products contain a part from that lot, which
+shipments carried them, and which customers received them? Answering it by querying
+flat tables is slow and error-prone — but the supply chain *is* a graph (supplier →
+part → product → shipment), and one traversal turns a recalled lot into the exact,
+minimal recall list.
+
+This scenario centres on **graph traceability**: model the chain of custody as
+relationships, and trace a recall downstream from a single bad lot.
+
+## Set up demo data
+
+New workspace? Run this once (Python) to create the chain of custody — a supplier lot built
+into a product, shipped out — so the downstream recall trace returns the affected items.
+Safe to re-run.
+
+```python
+import intellistream_datahub_sdk
+
+client = intellistream_datahub_sdk.DataHubClient.from_env()
+
+client.resources.create(
+ [intellistream_datahub_sdk.Resource(external_id=x, name=x, labels=[x]) for x in
+ ["lot_acme_8842", "part_bearing_55", "product_gearbox_910", "shipment_eu_2204"]],
+ [intellistream_datahub_sdk.RelForm.by_external_ids("lot_acme_8842", "part_bearing_55", "supplied"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("part_bearing_55", "product_gearbox_910", "built_into"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("product_gearbox_910", "shipment_eu_2204", "shipped_in")])
+```
+
+## 1. Model the chain of custody
+
+Each hop is a relationship: a supplier lot `supplied` a part, a part `built_into` a
+product, a product `shipped_in` a shipment.
+
+
+
+
+```java
+List chain = List.of(
+ rel("supplied", "lot_acme_8842", "part_bearing_55"),
+ rel("built_into", "part_bearing_55", "product_gearbox_910"),
+ rel("shipped_in", "product_gearbox_910", "shipment_eu_2204"));
+
+client.resources().create(nodes, chain);
+```
+
+```java
+static RelForm rel(String type, String from, String to) {
+ RelForm r = new RelForm();
+ r.setName(type);
+ r.setFromExternalId(from);
+ r.setToExternalId(to);
+ return r;
+}
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+chain = [
+ intellistream_datahub_sdk.RelForm.by_external_ids("lot_acme_8842", "part_bearing_55", "supplied"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("part_bearing_55", "product_gearbox_910", "built_into"),
+ intellistream_datahub_sdk.RelForm.by_external_ids("product_gearbox_910", "shipment_eu_2204", "shipped_in"),
+]
+client.resources.create(nodes, chain)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::relations::RelForm;
+
+let chain = vec![
+ RelForm::by_external_ids("lot_acme_8842", "part_bearing_55", "supplied"),
+ RelForm::by_external_ids("part_bearing_55", "product_gearbox_910", "built_into"),
+ RelForm::by_external_ids("product_gearbox_910", "shipment_eu_2204", "shipped_in"),
+];
+api.resources.create(nodes, chain).await?;
+```
+
+
+
+
+## 2. Trace a recall downstream
+
+A bad lot is found. Walk the graph out from `lot_acme_8842` and everything reachable
+— parts, products, shipments — is exactly what the recall must cover. Nothing more,
+nothing missed.
+
+
+
+
+```java
+ResourceNetwork impacted = client.resources().fetchRelated("lot_acme_8842", 10);
+
+impacted.nodes().forEach(n -> {
+ if (n.getExternalId().startsWith("shipment_"))
+ System.out.println("recall shipment: " + n.getExternalId());
+ if (n.getExternalId().startsWith("product_"))
+ System.out.println("affected product: " + n.getExternalId());
+});
+```
+
+
+
+
+```python
+impacted = client.resources.fetch_related(external_id="lot_acme_8842", depth=10)
+
+for node in impacted.nodes:
+ if node.external_id.startswith("shipment_"):
+ print("recall shipment:", node.external_id)
+ elif node.external_id.startswith("product_"):
+ print("affected product:", node.external_id)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let impacted = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("lot_acme_8842").with_depth(10)).await?;
+
+for node in impacted.nodes() {
+ if node.external_id().starts_with("shipment_") {
+ println!("recall shipment: {}", node.external_id());
+ }
+ if node.external_id().starts_with("product_") {
+ println!("affected product: {}", node.external_id());
+ }
+}
+```
+
+
+
+
+A generous `depth` follows the chain as far as it goes; filtering on the supply
+relationship types keeps the walk to the chain of custody. See
+[Correlate alarms with the graph](/guides/correlate-alarms) for the traversal pattern.
+
+## 3. Track flow, segment by facility
+
+Throughput and inventory are series (`dc_oslo_throughput_units`,
+`sku_4471_on_hand`); group each distribution centre's data with a
+[dataset](/reference/datasets) so each site sees only its own. A stockout becomes a
+`stockout` [event](/guides/detect-events) for the planning team.
+
+## See the result
+
+Tracing downstream from the bad lot returns exactly what to recall:
+
+```text
+affected product: product_gearbox_910
+recall shipment: shipment_eu_2204
+```
+
+## See also
+
+- [Model assets as a graph](/guides/model-assets-graph) — modeling the chain of custody.
+- [Correlate alarms with the graph](/guides/correlate-alarms) — downstream traversal in detail.
+- [Datasets reference](/reference/datasets) — segment data per facility.
diff --git a/versioned_docs/version-1.0/intro.md b/versioned_docs/version-1.0/intro.md
new file mode 100644
index 0000000..ab2fbf5
--- /dev/null
+++ b/versioned_docs/version-1.0/intro.md
@@ -0,0 +1,42 @@
+---
+sidebar_position: 1
+slug: /
+title: DataHub SDK
+---
+
+# DataHub SDK
+
+A thin client for the **DataHub Platform**: manage resources, time series,
+events, files, and stream live data from your own application.
+
+The SDK is available for **Java**, **Python** and **Rust**. Pick your language once
+(top-right of any code block) and every example on the site gives you a working
+guide in your language of choice.
+
+:::tip Five-minute start
+Head to the [Quick start](/quickstart) to create a client and write your first
+datapoint, or jump to [high-throughput ingestion](/guides/ingest-timeseries) to ingest data at scale.
+:::
+
+## What you can do
+
+- **Time series**: create series, ingest datapoints in parallel, query raw values and aggregates.
+- **Resources**: model assets and their relationships as a graph.
+- **Events**: record and query operational events.
+- **Files**: attach documents and images to your assets.
+- **Subscriptions**: tail live data over a streaming connection.
+
+## Licence
+
+The three SDKs, and the Java wire-contract model the Java SDK depends on, are
+[Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0). They were placed under that licence
+so that linking one into your application carries no copyleft obligation into your code. The
+platform they talk to is [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.html); that licence
+covers the server, and using the server over its API is not what it restricts.
+
+## Where to go next
+
+- **[Tutorial](/tutorial)**: build a small metrics agent end to end: create series, ingest on a schedule, and survive API outages.
+- **[Guides](/guides/ingest-timeseries)**: task recipes: ingestion, graph modeling, querying & aggregation, live consumption, and event detection.
+- **[Industry scenarios](/industries/overview)**: end-to-end walkthroughs across ten sectors, from oil & gas and energy to finance and healthcare.
+- **[API reference](/reference/client)**: every service, method and option, in Java, Python and Rust.
diff --git a/versioned_docs/version-1.0/mcp-server.mdx b/versioned_docs/version-1.0/mcp-server.mdx
new file mode 100644
index 0000000..9e07edb
--- /dev/null
+++ b/versioned_docs/version-1.0/mcp-server.mdx
@@ -0,0 +1,328 @@
+---
+sidebar_position: 4
+title: MCP servers
+description: Connect an AI agent to DataHub over the Model Context Protocol - the two endpoints, authentication, the 38 tools they expose, and the limits worth knowing before you ship one.
+---
+
+# MCP servers
+
+DataHub ships two [MCP](https://modelcontextprotocol.io/) servers, so an agent can work
+against your data model directly instead of you hand-writing an integration layer for it:
+
+| Server | Endpoint | Exposes |
+| --- | --- | --- |
+| API | `/mcp` (port 8081 locally) | **37 tools** across data sets, resources, relationships, time series, events, labels and units |
+| Analysis | the analysis service's `/mcp` (port 8082 locally) | **one tool**, [`analysis_related_series`](#the-analysis-server) |
+
+The important part is what they *aren't*: each MCP endpoint is an ordinary Spring endpoint on
+the same security chain as the REST API. There is no separate agent door, no service account
+with blanket access, and no MCP-specific bypass. An agent sees exactly what the identity in
+its token is allowed to see.
+
+## Endpoints
+
+Both servers speak the same transport:
+
+| | |
+| --- | --- |
+| Path | `POST /mcp`, the only route |
+| Transport | Streamable HTTP, Spring AI WebMVC |
+| Protocol mode | `STATELESS`: no initialize handshake, no session to hold |
+| Server version | `1.0.0` |
+
+## Authentication
+
+Identical for both servers, and identical to every other call in this documentation: an
+OAuth2 Bearer JWT in the `Authorization` header, on each request.
+
+```http
+POST /mcp HTTP/1.1
+Host: api.intellistream.ai
+Authorization: Bearer
+Content-Type: application/json
+```
+
+Three things are checked before any tool body runs:
+
+1. **Signature and issuer**, by the same `JwtDecoder` the REST API uses.
+2. **The `DATAHUB_ACCESS` role.** Every non-public endpoint requires `ROLE_DATAHUB_ACCESS`,
+ not merely a valid token. A token with a good organization claim but without this role
+ authenticates and can then read and mutate nothing.
+3. **Tenant.** The `organization.*.id` claim populates the tenant context through the same
+ `OrganizationValidator` used for REST, so an agent cannot reach another tenant's data
+ even by asking for an id it happens to know.
+
+Past those gates, per-data-set [access grants](/reference/datasets#access-control) apply
+exactly as on REST: rows in data sets the identity cannot read are omitted, writes it lacks
+a grant for are refused.
+
+The practical consequence: **an agent inherits the permissions of whoever it is acting for.** If you want an agent that can read
+but not write, issue it a token that can read but not write. There is no second permission
+model to keep in sync.
+
+For an unattended agent, get the token from the OAuth2 **client-credentials** grant. Whether
+the token request must also name a scope such as `organization:*` depends on how your realm
+issues the organization claim that check 3 reads: see
+[provider-specific parameters](/reference/client#provider-specific-parameters) in the client
+reference.
+
+## Connecting a client
+
+Any MCP client that speaks streamable HTTP and can attach a bearer token will work. The two
+servers are two entries; substitute the analysis URL for wherever your deployment exposes
+the analysis service:
+
+```json
+{
+ "mcpServers": {
+ "datahub": {
+ "type": "http",
+ "url": "https://api.intellistream.ai/mcp",
+ "headers": {
+ "Authorization": "Bearer ${DATAHUB_TOKEN}"
+ }
+ },
+ "datahub-analysis": {
+ "type": "http",
+ "url": "https:///mcp",
+ "headers": {
+ "Authorization": "Bearer ${DATAHUB_TOKEN}"
+ }
+ }
+ }
+}
+```
+
+Tokens are short-lived. For anything long-running, refresh through your IdP and update the
+header rather than pinning a token into a config file.
+
+### By hand, with curl
+
+Well-known MCP clients handle the transport for you. If you roll your own, two details
+matter: the body is plain JSON-RPC 2.0, and the `Accept` header must be **exactly**
+`application/json, text/event-stream`. Both media types, in that order, no `q=` parameters;
+anything else earns an empty `400`.
+
+```bash
+curl -s https://api.intellistream.ai/mcp \
+ -H "Authorization: Bearer $DATAHUB_TOKEN" \
+ -H "Content-Type: application/json" \
+ -H "Accept: application/json, text/event-stream" \
+ -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'
+```
+
+Calling a tool is the same request with a `tools/call` body:
+
+```json
+{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"resource_fetch_nearest","arguments":{"externalId":"pump_12","endLabels":"TIMESERIES"}}}
+```
+
+## The API server's tools
+
+Named `_` in snake_case, deliberately, so the listing reads the same to you
+and to the model.
+
+### Data sets
+
+| Tool | What it does |
+| --- | --- |
+| `dataset_list` | List data sets visible to the caller |
+| `dataset_search` | Full-text search, returns ids needed by `timeseries_create` |
+| `dataset_create` | Create one data set, returns the server-assigned id |
+| `dataset_update` | Update one data set by id. Only provided fields change |
+| `dataset_delete` | Delete by id or externalId. Does **not** cascade to members |
+
+### Resources
+
+A resource is any node in the graph that is not a data set or a time series: equipment, work
+items, documents.
+
+| Tool | What it does |
+| --- | --- |
+| `resource_search` | Full-text search over name, externalId and description, across every node type, ranked by relevance |
+| `resource_get` | Fetch one or more by id or externalId |
+| `resource_create` | Create one node. At least one label is required |
+| `resource_update` | Update common fields by id |
+| `resource_delete` | Delete by id or externalId. Connected edges go with it |
+| `resource_fetch_related` | Walk outward and return the connected neighbourhood |
+| `resource_fetch_nearest` | Breadth-first to the nearest nodes carrying a wanted label |
+
+`resource_fetch_related` is the one to reach for when a question is really about context:
+it returns the neighbourhood in a single call rather than making the agent traverse edge by
+edge, which is both slower and worse for the context window.
+
+`resource_fetch_nearest` asks the sharper question: not "what surrounds this node" but
+"where is the nearest X". Breadth-first from a starting resource (`externalId` or `id`), it
+returns the closest `limit` nodes (default 10) carrying one of the `endLabels`
+(comma-separated, such as `TIMESERIES`), plus the sub-graph connecting them.
+`relationshipTypes` restricts which edge types are followed; `excludedLabels` prunes
+branches. The cap counts matching end nodes, not hops, so a match five hops out is still
+found. Prefer it over `resource_fetch_related` whenever the agent hunts a specific kind of
+neighbour ("the time series measuring this pump").
+
+### Relationships
+
+| Tool | What it does |
+| --- | --- |
+| `edge_list_types` | List relationship types for the tenant, e.g. `PROCESSED_BY` |
+| `edge_create_type` | Create a type. Coerced to `SCREAMING_SNAKE_CASE` server-side |
+| `edge_create` | Connect two existing nodes, by externalId or numeric id |
+| `edge_get` | Fetch a single edge by numeric id |
+| `edge_delete` | Delete edges by id. Nodes on either side are untouched |
+
+### Time series
+
+| Tool | What it does |
+| --- | --- |
+| `timeseries_list` | Browse when you do not yet have a search term |
+| `timeseries_search` | Full-text search over name, externalId and description, ranked by relevance |
+| `timeseries_get` | Look up by externalId or numeric id |
+| `timeseries_create` | Create one time series, returns the assigned id. Needs a `unit` or a `unitExternalId`; a `unitExternalId` must be in the catalogue (see `unit_list`) |
+| `timeseries_update` | Update common fields by id |
+| `timeseries_delete` | Delete the definition. Its datapoints are purged afterwards, asynchronously |
+| `timeseries_get_latest` | Most recent datapoint, for "what is it reading now?" |
+| `timeseries_fetch_datapoints` | History between two timestamps, ISO-8601 or epoch millis |
+| `timeseries_send_datapoint` | Append a single datapoint |
+
+### Events
+
+| Tool | What it does |
+| --- | --- |
+| `event_search` | Substring search over externalId, description and metadata values, newest first |
+| `event_filter` | Structured query by time range or exact field values |
+| `event_get` | Full detail for specific events by externalId and/or UUID |
+| `event_create` | Record something that happened: an alarm, a state transition |
+| `event_update` | Update common fields by id or externalId |
+| `event_delete` | Delete by externalId and/or UUID, many in one call |
+
+Use `event_filter` rather than `event_search` for anything time-bounded or exact. Search is
+for when the agent only has words to go on.
+
+### Labels and units
+
+| Tool | What it does |
+| --- | --- |
+| `label_list` | List tenant labels, e.g. `PRODUCTION` |
+| `label_create` | Create one label, coerced to `SCREAMING_SNAKE_CASE` |
+| `label_update` | Update description, i18n code or colour by id |
+| `unit_list` | Units of measure known to DataHub, e.g. Celsius |
+| `unit_get` | Look up one unit by externalId, e.g. `temperature_deg_c` |
+
+## The analysis server
+
+The analysis service serves its own `/mcp` with a single tool, **`analysis_related_series`**:
+given one series, which others move with it over a window, and can the relation be trusted?
+It walks the knowledge graph outward from the focus series to collect physically related
+candidates, then tests each pair statistically and returns the candidates ranked.
+
+| Parameter | |
+| --- | --- |
+| `focusExternalId` | The series to explain. Required |
+| `start`, `end` | Window, ISO-8601 or epoch millis. Required |
+| `limit` | Candidates to test. Default 10, max 200 |
+| `relationshipTypes` | Optional comma-separated list; restricts the graph walk to these edge types |
+| `analyses` | Optional comma-separated subset of the tests below; default all |
+
+| `analyses` value | Question it answers |
+| --- | --- |
+| `raw` | Lagged cross-correlation on the aggregated series |
+| `whitened` | The same after ARIMA prewhitening, with Haugh-Box significance: does the relation survive removing trend and autocorrelation? |
+| `cointegration` | Engle-Granger: do the two series share a long-run equilibrium? |
+| `stability` | Does the correlation hold across subwindows, or ride one burst? |
+| `coherence` | Welch coherence: at what period is the shared periodicity strongest? |
+
+### Reading a result
+
+One entry from a call with focus `pump_flow`:
+
+```json
+{
+ "externalId": "discharge_pressure",
+ "name": "Discharge pressure",
+ "overlapCount": 1380,
+ "path": "pump_flow -FEEDS-> #55 -MEASURED_BY-> discharge_pressure",
+ "rawCorrelation": 0.84,
+ "rawLagSeconds": 120,
+ "whitenedCorrelation": 0.61,
+ "whitenedLagSeconds": 120,
+ "haughBoxPValue": 0.004,
+ "whitenedSignificant": true,
+ "cointegrated": true,
+ "stable": true,
+ "peakCoherence": 0.9,
+ "peakCoherencePeriodSeconds": 3600,
+ "rankScore": 0.87
+}
+```
+
+Read it as:
+
+- **`path`**: how the candidate is physically connected to the focus, flattened to one
+ string (`#55` is an intermediate node without an externalId).
+- **`rawCorrelation` at `rawLagSeconds`**: correlation 0.84 with the candidate shifted by
+ 120 s. A **positive lag means the candidate lags the focus** (the focus leads); here,
+ pressure follows flow by two minutes.
+- **`whitenedSignificant`, `haughBoxPValue` 0.004**: the correlation survives prewhitening,
+ so it is not an artifact of two series each trending or oscillating on their own. Trust
+ this over the raw number.
+- **`cointegrated`**: the pair shares a long-run equilibrium; after a shock they drift back
+ together.
+- **`stable`**: the correlation holds across subwindows rather than being carried by one
+ event.
+- **`peakCoherence` at `peakCoherencePeriodSeconds`**: the strongest shared cycle sits at
+ one hour.
+- **`overlapCount`**: how many aligned samples the tests ran on. Treat verdicts from a thin
+ overlap with suspicion.
+- **`rankScore`**: the combined score the list is sorted by.
+
+Fields for analyses you did not request are absent from each entry. Around the list, the
+response carries `focusExternalId` and `focusName`, `bucketSeconds` (the aggregation bucket
+the series were resampled to), `maxLagSeconds` (how far the lag scan went), `skipped`
+(candidates that could not be tested) and `message`.
+
+## How results are shaped
+
+The API server is written for a reader with a finite context window, which is not something
+the REST wire contract has to care about.
+
+- **Nothing empty is sent.** Every tool serializes with `NON_EMPTY` inclusion, so nulls,
+ empty collections and empty strings never reach the model. The REST DTOs keep their own
+ `ALWAYS` contract, unchanged.
+- **List and search return lean projections.** `LeanResource`, `LeanTimeseries` and friends
+ drop audit timestamps, empty metadata and embedded edge lists. Call the matching `*_get`
+ when the agent actually needs the full record, or `resource_fetch_related` for edges.
+- **Timestamps match the REST API**, so a value reads the same whichever way the agent reached
+ it. They come back as ISO-8601 strings, and a timestamp argument takes ISO-8601 or epoch
+ milliseconds, on the same [rules](/reference/timeseries#write-datapoints).
+
+The pattern to encourage in your prompts: search or list to find candidates, then fetch
+detail for the few that matter. An agent that opens with a full listing burns its context on
+records it will not use.
+
+## Limits worth knowing
+
+**The servers do not ask permission.** They run in `STATELESS` mode, which has no
+elicitation: a tool cannot open a channel back to the user to confirm a mutating action.
+`resource_delete`, `event_delete` and `timeseries_send_datapoint` execute the moment they are
+called. If your agent should pause for a human before it writes, **that gate belongs in the
+agent you build**, not in the server. Give the agent a read-only token if it has no business
+writing at all.
+
+**The tools spend the REST budget.** `/mcp` sits on the same chain as everything else, so an
+agent's calls count against the same per-minute [rate limit](/reference/limits#rate-limits)
+and daily ingest quota as your own jobs, and hit the same field, batch and body-size caps. An
+agent that lists in a loop can rate-limit the pipeline it shares a tenant with. The refusal
+comes back as a `429` carrying `Retry-After`, which is worth teaching the agent to honour.
+
+**Deletes do not cascade.** Deleting a data set that still has resources or time series in it
+fails: remove or reassign the members first. (Deleting a time series is the exception: its
+datapoints are purged after it, asynchronously.) This is deliberate, but it means an agent
+tidying up will need more than one call, in the right order, and a half-finished tidy leaves
+orphans.
+
+**Name coercion is server-side, and not uniform.** `edge_create_type` and `label_create`
+normalise a name to `SCREAMING_SNAKE_CASE`, so `derived from` becomes `DERIVED_FROM`, and the
+labels given to `resource_create` are canonicalised the same way when stored. `edge_create`
+only upper-cases its `relationshipType`, so `derived from` becomes `DERIVED FROM`, a different
+type from `DERIVED_FROM`. Agents that construct a name and then search for the string they
+constructed will miss; have the agent write relationship types as `SCREAMING_SNAKE_CASE`.
diff --git a/versioned_docs/version-1.0/quickstart.mdx b/versioned_docs/version-1.0/quickstart.mdx
new file mode 100644
index 0000000..032a88d
--- /dev/null
+++ b/versioned_docs/version-1.0/quickstart.mdx
@@ -0,0 +1,218 @@
+---
+sidebar_position: 2
+title: Quick start
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Quick start
+
+From zero to a stored datapoint in about five minutes.
+
+## 1. Install
+
+:::note Which version this page describes
+This is the documentation for the 1.0.0 release of the Python, Rust and Java SDKs.
+
+| SDK | Release |
+| --- | --- |
+| Python | 1.0.0, on PyPI |
+| Rust | 1.0.0, on crates.io |
+| Java | 1.0.0, on Maven Central |
+:::
+
+
+
+
+The Java SDK is on Maven Central and needs Java 25 or newer. It brings
+`datahub-api-model` with it, so one dependency is enough:
+
+```kotlin
+// build.gradle.kts
+repositories {
+ mavenCentral()
+}
+
+dependencies {
+ implementation("ai.intellistream:datahub-sdk:1.0.0")
+}
+```
+
+To try calls newer than the release, build the SDK from the
+[platform repository](https://github.com/IntelliStream-DataHub/datahub-platform) with
+`./gradlew :datahub-api-model:publishToMavenLocal :datahub-java-sdk:publishToMavenLocal`,
+add `mavenLocal()` to `repositories`, and use the `version` from its `gradle.properties`.
+
+
+
+
+```bash
+pip install intellistream-datahub-sdk
+```
+
+
+
+
+```toml
+# Cargo.toml
+[dependencies]
+intellistream-datahub-sdk = "1.0"
+```
+
+
+
+
+:::tip The name is long on purpose, and you can shorten it
+`datahub-sdk` was already taken on PyPI by an unrelated project, and the `datahub` name is
+taken, so the package is qualified with the organization and the Rust crate takes the same
+name. Alias it if you would rather not type it:
+
+```python
+import intellistream_datahub_sdk as dh
+```
+
+```toml
+# Cargo.toml — a crate-wide rename
+dh = { package = "intellistream-datahub-sdk", version = "1.0" }
+```
+:::
+
+## 2. Configure credentials
+
+The client reads configuration from the environment (or a `.env` file): a base URL and
+**either** a static token **or** OAuth2 client-credentials.
+
+```bash
+BASE_URL=https://api.intellistream.ai
+TOKEN=eyJhbGciOi... # or CLIENT_ID / CLIENT_SECRET / TOKEN_URI
+```
+
+## 3. Create a client and write a datapoint
+
+
+
+
+```java
+var client = DatahubClient.fromEnv();
+
+var series = new Timeseries();
+series.setExternalId("engine_temperature");
+series.setName("Engine temperature");
+series.setUnit("celsius");
+series.setValueType("float");
+client.timeseries().create(series);
+
+client.timeseries().ingest(Map.of(
+ "engine_temperature", List.of(
+ Datapoint.of(Instant.now(), 92.4))));
+```
+
+
+
+
+```python
+from datetime import datetime, timedelta, timezone
+
+import intellistream_datahub_sdk
+from intellistream_datahub_sdk import DataHubClient
+
+client = DataHubClient.from_envfile(None) # the environment, plus a .env file if there is one
+
+client.timeseries.create([
+ intellistream_datahub_sdk.TimeSeries(external_id="engine_temperature", unit="celsius", value_type="float")])
+
+client.timeseries.insert_from_lists(
+ timestamps=[datetime.now(timezone.utc)], values=[92.4], ts="engine_temperature")
+```
+
+
+
+
+Every call is `async`, so run these lines inside
+`#[tokio::main] async fn main() -> Result<(), Box>`, with
+`tokio = { version = "1", features = ["full"] }` and `chrono = "0.4"` added to your
+dependencies.
+
+```rust
+use intellistream_datahub_sdk::create_api_service;
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use chrono::Utc;
+
+let api = create_api_service();
+
+let mut ts = TimeSeries::new("engine_temperature", "Engine temperature");
+ts.set_unit("celsius");
+ts.set_value_type("float");
+api.time_series.create_one(&ts).await?;
+api.time_series
+ .insert_datapoint(None, Some("engine_temperature".into()), Utc::now(), "92.4".into())
+ .await?;
+```
+
+
+
+
+## 4. Read it back
+
+
+
+
+```java
+var filter = new RetrieveFilter();
+filter.setExternalId("engine_temperature");
+filter.setStart(ZonedDateTime.now().minusHours(1));
+filter.setEnd(ZonedDateTime.now());
+
+var request = new DataRetriever();
+request.setItems(List.of(filter));
+
+client.timeseries().retrieve(request).getItems()
+ .forEach(c -> c.getDatapoints().forEach(System.out::println));
+```
+
+
+
+
+```python
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=datetime.now(timezone.utc) - timedelta(hours=1),
+ end=datetime.now(timezone.utc))
+
+for dp in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints():
+ print(dp.timestamp, dp.value)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+
+let filter = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(1)),
+ end: Some(Utc::now()),
+ ..Default::default()
+};
+let points = api.time_series.retrieve_datapoints(&DataWrapper::from(vec![filter])).await?;
+for c in points.get_items() {
+ for p in &c.datapoints {
+ println!("{} = {:?}", p.timestamp, p.value);
+ }
+}
+```
+
+
+
+
+:::note Pick once, follow everywhere
+The tabs above all switch together, and your choice persists as you move through the
+docs. One selection for the whole site.
+:::
+
+## Next: a real program
+
+The [tutorial](/tutorial) turns these calls into a complete agent: it samples your
+machine's memory every few seconds, creates its series idempotently, and survives API
+outages with the durable ingest buffer.
diff --git a/versioned_docs/version-1.0/reference/_category_.json b/versioned_docs/version-1.0/reference/_category_.json
new file mode 100644
index 0000000..946c8d4
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/_category_.json
@@ -0,0 +1,9 @@
+{
+ "label": "Reference",
+ "position": 8,
+ "link": {
+ "type": "generated-index",
+ "title": "Java client reference",
+ "description": "Per-service reference for the Java client — every method, its parameters, and what it returns. All three clients (Java, Python, Rust) target the same DataHub REST API; the signatures below are the Java ones."
+ }
+}
diff --git a/versioned_docs/version-1.0/reference/binary-datapoints.md b/versioned_docs/version-1.0/reference/binary-datapoints.md
new file mode 100644
index 0000000..b94138e
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/binary-datapoints.md
@@ -0,0 +1,297 @@
+---
+sidebar_position: 4.5
+title: Binary datapoint frames
+description: The wire format of POST /timeseries/data/binary, the high-throughput datapoint insert, for anyone producing frames outside the SDKs.
+---
+
+# Binary datapoint frames
+
+`POST /timeseries/data/binary` is the high-throughput form of
+[`POST /timeseries/data`](./timeseries#write-datapoints). The body is one or more
+**datapoint frames**: a 28-byte envelope, a directory of the series in the frame, and a
+zstd-compressed Arrow IPC stream in one canonical schema per value type, rows sorted by series
+and timestamp. Nothing is parsed as text, so a request carries up to a million points in a few
+megabytes.
+
+The SDKs build frames for you, see [binary ingest](./timeseries#binary-ingest): `ingestBinary` in
+Java, `insert_datapoints_binary` in Rust and in Python's synchronous client. This page is the
+byte-level contract for everyone else: a producer with pyarrow or arrow-rs, or a gateway
+forwarding frames it did not build.
+
+| | |
+| --- | --- |
+| Method and path | `POST /timeseries/data/binary` |
+| `Content-Type` | `application/vnd.intellistream.datapoint-block` |
+| `Content-Encoding` | None (or `identity`). Compression lives inside each frame; any other body-level encoding is a `415`. |
+| Body | One or more frames back to back, the last ending exactly at the end of the body. |
+| Success | `204`, no body. Every frame in the request was validated and published. |
+| Failure | An [`application/problem+json`](#responses) document, and **nothing** was inserted. |
+
+## The envelope {#envelope}
+
+All integers are little-endian.
+
+| Offset | Size | Field |
+| --- | --- | --- |
+| 0 | 4 | Magic, the ASCII bytes `DHDP`. |
+| 4 | 1 | Version, `1`. |
+| 5 | 1 | Value type id: `1` bigint, `2` float, `3` numeric, `4` text, `5` decimal32, `6` mixed, `7` float32. |
+| 6 | 1 | Codec, `1` = Arrow IPC stream. `0` is reserved. |
+| 7 | 1 | Compression, `1` = zstd. Required: `0` is refused with `uncompressed-frame`. |
+| 8 | 4 | `u32` row count, at least 1. |
+| 12 | 4 | `u32` series count, the number of distinct series ids in the payload. |
+| 16 | 4 | `u32` directory length in bytes. |
+| 20 | 4 | `u32` payload length in bytes, as carried (compressed). |
+| 24 | 4 | `u32` payload length in bytes after decompression. |
+| 28 | | The [directory](#directory), then the [payload](#payload). |
+
+The zstd level is the sender's choice. The API decompresses the payload into exactly the
+declared number of bytes, and a stream that ends early or runs long is refused.
+
+## The directory {#directory}
+
+Series count entries, each an `int64` series id, a `varuint` byte length, then that many bytes
+of UTF-8 external id (1 to 1 024 bytes). `varuint` is unsigned LEB128: seven bits per byte, low
+group first, high bit set on every byte but the last.
+
+Ids are **strictly ascending** and must be exactly the set of ids in the payload. The directory
+is left uncompressed so the API can read which series a frame touches without touching the
+payload.
+
+Series are named by their **internal id**, which is why a producer looks them up first.
+`POST /timeseries/byids` returns `id` (a JSON string, see
+[entity ids](./client#results--errors)), `externalId` and `valueType` for each external id. The
+external id you write into the directory is checked against the series on every request, which
+is how a renamed or recreated series is caught: a mismatch is a `422` telling you to refresh your
+cache, not a silent write into the wrong series.
+
+:::note Read access as well as write
+`/timeseries/byids` omits series in data sets you cannot **read**, so a binary producer needs
+read and write on the data set. The JSON path names series by external id and needs write
+alone. See [access control](./datasets#access-control).
+:::
+
+## The payload {#payload}
+
+One zstd frame containing one Arrow IPC **stream**: one schema message, one or more record
+batches, and the end-of-stream marker, with nothing after it. Little-endian throughout.
+
+The stream is read by a minimal codec that accepts exactly the subset a plain writer emits and
+refuses the rest, with the `reason` on the right:
+
+| Rule | `reason` |
+| --- | --- |
+| Every message uses the continuation-marker framing (`0xFFFFFFFF`, then the metadata length), metadata version V4 or newer. | `payload-invalid` |
+| Exactly one schema message, before any record batch, with no body. | `payload-invalid` |
+| The schema matches the [canon](#schemas) for the frame's value type: field count, names, types with their parameters, nullability; no dictionary encoding, no children. Custom metadata is ignored. | `schema-mismatch` |
+| No dictionary batches; record batch bodies are not compressed in-band; no variadic buffers. | `payload-invalid` |
+| Buffers lie inside the body, 8-byte aligned; a non-nullable field declares zero nulls; `Utf8` offsets never decrease or run past the data. | `payload-invalid` |
+| Nothing after the end-of-stream marker. | `trailing-bytes` |
+| The rows of all batches add up to the envelope's row count. | `row-count-mismatch` |
+| Rows are sorted by `(timeseries_id, timestamp)` ascending across the whole frame. | `unsorted` |
+| No two rows share a `(timeseries_id, timestamp)`. | `duplicate-row` |
+| A text value is valid UTF-8, at most 64 characters and 256 bytes; a decimal fits its precision; a `mixed` row sets exactly one of its two value columns. | `value-invalid` |
+
+Timestamps are epoch milliseconds, UTC. There is no seconds heuristic: the type is
+unambiguous, so a millisecond value is taken as given.
+
+### Canonical schemas {#schemas}
+
+Every frame starts with the same two fields, then one value field decided by the envelope's
+value type (two for `mixed`):
+
+| Field | Arrow type | Nullable |
+| --- | --- | --- |
+| `timeseries_id` | `Int64` | no |
+| `timestamp` | `Timestamp(MILLISECOND, "UTC")` | no |
+
+| Value type | Value field(s) |
+| --- | --- |
+| `bigint` | `value: Int64` |
+| `float` | `value: Float64` |
+| `float32` | `value: Float32` |
+| `numeric` | `value: Decimal128(18, 6)` |
+| `decimal32` | `value: Decimal128(9, 4)` |
+| `text` | `value: Utf8` |
+| `mixed` | `value_numeric: Float64` nullable and `value_text: Utf8` nullable, exactly one set per row |
+
+The timestamp's unit and zone must be exactly `MILLISECOND` and `"UTC"`, and the decimals must
+be 128-bit with exactly that precision and scale. These are the storage types, which is why the
+server casts nothing on the way in. See [value types](./timeseries#value-types) for which type
+to give a series.
+
+## Caps {#caps}
+
+Fixed in the format, not deployment policy, except the last row:
+
+| Cap | Value | Over it |
+| --- | --- | --- |
+| Rows per frame, numeric types | 100 000 | `400 row-count-mismatch` |
+| Rows per frame, `text` or `mixed` | 10 000 | `400 row-count-mismatch` |
+| Series per frame | 10 000 | `400 directory-invalid` |
+| External id in the directory | 1 024 bytes | `400 directory-invalid` |
+| Text value | 64 characters and 256 bytes | `400 value-invalid` |
+| Decompressed payload per frame | 4 MiB | `413 frame-too-large` |
+| Frames per request | 32 | `413 too-many-frames` |
+| Decompressed total per request | 64 MiB | `413 request-too-large` |
+| Compressed body per request | 64 MiB by default, `datahub.limits.max-body-bytes-datapoints-binary` | `413`, no `reason` |
+
+A frame is published as one message, so the 4 MiB keeps it under the broker's limit with room
+to spare. 100 000 float points are about 2.4 MB decompressed and a few hundred KB compressed, so
+a full frame is comfortably inside it; a producer only needs to split on the row cap.
+
+## Responses {#responses}
+
+Every refusal is an [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) problem document carrying
+the `type` its status calls for, with a stable `reason` beside it as the sub-code, plus
+`frameIndex` (0-based) when one frame is at fault and `timeseriesIds` when particular series are.
+Branch on the `type`, and on `reason` to tell apart the thirteen ways a frame can be malformed:
+
+```json
+{
+ "type": "https://intellistream.ai/errors/external-id-mismatch",
+ "title": "External id mismatch",
+ "status": 422,
+ "detail": "2 series in frame 3 have a different external id than the directory says; refresh the client's series cache.",
+ "reason": "external-id-mismatch",
+ "frameIndex": 3,
+ "timeseriesIds": [1041, 1042]
+}
+```
+
+Every `type` below is prefixed `https://intellistream.ai/errors/`.
+| Status | `type` | `reason` | Meaning |
+| --- | --- | --- | --- |
+| `204` | | | Accepted and published. |
+| `400` | `invalid-frame` | `malformed-frame`, `unsupported-version`, `unsupported-codec`, `unknown-value-type`, `uncompressed-frame`, `trailing-bytes`, `directory-invalid`, `payload-invalid`, `schema-mismatch`, `row-count-mismatch`, `unsorted`, `duplicate-row`, `value-invalid` | The frame at `frameIndex` breaks the format; `detail` says where. Fix the producer. |
+| `403` | `dataset-forbidden`, or `tenant-limit-reached` | | The caller lacks write on a series' data set (the usual [data set `403`](./datasets#access-control)), or the tenant is at a [lifetime ceiling](./limits#lifetime-ceilings). Neither carries a `reason`. |
+| `404` | `unknown-timeseries` | `unknown-timeseries` | `timeseriesIds` do not exist in this tenant. Re-resolve them. |
+| `413` | `request-too-large` | `frame-too-large`, `too-many-frames`, `request-too-large` | Over a [cap](#caps). Split; a retry as-is never succeeds. A body over the compressed cap is refused before any frame is read, without a `reason`. |
+| `415` | `unsupported-media-type` | `unsupported-content-encoding` | A `Content-Encoding` header. The wrong `Content-Type` is also a `415`, from the framework, without a `reason`. |
+| `422` | `value-type-mismatch` | `value-type-mismatch` | `timeseriesIds` in `frameIndex` have another value type than the envelope declares. |
+| `422` | `external-id-mismatch` | `external-id-mismatch` | `timeseriesIds` in `frameIndex` have another external id than the directory says. Drop them from your cache and resolve again. |
+| `429` | `too-many-in-flight` | `too-many-in-flight` | This API instance is already validating its limit of binary requests; `Retry-After` is one second and [`retry`](./client#problem-documents) is `same-request`. |
+| `429` | `rate-limit-exceeded`, `ingest-quota-exceeded` | | The ordinary [rate limit](./limits#rate-limits) or [daily quota](./limits#daily-ingest-quotas), with no `reason`. |
+
+Two of the seven binary types are the API's own rather than this endpoint's: an oversized request
+is `request-too-large` and a rejected `Content-Encoding` is `unsupported-media-type` whichever
+path produced it, so a client that already handles them needs nothing new here. The other five
+are specific to binary frames.
+
+All seven used to be one type, `datapoint-block-rejected`, which therefore answered `400`, `404`,
+`413`, `415`, `422` and `429`; RFC 9457 gives a type one status. A client matching on that type
+matches nothing now. `reason` is unchanged, so one matching on `reason` still works.
+
+### Retrying {#retrying}
+
+Every frame is validated before any is published, so a rejected request inserted **nothing** and
+is retried **as a whole**. Datapoints are keyed by `(series, timestamp)`, so replaying a request
+that did reach the store, say after a lost response, is harmless.
+
+- `429` with `Retry-After`, a `502`/`503`/`504` and a dropped connection: wait and resend the same
+ body. The document says so itself in `retry`, and a refusal marked `needs-operator` (a `500`,
+ say) is not worth resending.
+- `404 unknown-timeseries` and `422 external-id-mismatch`: forget the named ids, look them up
+ again by external id, rebuild the frames and resend once. If it fails the same way, the series
+ really is gone.
+- Everything else: fix the request.
+
+The Java and Rust SDKs handle the first two for you, and Python's binary methods do through
+Rust. Rust waits a second longer before each retry rather than reading `Retry-After`.
+
+Quotas are charged after validation, so a refused request costs nothing. The
+[`ingested bytes`](./limits#daily-ingest-quotas) quota counts **decompressed** bytes on this
+path: compressing harder does not stretch the allowance.
+
+## Producing frames from pyarrow {#pyarrow}
+
+:::info Needs a sandbox
+The example writes into `engine_temperature`, which section **A** of
+[Seed a sandbox](/guides/seed-a-sandbox) creates.
+:::
+
+An IPC stream written by pyarrow or arrow-rs with the schema above, no in-band compression and
+no dictionary encoding is exactly what the codec accepts. What is left is the envelope and the
+zstd wrap. The example resolves the series, picks the schema its value type calls for, writes an
+hour of readings and reads them back:
+
+```python
+import struct
+import pyarrow as pa, pyarrow.ipc as ipc, requests, zstandard
+
+BASE, HEADERS = "https://api.intellistream.ai", {"Authorization": "Bearer ..."}
+CANON = {"float": (2, pa.float64()), "float32": (7, pa.float32()), "bigint": (1, pa.int64())}
+
+# 1. Resolve the series: internal id, external id and value type. Needs read on its data set.
+found = requests.post(f"{BASE}/timeseries/byids", headers=HEADERS,
+ json={"items": [{"externalId": "engine_temperature"}]}).json()["items"][0]
+sid, ext = int(found["id"]), found["externalId"].encode()
+type_id, value_type = CANON[found["valueType"]]
+
+# 2. One record batch in that canon, sorted by (timeseries_id, timestamp).
+schema = pa.schema([
+ pa.field("timeseries_id", pa.int64(), nullable=False),
+ pa.field("timestamp", pa.timestamp("ms", tz="UTC"), nullable=False),
+ pa.field("value", value_type, nullable=False)])
+t0, rows = 1_767_139_200_000, 3600 # 2025-12-31T00:00:00Z, one reading a second
+table = pa.table({
+ "timeseries_id": pa.array([sid] * rows, pa.int64()),
+ "timestamp": pa.array([t0 + i * 1000 for i in range(rows)], pa.timestamp("ms", tz="UTC")),
+ "value": pa.array([88.0 + (i % 40) / 10 for i in range(rows)], value_type),
+}, schema=schema).sort_by([("timeseries_id", "ascending"), ("timestamp", "ascending")])
+
+# 3. The IPC stream, then zstd around it.
+sink = pa.BufferOutputStream()
+with ipc.new_stream(sink, schema) as writer: # default options: no in-band compression
+ writer.write_table(table)
+raw = sink.getvalue().to_pybytes()
+payload = zstandard.ZstdCompressor(level=9).compress(raw)
+
+# 4. Directory and envelope.
+def varuint(n):
+ out = bytearray()
+ while n >= 0x80:
+ out.append((n & 0x7F) | 0x80)
+ n >>= 7
+ out.append(n)
+ return bytes(out)
+
+directory = struct.pack(" = writer.build(9)?.bytes; // zstd level; one frame, ready to POST
+```
+
+`build` refuses a frame over the [caps](#caps). Check `is_full_for(id)` before each `add` and
+start a new writer when it says so; frames go in one body back to back.
diff --git a/versioned_docs/version-1.0/reference/client.md b/versioned_docs/version-1.0/reference/client.md
new file mode 100644
index 0000000..0ffafb7
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/client.md
@@ -0,0 +1,734 @@
+---
+sidebar_position: 1
+title: Client & configuration
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Client & configuration
+
+The client is the entry point: it owns a shared HTTP connection and token handling and
+exposes one accessor per service. It is safe to share: **create one and reuse it** for
+the lifetime of your application.
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.client.DatahubClient;
+import ai.intellistream.datahub.sdk.client.DatahubConfig;
+
+// from the environment (and a .env file, if present)
+DatahubClient client = DatahubClient.fromEnv();
+
+// or explicitly
+DatahubClient client = DatahubClient.create(DatahubConfig.builder()
+ .baseUrl("https://api.intellistream.ai")
+ .token(System.getenv("TOKEN"))
+ .build());
+```
+
+
+
+
+```python
+from intellistream_datahub_sdk import DataHubClient
+
+# from the environment (or an explicit .env file)
+client = DataHubClient.from_env()
+client = DataHubClient.from_envfile("/path/to/.env")
+
+# or explicitly
+client = DataHubClient(base_url="https://api.intellistream.ai", token="...")
+```
+
+For `async`/`await`, use `AsyncDataHubClient` instead, same methods, awaited (except the
+[binary datapoint](./binary-datapoints) ingest, `insert_datapoints_binary` and
+`insert_from_lists_binary`, which only `DataHubClient` has):
+
+```python
+from intellistream_datahub_sdk import AsyncDataHubClient
+client = AsyncDataHubClient.from_env()
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::create_api_service;
+
+// from the environment (and a .env file, if present)
+let api = create_api_service();
+```
+
+Every method is `async`, so call them from an async runtime (e.g. `#[tokio::main]`) and
+`.await` the result.
+
+Don't want async? Enable the `blocking` cargo feature and use
+`intellistream_datahub_sdk::blocking` instead, the same calls without `.await`, driven by the
+SDK's own runtime (the `reqwest` / `reqwest::blocking` split):
+
+```rust
+use intellistream_datahub_sdk::blocking;
+
+let api = blocking::create_api_service();
+```
+
+The blocking client covers most of the async one, not all of it. It has no `subscriptions`, and
+it lacks `time_series.filter`, `time_series.listen_datapoints`, `resources.filter`,
+`resources.get_by_id`, `resources.fetch_nearest`, `events.get`, `events.update` and
+`events.count`. `api.async_api()`
+hands you the async service for those.
+
+
+
+
+## Services
+
+| Service | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Resources | `client.resources()` | `client.resources` | `api.resources` |
+| [Assets](./resources#assets) | `client.assets()` | `client.assets` | `api.assets` |
+| [Functions](./resources#functions) | `client.functions()` | `client.functions` | `api.functions` |
+| Time series | `client.timeseries()` | `client.timeseries` | `api.time_series` |
+| Datasets | `client.datasets()` | `client.datasets` | `api.datasets` |
+| Events | `client.events()` | `client.events` | `api.events` |
+| Units | `client.units()` | `client.units` | `api.units` |
+| Files | `client.files()` | `client.files` | `api.files` |
+| Subscriptions | `client.subscriptions()` | `client.subscriptions` | `api.subscriptions` |
+| Edges | `client.edges()` | `client.edges` | `api.edges` |
+| [Labels](./labels) | `client.labels()` | `client.labels` | `api.labels` |
+| [Tenant](./tenant) | `client.tenant()` | `client.tenant` | `api.tenant` |
+
+Assets and functions are the `ASSET`- and `FUNCTION`-labelled resources, typed. A service being
+in a client does not mean every call is: each reference page has a "What each client covers"
+table.
+
+The Java client adds two more services:
+
+| Service | Java | Covers |
+| --- | --- | --- |
+| [Policies](./policies) | `client.policies()` | `/policies`, including the naming dry-run |
+| [Governance templates](./policies#governance-templates) | `client.governance()` | `/governance/templates` |
+
+:::note One endpoint no client wraps, on purpose
+`GET /stats` is internal to the console and marked hidden: it is not part of the published
+contract, so nothing should call it.
+:::
+
+## Authentication
+
+All three clients read the same configuration: a base URL plus **either** a static
+bearer token **or** OAuth2 client-credentials (the SDK fetches and refreshes the token).
+
+| Variable | Meaning |
+| --- | --- |
+| `BASE_URL` | API base URL (required) |
+| `TOKEN` | Static bearer token |
+| `CLIENT_ID` / `CLIENT_SECRET` / `TOKEN_URI` | OAuth2 client-credentials (all three) |
+| `PROJECT_NAME` | Optional. Every client accepts it (`.projectName(...)`, `project_name=`), but no request uses it |
+
+Java's `fromEnv()` and Rust's `create_api_service()` read these from the environment, falling
+back to a `.env` file in the working directory (real environment variables win). Python's
+`from_env()` and Rust's `DataHubConfig::from_env()` read the process environment only. To load a
+file there, use `from_envfile(path)` (Python) or `DataHubConfig::from_envfile(Some(path))`
+(Rust), where real environment variables also win. Rust and Python read
+`DATAPOINT_INSERT_PARALLELISM` from the same place, see
+[what the SDKs do about limits](./limits#sdk-behaviour).
+
+### Provider-specific parameters
+
+Every client asks for `openid` in every token request to `TOKEN_URI`, and `scope` **adds** to
+it rather than replacing it: `SCOPE=organization:*` asks for `openid organization:*`. The API
+reads your dataset grants from the identity provider's UserInfo endpoint, which refuses a token
+without `openid`. `audience` is left out unless you set it.
+
+What DataHub needs is a token carrying the `organization` claim naming exactly one
+organization (tenant routing and [dataset grants](/reference/datasets#access-control) ride on
+it); whether that takes a scope depends on how your realm issues the claim. A realm using a
+client protocol mapper (the common production setup) puts it on every token, so leave `SCOPE`
+unset. A realm using Keycloak Organizations only issues it when the request names
+`organization:*` or `organization:`. When the claim is missing, every call fails
+`401 invalid_token`, which looks like a credentials problem but is not.
+
+When the claim is present but names an organization this deployment holds no tenant for
+(never onboarded, or since removed), every call fails **`403`** with an
+`application/problem+json` body of `type: ".../errors/unknown-tenant"` naming the refused
+`organizationId`. Retrying never helps: an administrator has to register the organization.
+
+| Variable | Java builder | Python kwarg | Rust setter | When you need it |
+| --- | --- | --- | --- | --- |
+| `SCOPE` | `.scope(...)` | `scope=` | `set_scope(...)` | `organization:*` if your realm issues the organization claim through Keycloak Organizations (see above). Space-separate several. Entra ID's `api:///.default` does not go here: Entra rejects it next to `openid`, so it belongs in [`ASSERTION_SCOPE`](#exchanging-an-external-token-jwt-bearer). |
+| `AUDIENCE` | `.audience(...)` | `audience=` | `set_audience(...)` | Auth0 requires it. Keycloak ignores it. |
+
+### When a call returns 401
+
+`401 invalid_token` has two causes, and they need different fixes.
+
+| Cause | What you see | Fix |
+| --- | --- | --- |
+| The token carries no `organization` claim | Every call fails, from the first one | Set `SCOPE` (see above) |
+| The identity provider refused the token | Calls succeed, then start failing part-way through a run | Get a new token |
+
+The second one catches long-running processes. The API checks your token locally (signature,
+expiry, issuer) and separately reads your data set grants from the identity provider's UserInfo
+endpoint. A token can pass the first check and still be refused by the second: it is unexpired,
+but the session behind it has ended, because an idle or maximum session lifetime elapsed or
+somebody signed out. The response carries `WWW-Authenticate: Bearer error="invalid_token"` and a
+problem+json body with `type: ".../errors/token-rejected"`.
+
+Retrying does not clear that. Only a new token does.
+
+:::note Not the same as 503
+`503` with `type: ".../errors/permissions-unavailable"` means the API could not reach the identity
+provider to check your grants. That one is temporary, and worth retrying.
+:::
+
+### Exchanging an external token (jwt-bearer)
+
+A token minted by one issuer is not accepted by an API that trusts another. To bridge them, the
+SDK can present an externally-issued JWT as an [RFC 7523](https://www.rfc-editor.org/rfc/rfc7523)
+`assertion` and exchange it for a token the API *does* accept. The common case is reaching a
+Keycloak-backed API with an **Entra ID service principal**:
+
+```
+1. client_credentials 2. jwt-bearer 3. Bearer
+ ────────────────► ────────────────► ────────────────►
+ Entra token endpoint Keycloak token endpoint DataHub API
+ → the assertion → the token you use
+```
+
+With `CLIENT_SECRET` set, an assertion source switches the request at `TOKEN_URI` from
+client-credentials to `jwt-bearer`. `CLIENT_ID`/`CLIENT_SECRET`/`TOKEN_URI` then describe the
+client performing the *exchange*, and the `ASSERTION_*` keys describe where the assertion comes
+from:
+
+| Variable | Java builder | Python kwarg | Rust setter | Meaning |
+| --- | --- | --- | --- | --- |
+| `ASSERTION` | `.assertion(...)` | `assertion=` | `set_assertion(...)` | A ready-made JWT. Never refreshed, prefer the credentials below. |
+| `ASSERTION_CLIENT_ID` / `ASSERTION_CLIENT_SECRET` / `ASSERTION_TOKEN_URI` | `.assertionCredentials(...)` | `assertion_client_id=` / `assertion_client_secret=` / `assertion_token_url=` | `set_assertion_credentials(...)` | Fetch the assertion with client credentials from another provider (all three). |
+| `ASSERTION_SCOPE` | `.assertionScope(...)` | `assertion_scope=` | `set_assertion_scope(...)` | `scope` for the assertion request, sent as given with no `openid` added. Entra ID requires `api:///.default`. |
+| `ASSERTION_AUDIENCE` | `.assertionAudience(...)` | `assertion_audience=` | `set_assertion_audience(...)` | `audience` for the assertion request. |
+| `ASSERTION_GRANT` | — | `assertion_grant=` | `set_assertion_grant(...)` | Without `CLIENT_SECRET` only: `client_credentials` (default) or `jwt-bearer`. |
+
+Rust and Python can also do the exchange without `CLIENT_SECRET`. The assertion then
+authenticates the client itself, sent as the RFC 7523 `client_assertion` (Keycloak's *Signed
+JWT - Federated* client authenticator), and no `client_id` is sent. `ASSERTION_GRANT` picks the
+grant in that mode: `client_credentials` (the default) gets a token for the client's service
+account, `jwt-bearer` one for the user linked to the assertion's subject. It needs Keycloak 26.6
+or later, with the `federated-jwt` execution in the realm's client authentication flow. Java
+always needs `CLIENT_SECRET` for the exchange.
+
+:::note Python names the URL parameters `*_url`
+The Python client already spells `TOKEN_URI` as `token_url`, so the assertion equivalent is
+`assertion_token_url`. The environment variables keep the `_URI` spelling in all three SDKs.
+:::
+
+
+
+
+```java
+DatahubClient client = DatahubClient.create(DatahubConfig.builder()
+ .baseUrl("https://api.intellistream.ai")
+ // leg 2 — the confidential Keycloak client that performs the exchange
+ .clientCredentials("datahub-jwt-grant", keycloakSecret,
+ "https://keycloak.example.com/realms/datahub/protocol/openid-connect/token")
+ // leg 1 — the Entra app registration the assertion comes from
+ .assertionCredentials(entraAppId, entraSecret,
+ "https://login.microsoftonline.com/" + tenantId + "/oauth2/v2.0/token")
+ .assertionScope("api://" + entraAppId + "/.default")
+ .build());
+```
+
+
+
+
+```python
+client = DataHubClient(
+ "https://api.intellistream.ai",
+ # leg 2 — the confidential Keycloak client that performs the exchange
+ client_id="datahub-jwt-grant",
+ client_secret=keycloak_secret,
+ token_url="https://keycloak.example.com/realms/datahub/protocol/openid-connect/token",
+ # leg 1 — the Entra app registration the assertion comes from
+ assertion_client_id=entra_app_id,
+ assertion_client_secret=entra_secret,
+ assertion_token_url=f"https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token",
+ assertion_scope=f"api://{entra_app_id}/.default",
+)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::{ApiService, datahub::DataHubConfig};
+
+// leg 2 — BASE_URL and the Keycloak client (CLIENT_ID, CLIENT_SECRET, TOKEN_URI) from the env
+let mut config = DataHubConfig::from_env()?;
+// leg 1 — the Entra app registration the assertion comes from
+config.set_assertion_credentials(
+ &entra_app_id,
+ &entra_secret,
+ &format!("https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token"),
+);
+config.set_assertion_scope(format!("api://{entra_app_id}/.default"));
+let api = ApiService::new(config);
+```
+
+
+
+
+Or entirely from the environment:
+
+```bash
+BASE_URL=https://api.intellistream.ai
+CLIENT_ID=datahub-jwt-grant
+CLIENT_SECRET=...
+TOKEN_URI=https://keycloak.example.com/realms/datahub/protocol/openid-connect/token
+ASSERTION_CLIENT_ID=
+ASSERTION_CLIENT_SECRET=...
+ASSERTION_TOKEN_URI=https://login.microsoftonline.com//oauth2/v2.0/token
+ASSERTION_SCOPE=api:///.default
+```
+
+The exchanged token is cached and refreshed exactly like a client-credentials one. The assertion
+itself is **never** cached, providers commonly reject a replayed assertion, so every exchange
+starts from a fresh request.
+
+:::caution Server-side setup is required
+The identity provider must be configured to trust the external issuer, and the external identity
+must map to a real user on that side. For Keycloak that means an Identity Provider with **JWT
+Authorization Grant** enabled (Keycloak 26.5+), a client with the matching capability, and a
+linked user carrying the roles and tenant claim. See
+[`EntraID.md`](https://github.com/IntelliStream-DataHub/datahub-platform/blob/master/EntraID.md)
+in the platform repository for the full walkthrough, including the audience and
+assertion-lifetime settings that trip up a first attempt.
+:::
+
+### From HashiCorp Vault (Java)
+
+The Java client can also read the same keys from a Vault KV v2 secret, with a token or
+with AppRole:
+
+```java
+DatahubConfig cfg = DatahubConfig.fromVault(vaultAddr, vaultToken, "datahub/sdk");
+DatahubConfig cfg = DatahubConfig.fromVaultEnv("datahub/sdk"); // VAULT_ADDR + VAULT_TOKEN
+DatahubConfig cfg = DatahubConfig.fromVaultAppRole(vaultAddr, roleId, secretId, "datahub/sdk");
+DatahubConfig cfg = DatahubConfig.fromVaultAppRoleEnv("datahub/sdk"); // VAULT_ADDR + VAULT_ROLE_ID + VAULT_SECRET_ID
+```
+
+## Durable ingest buffering
+
+Optional and **off by default**. When enabled, datapoint and event ingestion that can't reach the
+API, or is rejected with an auth failure (HTTP 401/403, e.g. an expired or rotated token), spools
+to disk and is flushed automatically on the next ingest call. Neither a transient outage nor a
+credential hiccup loses data or raises. The buffer is a segmented, compressed log (gzip in Java,
+zstd in Rust/Python) bounded on two axes, either of which may be left unset; an unset axis defaults
+to **72 hours** / **5 GiB** once buffering is on:
+
+- **time**: datapoints/events older than the window are dropped.
+- **size**: when the on-disk spool exceeds the cap, the oldest segment is dropped.
+
+It is memory-safe: the spool is drained in segments, so even a multi-gigabyte buffer never loads
+into memory, and it is recovered from disk on the next start.
+
+
+
+
+```java
+DatahubClient client = DatahubClient.create(DatahubConfig.builder()
+ .baseUrl("https://api.intellistream.ai")
+ .token(System.getenv("TOKEN"))
+ .enableBuffering() // 72 h / 5 GiB defaults
+ // .bufferRetention(Duration.ofMinutes(60)) // override the time window
+ // .bufferMaxBytes(2L * 1024 * 1024 * 1024) // override the size cap
+ // .bufferDirectory(Path.of("datahub-spool")) // default: .datahub-spool
+ .build());
+
+IngestResult r = client.timeseries().ingest(byExternalId);
+if (r.buffered() > 0) {
+ // server unreachable: r.buffered() datapoints are spooled, retried on the next call
+}
+```
+
+`fromEnv()` instead reads `BUFFER_RETENTION` (an ISO-8601 duration, e.g. `PT72H`),
+`BUFFER_MAX_BYTES` and `BUFFER_DIRECTORY`. Setting either bound turns buffering on.
+
+
+
+
+```python
+client = DataHubClient(
+ base_url="https://api.intellistream.ai",
+ token="...",
+ enable_buffering=True, # 72 h / 5 GiB defaults
+ buffer_retention_secs=3600, # optional: override the time window
+ buffer_max_bytes=2 * 1024**3, # optional: override the size cap
+ buffer_dir="datahub-spool", # optional, default .datahub-spool
+)
+```
+
+`from_env()` / `from_envfile(path)` instead read `ENABLE_BUFFERING`, `BUFFER_RETENTION_SECS`,
+`BUFFER_MAX_BYTES` and `BUFFER_DIR` from the environment.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::{ApiService, datahub::DataHubConfig};
+
+let mut config = DataHubConfig::from_env().unwrap();
+config
+ .enable_buffering() // 72 h / 5 GiB defaults
+ .set_buffer_retention_secs(3600) // optional: override the time window
+ .set_buffer_max_bytes(2 * 1024 * 1024 * 1024) // optional: override the size cap
+ .set_buffer_dir("datahub-spool"); // optional, default .datahub-spool
+let api = ApiService::new(config);
+```
+
+Or via the environment (read by `create_api_service()`): `ENABLE_BUFFERING=true`,
+`BUFFER_RETENTION_SECS`, `BUFFER_MAX_BYTES`, `BUFFER_DIR`.
+
+
+
+
+:::note Java never spools one `403`
+A [lifetime ceiling](./limits#lifetime-ceilings) answers `403` too, and the Java client
+**surfaces it rather than buffering it**. The auth failures are worth spooling because a rotated
+token or a missing grant is fixed out of band and the data then flushes; a ceiling never becomes
+acceptable by being replayed, so spooling it would fill the buffer with data the server
+refuses every time. Java matches the problem `type`, so an ordinary permission `403` is
+buffered.
+
+Rust and Python do not make that exception: with buffering on they spool every `401` and `403`,
+a ceiling included. The call then returns without an error, and the refused data stays in the
+spool until the time window or size cap drops it.
+:::
+
+:::note Retries are idempotent
+A flush re-sends buffered data, which is safe: datapoints are keyed by `(series, timestamp)` and
+events by `id`, so the backend collapses duplicates. The SDK stamps each event with a time-ordered
+UUID v7 before the first send, so a retried event keeps the same id (see [Events](./events)).
+:::
+
+## Results & errors
+
+Most calls return the entity (or a thin wrapper around a list of them); a non-2xx
+response surfaces as an exception/error carrying the HTTP status, the raw body, and the
+[problem document](#problem-documents) the API explained itself with, when it sent one.
+
+
+
+
+Methods return `DataWrapper`: `getItems()` holds the results, as a `List` in the order the
+API returned them, so a single result is `getItems().get(0)`. Non-2xx throws
+`DatahubApiException`:
+
+```java
+import ai.intellistream.datahub.models.IdCollection;
+import ai.intellistream.datahub.sdk.http.DatahubApiException;
+
+try {
+ DataWrapper r = client.resources().byIds(List.of(IdCollection.createFromExternalId("pump_1")));
+ r.getItems().forEach(System.out::println);
+} catch (DatahubApiException e) {
+ System.err.println(e.statusCode() + ": " + e.body());
+}
+```
+
+`e.problem()` reads the same failure as the document the API explained it with, and
+`e.retryAfterSeconds()` the `Retry-After` it carried: see
+[reading the problem document](#problem-documents).
+
+
+
+
+Most methods return plain `list[T]`. Non-2xx raises `DataHubException`:
+
+```python
+from intellistream_datahub_sdk import DataHubException
+
+try:
+ resources = client.resources.by_ids(["pump_1"])
+except DataHubException as e:
+ print(e.status_code, e.message)
+```
+
+
+
+
+Most methods return `Result, ResponseError>`, where `get_items()` holds the
+results. `resources.by_ids`, `create`, `update` and `delete`, `assets.update`,
+`functions.update` and `edges.by_ids` return a
+`GraphDataWrapper` instead, whose `nodes()` holds them. `ResponseError` exposes
+`get_status()` and `get_message()` (its `Display` prints both):
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+match api.resources.by_ids(&vec![IdAndExtId::from_external_id("pump_1")]).await {
+ Ok(wrapper) => for node in wrapper.nodes().unwrap_or_default() { println!("{:?}", node); }
+ Err(e) => eprintln!("{}: {}", e.get_status(), e.get_message()),
+}
+```
+
+
+
+
+:::note Entity ids are JSON strings on the wire
+64-bit ids are serialized as JSON **strings** so they survive JavaScript's 2⁵³ number
+limit. Each client reads them back into a native integer, so this only matters if you
+inspect raw responses.
+:::
+
+### Reading the problem document {#problem-documents}
+
+A refusal the API explains comes back as an
+[RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) `application/problem+json` document:
+`type`, `title`, `status`, `detail` and `instance`, plus extension members. Every `type` this
+API mints sits under `https://intellistream.ai/errors/`, and the **`type` is the contract**.
+Branch on it, or on its kebab-case tail (the *slug*), never on `title` or `detail`: those are
+prose and may be reworded at any time (RFC 9457 §3.1.1).
+
+
+
+
+`DatahubApiException.problem()` returns a `Problem`, and it is **never null**: a body that was
+empty, HTML from a proxy or JSON of another shape yields a `Problem` carrying the HTTP status
+alone, so `slug()` is simply `null` there. It exposes `type()`, `slug()`, `is("some-slug")`,
+`title()`, `status()`, `detail()`, `instance()`, `retry()`, `retryable()` (true exactly when
+`retry()` is `same-request`), `requestId()`,
+`fields()` (a `List` of `field()`, `message()`, `code()` and `rejected()`) and
+`extensions()`, a `Map` of every other member (`duplicated`, `blockedBy`,
+`reason`, `pointer`, ...), unknown ones included. `e.body()` is still the raw body and
+`e.retryAfterSeconds()` the `Retry-After` in seconds, or `-1`:
+
+```java
+import ai.intellistream.datahub.api.errors.Problem;
+import ai.intellistream.datahub.sdk.http.DatahubApiException;
+
+try {
+ client.timeseries().create(List.of(series));
+} catch (DatahubApiException e) {
+ Problem p = e.problem(); // never null
+ if (p.slug() == null) { // not a problem document, see the caution below
+ System.err.println(e.statusCode() + ": " + e.body());
+ } else if (p.is("validation-failed")) {
+ for (Problem.FieldProblem f : p.fields()) {
+ System.err.println(f.field() + ": " + f.message());
+ }
+ } else if (p.is("unreadable-request-body")) {
+ System.err.println(p.extensions().get("errors"));
+ } else {
+ System.err.println(p.type() + ": " + p.detail());
+ }
+}
+```
+
+
+
+
+`DataHubException` carries the whole document as `problem` (a `dict`), its `type` as
+`problem_type`, and the slug as `problem_slug`:
+
+```python
+from intellistream_datahub_sdk import DataHubException
+
+try:
+ client.timeseries.create([ts])
+except DataHubException as e:
+ if e.problem is None: # not a problem document, see the caution below
+ print(e.status_code, e.message)
+ elif e.problem_slug == "validation-failed":
+ for field in e.problem.get("fields", []):
+ print(field["field"], field["message"])
+ elif e.problem_slug == "unreadable-request-body":
+ for offender in e.problem.get("errors", []):
+ print(offender["pointer"], "accepts", offender["allowedFields"])
+ else:
+ print(e.problem_type, e.problem.get("detail"))
+```
+
+
+
+
+`ResponseError::problem()` returns `Option`, `problem_slug()` the slug on its own,
+and `content_type()` the response media type. `ProblemDetail` holds the standard members as public
+fields and reads the extensions through `fields()`, `unknown_fields()`, `duplicated()`,
+`blocked_by()`, `retry()`, `request_id()`, `docs()`, `location()` and `pointer()`:
+
+```rust
+if let Err(e) = api.time_series.create_one(&series).await {
+ match e.problem() {
+ // Not a problem document, see the caution below.
+ None => eprintln!("{}: {}", e.get_status(), e.get_message()),
+ Some(problem) => match problem.slug() {
+ Some("validation-failed") => for f in problem.fields() {
+ eprintln!("{:?}: {:?}", f.field, f.message);
+ }
+ Some("unreadable-request-body") => for o in problem.unknown_fields() {
+ eprintln!("{:?} accepts {:?}", o.pointer, o.allowed_fields);
+ }
+ // A document with no `type` still carries a status and prose.
+ _ => eprintln!("{:?}: {:?}", problem.status, problem.detail),
+ },
+ }
+}
+```
+
+
+
+
+Beside the standard members, a problem carries whichever of these the failure has something to
+say with. Members a client does not recognise are **kept, not dropped** (RFC 9457 §3.2), so one
+the API adds later is readable without a client upgrade.
+
+| Member | Sent with | Holds |
+| --- | --- | --- |
+| `fields` | Validation failures | One entry per rejected field: `field`, `message`, `code` (the i18n key behind the message, so you can localise rather than parse English) and `rejected` (an argument such as an offending length, never the value you sent) |
+| `errors` | [Unknown fields](#unknown-fields) | One entry per unrecognised property: `pointer` to it, and `allowedFields`, the names accepted at that position |
+| `duplicated` | A `409` on an identifier already taken | Each collision, as field to value |
+| `blockedBy` | A refused delete | What stands in the way, each named by its own ids so you can go and clear it |
+| `retry` | Most refusals | `same-request`, `change-request` or `needs-operator` |
+| `requestId` | Most refusals | Quote it when you ask an operator about the failure |
+| `docs` | Types with a page that explains them | A link to that page. Absent when nothing is written yet |
+
+`retry` is **advisory**. It says whether repeating the request unchanged could ever work, which is
+not the same question as what a client should do. Java's in-process retry does act on it (below),
+but buffering does not, and on purpose: a `401` is marked `change-request` and a `403`
+`needs-operator`, yet [durable buffering](#durable-ingest-buffering) still spools both, so a
+rotated credential does not cost the batch. What each ingest path does with either is
+[which failures are worth retrying](#retryable-failures).
+
+:::caution Not every failure is a problem document
+The API now answers every refusal with a typed problem document, but a proxy or gateway in front
+of it, or an older API, can still hand you plain text, HTML, a Spring whitelabel body, the legacy
+`{"error": {...}}` wrapper or a success-shaped `{"items": [...]}` envelope. Several of those
+arrive labelled `application/json`, so the `Content-Type` does not separate them and the clients
+read the body's structure instead. `problem` is absent for every one of them in Python and Rust;
+in Java `problem()` is never null, but carries the HTTP status alone and leaves `slug()` null. A
+problem document from something other than this API may also carry no `type`, which leaves the
+slug absent too. Write that branch first and keep the status and raw message as the fallback.
+:::
+
+### Which failures are worth retrying {#retryable-failures}
+
+The API answers a limit it will forgive differently from one it will not, so a client can tell
+them apart from the status alone:
+
+| Response | Meaning | Do |
+| --- | --- | --- |
+| `429` + `Retry-After` | A [rate limit or daily quota](./limits) | Wait the seconds it names and replay |
+| `413` | The [request body](./limits#request-body-size) is too large | Split the batch, never retry as-is |
+| `403` with `type: ".../errors/tenant-limit-reached"` | A [lifetime ceiling](./limits#lifetime-ceilings) | Nothing to wait for: it is raised by asking |
+| `400` / `422` | Validation, including the [field and batch caps](./limits#field-caps) | Fix the request |
+| `422` with `type: ".../errors/invalid-timestamp"` or `".../errors/invalid-datapoint"` | A [timestamp](#timestamps) in neither accepted form, or a [datapoint value](./timeseries#write-datapoints) its series' value type cannot parse | Fix the value; `retry` is `change-request` |
+| `404` with `type: ".../errors/unknown-timeseries"` or `422` with `type: ".../errors/external-id-mismatch"` | A [binary datapoint request](./binary-datapoints#responses) naming a series that was removed or renamed since you cached it | Re-resolve the ids in `timeseriesIds`, rebuild, send once more; the Java SDK does |
+
+The ingest paths act on that split for you, differently per client. Java's `ingest` follows the
+problem's `retry` member wherever the answer carries one: `same-request` is replayed with backoff,
+`change-request` and `needs-operator` are surfaced. So a `409 optimistic-lock` is retried and a
+`500 internal` is not. Where there is no problem document (a network failure, a proxy's HTML
+`502`), the status decides as before: status `0`, `429` and `5xx` are retried. Java floors its
+backoff at the `Retry-After` the response asked for, and fails the batch at once rather than
+sleeping when that is over 30 seconds, so a spent daily quota reaches your code, or the spool,
+instead of parking a thread until midnight UTC.
+
+Rust and Python retry JSON ingest through the [durable spool](#durable-ingest-buffering): with
+buffering on, a `408`, `429`, `5xx` or network failure and a `401` or `403` are written to disk and sent
+again, oldest first, by the next ingest call; with it off, they reach you. Their binary path,
+`insert_datapoints_binary`, retries `429`, `5xx` and network failures up to three times by
+default, 1, 2 and 3 seconds apart, and rebuilds a request refused for a removed or renamed
+series once. Neither waits the `Retry-After`. [Limits & quotas](./limits#sdk-behaviour) has
+the numbers.
+
+### Batch writes are all-or-nothing
+
+Every call that takes a list is validated in full before anything is written, so one bad item
+in 500 creates nothing and the error names every offending item rather than the first. Retry
+the whole batch once you have fixed them. A [binary datapoint request](./binary-datapoints) is
+the same: every frame is validated before any is published. A `validation-failed` problem names
+each offender in its `fields` member ([reading it](#problem-documents)).
+
+Two responses are worth recognising by shape:
+
+- **`400` with `type: ".../errors/naming-policy"`**: one or more external ids broke the
+ configured [naming policy](./external-ids#the-naming-policy). Nothing was created; the
+ `violations` array names each one and suggests a replacement.
+- **A `warnings` array beside `items` on a `2xx`**: the write succeeded, and the ids in it
+ are in a data steward's queue. The field is absent when empty.
+
+Both shapes, and the rules behind them, are in
+[External ids & naming](./external-ids).
+
+### Unknown fields are refused {#unknown-fields}
+
+A request body naming a field the endpoint does not have is a `400`, not a silent success. A
+typo, or a field that has since been retired, would otherwise be dropped and answered `200`,
+telling you a change was applied when nothing happened. The body is an
+[RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) problem document of
+`type: ".../errors/unreadable-request-body"`, with one `errors` entry per offender, each
+located by a JSON Pointer and listing the names accepted at that position:
+
+```json
+{
+ "type": "https://intellistream.ai/errors/unreadable-request-body",
+ "title": "Bad Request",
+ "status": 400,
+ "detail": "Unknown field: eventTime",
+ "errors": [
+ {
+ "detail": "Unknown field",
+ "pointer": "#/items/0/update/eventTime",
+ "allowedFields": ["dataSetId", "description", "externalId", "metadata",
+ "relatedResources", "source", "status", "subType", "type"]
+ }
+ ]
+}
+```
+
+Every offender in the body is reported at once, at whatever depth it sits, so several stale
+fields cost one round trip rather than one each. A body that cannot be parsed at all,
+malformed JSON or a value of the wrong shape, answers with the same `type` and a `detail`
+naming the problem, plus `line` and `column` where the parser can say.
+
+The clients only ever send fields they declare, so this reaches you when you build a body by
+hand, or keep an old field name in one. How each client hands you the `errors` entries:
+[Reading the problem document](#problem-documents).
+
+## Timestamps {#timestamps}
+
+Every timestamp the API accepts reads the same way: `eventTime`, datapoint timestamps, the
+`createdTime` / `lastUpdatedTime` / `eventTime` filter bounds, the datapoint delete window,
+file metadata dates, the `/analysis` window and the MCP tool parameters. Two forms are
+accepted.
+
+| Form | Example | Rules |
+| --- | --- | --- |
+| Epoch milliseconds, UTC | `1767225600000` | A bare number is always milliseconds, never seconds. 12 to 14 digits, which spans 1973-03-03 to the year 5138. |
+| ISO-8601 with an offset | `2026-01-01T00:00:00Z` | `Z`, `+02:00`, `-04:00` and a bracketed region id all work. Fractional seconds and minute precision both parse. |
+
+Two mistakes are refused rather than accepted and misread:
+
+- **Epoch seconds.** Ten digits is exactly the shape of a seconds value, so it falls outside
+ the accepted width and is refused, with an error that names the mistake and tells you to
+ multiply by 1000. Nothing is scaled for you. A caller that has been sending seconds to
+ `POST /events/create` now sees a `422` where the write used to succeed and store a time tens
+ of thousands of years out.
+- **ISO-8601 without an offset.** `2026-01-01T00:00:00` and a bare date `2026-01-01` are
+ refused rather than assumed to be UTC.
+
+For an instant **before 1973-03-03**, use the ISO-8601 form. The numeric form does not reach
+back that far.
+
+In the places below, a refused timestamp is answered with a `422` of
+`type: ".../errors/invalid-timestamp"` and `retry: change-request`. `detail` is the parser's own
+message, naming both accepted forms and the factor of 1000 for seconds. Where it was sent decides
+how the offender is located:
+
+| Sent in | Located by | Was |
+| --- | --- | --- |
+| A JSON body: an event's `eventTime`, a `createdTime` / `lastUpdatedTime` / `eventTime` filter bound, or the `start` / `end` of a [datapoint retrieve](./timeseries#retrieve-datapoints) | `pointer`, a JSON Pointer to the field (`#/eventTime`); no `line` or `column` | `400` of `type: ".../errors/unreadable-request-body"`, with `pointer`, `line` and `column` |
+| A bound of the [datapoint delete window](./timeseries#delete-datapoints) | `fields`, naming the bound and the series' `externalId` | `400` with `fields` |
+| A datapoint's `timestamp` on `POST /timeseries/data` | `fields`, naming `timestamp` and the series' `externalId` | `500` |
diff --git a/versioned_docs/version-1.0/reference/datasets.md b/versioned_docs/version-1.0/reference/datasets.md
new file mode 100644
index 0000000..494b709
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/datasets.md
@@ -0,0 +1,266 @@
+---
+sidebar_position: 5
+title: Data sets
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Data sets
+
+Logical groupings of resources and time series. Data sets can be nested (a data set can
+belong to a parent data set), and that hierarchy is live in queries: filtering time series
+by a data set also matches everything beneath it.
+[Filter series →](./timeseries#filter-series)
+
+The hierarchy is built from `BELONGS_TO` relationships, and the server enforces that: a
+relationship pointing at a data set must be `BELONGS_TO`, and a data set can only claim a
+time series that isn't already in another data set. Despite the name, the edge runs from the
+parent to the child, and the dependable way to build it is to create that edge explicitly rather
+than setting `connectedDataSets` (`connected_data_sets` in Python and Rust) on the new data set.
+[Relationship rules →](./resources#create-resources-and-relations)
+
+:::note External ids are stored exactly as you send them. In the Rust sdk
+`Dataset::new` derives one from the name, in snake_case; that is a client-side default,
+not a server rule.
+
+Uniqueness and lookup both ignore case, so `plant_a` collides with `PLANT_A` and either
+spelling finds the same data set. Data sets are also subject to the
+[naming policy](./external-ids#the-naming-policy) if an administrator has set one.
+[External ids & naming →](./external-ids)
+:::
+
+## Create
+
+
+
+
+```java
+DataSetModel dataset = new DataSetModel();
+dataset.setExternalId("plant_a");
+dataset.setName("Plant A");
+dataset.setMetadata(Map.of("tier", "gold"));
+
+client.datasets().create(List.of(dataset));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+dataset = intellistream_datahub_sdk.Dataset(
+ external_id="plant_a", name="Plant A", metadata={"tier": "gold"})
+client.datasets.create([dataset])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::datasets::Dataset;
+
+// external_id is derived as snake_case of the name → "plant_a"
+let mut dataset = Dataset::new("Plant A".into());
+dataset.add_metadata("tier".into(), "gold".into());
+api.datasets.create(&vec![dataset]).await?;
+```
+
+
+
+
+## Look up & delete
+
+`by_ids` takes ids and external ids together and leaves out what it cannot find rather than
+failing. `GET /datasets/{id}` reads one data set by numeric id and answers `404` for a miss.
+
+
+
+
+```java
+DataWrapper some = client.datasets()
+ .byIds(List.of(IdCollection.createFromExternalId("plant_a")));
+DataWrapper one = client.datasets().getById(some.getItems().get(0).getId());
+
+client.datasets().delete(List.of(IdCollection.createFromExternalId("plant_a")));
+```
+
+
+
+
+```python
+some = client.datasets.by_ids(["plant_a"])
+one = client.datasets.get_by_id(some[0].id)
+client.datasets.delete(["plant_a"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let some = api.datasets.by_ids(&vec![IdAndExtId::from_external_id("plant_a")]).await?;
+let one = api.datasets.get_by_id(some.get_items()[0].id.unwrap()).await?;
+api.datasets.delete(&vec![IdAndExtId::from_external_id("plant_a")]).await?;
+```
+
+
+
+
+## Filter {#filter}
+
+`POST /datasets/filter` finds data sets by structured criteria, combined with AND. It is exactly
+the criteria every node type shares, a data set has no `dataSetId` of its own, being the thing
+other nodes are scoped *by*:
+
+| Criterion | Matching |
+| --- | --- |
+| `id` | Numeric ids, matched exactly. |
+| `externalId`, `name`, `source` | Patterns, case-insensitive. `*` and `%` are wildcards, `_` is literal, and an entry with no wildcard matches exactly. |
+| `labels` | Data sets carrying **all** of these labels. |
+| `metadata` | Every key/value pair given must be present. **A null value matches the key alone**, whatever it holds. |
+| `createdTime`, `lastUpdatedTime` | `{ "min": …, "max": … }` bounds. |
+
+Each field except `labels` and `metadata` takes **either a bare value or an array**, whose
+entries are combined with OR, which is why they are named in the singular. `limit` is capped at
+10000, and the page can be ordered and walked exactly as
+[timeseries](./timeseries#sorting-and-paging) can. The server's default `limit` is 1000, but
+not every client leaves it to the server:
+
+
+
+
+```java
+DataSetFilter criteria = new DataSetFilter();
+criteria.setName(List.of("Plant *"));
+criteria.setMetadata(Map.of("tier", "gold"));
+
+DataWrapper matches = client.datasets().filter(criteria);
+```
+
+Pass a `DataSetRetreiver` instead of the bare criteria to set `limit`, `sort` or `cursor`.
+
+
+
+
+```python
+matches = client.datasets.filter(name="Plant *", metadata={"tier": "gold"}, limit=100)
+```
+
+Or build the criteria once as a `DatasetFilter` and pass it as `filter=`, keeping `limit`,
+`sort_by`, `sort_order` and `cursor` on the call. Either way it returns a list-like
+[`Page`](./timeseries#sorting-and-paging) carrying `.next_cursor`.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::datasets::{DatasetFilter, DatasetFilterForm};
+use std::collections::HashMap;
+
+let criteria = DatasetFilter::new()
+ .set_name(vec!["Plant *".to_string()])
+ .set_metadata(HashMap::from([("tier".to_string(), Some("gold".to_string()))]))
+ .build();
+let mut form = DatasetFilterForm::from_filter(criteria);
+form.set_limit(100);
+let matches = api.datasets.filter(&form).await?;
+```
+
+`DatasetFilter` is the criteria and `DatasetFilterForm` the request body around it, which also
+takes `set_paging` for `sort` and `cursor`.
+
+
+
+
+:::note There is no `writeProtected` or `deactivated` criterion
+A filter carrying either, or any other unknown field, is refused with a `400` as an
+[unknown field](./client#unknown-fields).
+:::
+
+## Search {#search}
+
+`POST /datasets/search` is a free-text search over data sets. The phrase is matched against
+`name`, `externalId` and `description`, fuzzily and word-aware. `limit` defaults to 100 and caps at
+**1 000**, lower than the 10 000 of `filter`, and `query` must be 3 to 140 characters.
+
+Results are **ranked by relevance** (`ts_rank`), strongest match first, with `id` as a tie-break so
+equal-scoring rows keep a stable order and repeated identical searches agree. Ranking means the
+database scores and sorts every match before applying `limit`, so a very broad phrase costs more
+than a narrow one.
+
+`filter` is optional and takes the same criteria as [`POST /datasets/filter`](#filter). It only
+ever removes matches: the phrase decides what the candidates are.
+
+```json
+{
+ "search": { "query": "work order" },
+ "filter": { "metadata": { "source_system": "sap" } },
+ "limit": 50
+}
+```
+
+No match is an empty list, not a `404`.
+
+## Access control {#access-control}
+
+Access to a data set is administered in Keycloak (or the directory behind it), not in
+DataHub. A grant is membership of an **organization group**, scoped to one organization:
+
+| Group | Grants |
+| --- | --- |
+| `/datasets//read` | Read everything in that data set, and in every data set beneath it. |
+| `/datasets//write` | Write, with the same inheritance. |
+| `/datasets/*/read` | Read every data set. |
+| `/datasets/*/write` | Write every data set. |
+
+Read and write are independent: a write grant does not imply read, the wildcard included.
+The `DATAHUB_ADMIN` realm role grants read and write to everything (an operator escape
+hatch). Entities outside any data set follow the wildcard too: reading them needs
+`/datasets/*/read`, writing or creating them needs `/datasets/*/write` (or admin).
+
+Two consequences worth knowing when you code against this:
+
+- **A missing grant is a `403`** with an `application/problem+json` body naming the
+ `dataSetId` and the `permission` (read or write) you lack. List, filter and search
+ endpoints never 403 on grants: rows in data sets you cannot read are silently
+ omitted instead.
+- **Relationship reads hide rather than refuse too.** Reading a relationship needs read
+ on both endpoints' data sets. One you may not read is a `404` from `GET /edges/{id}`
+ and omitted from `/edges/byids`, as if it did not exist.
+- **Managing a data set itself is stricter.** Creating, updating or deleting a data set
+ (as opposed to the data in it) requires the `/datasets/*/write` grant or `DATAHUB_ADMIN`.
+ Grants on individual data sets are never enough, deliberately: a data set is the unit
+ access is granted on, so renaming or re-parenting one changes what existing grants
+ cover.
+- **The rule follows the node, not the endpoint.** A `DATASET`- or `POLICY`-labelled
+ node reached through `/resources` answers the same way. The `403` detail spells this
+ out.
+
+The API reads grants from the identity provider's UserInfo endpoint, not from the token, and
+caches them: the grant cache refreshes after 45 seconds, behind a 10-second in-process cache.
+A changed grant therefore takes effect within about a minute, without a new token.
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Create | `datasets().create` | `datasets.create` | `datasets.create` |
+| Get by numeric id | `datasets().getById` | `datasets.get_by_id` | `datasets.get_by_id` |
+| Look up by id / external id | `datasets().byIds` | `datasets.by_ids` | `datasets.by_ids` |
+| List (`GET /datasets`) | `datasets().list(limit)` | `datasets.list(limit=None)` | `datasets.list(limit)` |
+| Filter | `datasets().filter` | `datasets.filter` | `datasets.filter` |
+| Search | `datasets().search` | `datasets.search(query, filter, limit)` | `datasets.search` / `search_by_query` |
+| Update | `datasets().update` | `datasets.update` (`DatasetUpdate`) | `datasets.update` (`DatasetUpdate`) |
+| Delete | `datasets().delete` | `datasets.delete` | `datasets.delete` |
+| Policies (`GET /datasets/policies`) | `datasets().policies` | `datasets.policies` | `datasets.policies` |
+
+`policies()` returns every policy in the tenant that a data set can be associated with, as
+graph resources. It is the dataset-facing view of the catalogue the
+[policy service](./policies) manages, so use it to offer "which policy" when creating or
+re-pointing a data set:
+
+```java
+DataWrapper choices = client.datasets().policies();
+```
diff --git a/versioned_docs/version-1.0/reference/edges.md b/versioned_docs/version-1.0/reference/edges.md
new file mode 100644
index 0000000..f9d6c8e
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/edges.md
@@ -0,0 +1,346 @@
+---
+sidebar_position: 10
+title: Edges
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Edges
+
+The relationships between resources, as objects in their own right. An edge is **directional**
+(`from` → `to`), **typed** by a relationship type, and **unique** per pair and type, two
+resources can be connected many ways, but only once each way.
+
+Most edges are born with their nodes: `POST /resources/create` takes `nodes` and `relations`
+together and writes them in one transaction. The `/edges` endpoints are for everything after
+that, linking resources that already exist, reading an edge back, cutting one without
+touching its endpoints, and managing the relationship-type catalogue.
+[Create resources and relations →](./resources#create-resources-and-relations)
+
+## The edge object {#body}
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `id` | number | Server-assigned. |
+| `start` | number | Id of the `from` node. |
+| `end` | number | Id of the `to` node. |
+| `type` | string | The relationship type name, upper-cased (`CONTAINS`, `FLOWS_TO`). |
+| `relationshipTypeId` | number | The type's id in the [catalogue](#types). |
+| `description` | string | Prose. |
+| `metadata` | map<string, string> | Flat key/value. |
+
+You write `fromExternalId`/`toExternalId` and read `start`/`end`: the write side speaks in
+your identifiers, the read side in the graph's.
+
+## Create {#create}
+
+`POST /edges/create` links resources that **already exist**. Both ends are resolved before
+anything is written; an end that isn't there is an error, not an implicit create.
+
+```http
+POST /edges/create
+{
+ "items": [
+ {
+ "fromExternalId": "plant_oslo",
+ "toExternalId": "pump_1",
+ "relationshipType": "contains",
+ "description": "Feeds the east wing",
+ "metadata": { "work_order": "wo-sap-12344" }
+ }
+ ]
+}
+```
+
+Name each end by external id (`fromExternalId`/`toExternalId`) or by numeric id
+(`fromId`/`toId`), and the relation by `relationshipType` or `relationshipTypeId`. A type
+name you haven't used before is created for you, so [pre-registering types](#types) is only
+for seeding the catalogue or attaching a description.
+
+The batch is **all-or-nothing**, one relation the server won't take and none of them are
+written. Success is a `201` with the created edges under `items`.
+
+
+
+
+```java
+RelForm contains = new RelForm();
+contains.setFromExternalId("plant_oslo");
+contains.setToExternalId("pump_1");
+contains.setRelationshipType("CONTAINS");
+
+DataWrapper created = client.edges().create(List.of(contains));
+System.out.println(created.getItems().get(0).getId());
+```
+
+`setName("Flows To")` is the alternative to `setRelationshipType`: it normalises the name to
+`FLOWS_TO` before it leaves the client. See [naming](#type-names).
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+contains = intellistream_datahub_sdk.RelForm(
+ relationship_type="CONTAINS",
+ from_external_id="plant_oslo",
+ to_external_id="pump_1",
+ description="Feeds the east wing",
+)
+
+created = client.edges.create([contains])
+print(created[0].id)
+```
+
+`RelForm.by_external_ids(from_external_id, to_external_id, relationship_type)` is the short
+form when you only need the three; `RelForm.by_ids` is its numeric-id counterpart. The service
+unwraps for you, `create` hands back a plain `list[EdgeProxy]`, not a wrapper.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::relations::RelForm;
+
+let contains = RelForm::by_external_ids("plant_oslo", "pump_1", "CONTAINS");
+
+let created = api.edges.create(&vec![contains]).await?;
+println!("{:?}", created.get_items()[0].id);
+```
+
+`RelForm::by_ids(from, to, ty)` is the numeric-id counterpart.
+
+
+
+
+### When it fails {#create-errors}
+
+| Status | Means |
+| --- | --- |
+| `400` | An end doesn't exist (the message names which), the relation has no type, or it breaks one of the data set or time series rules below. |
+| `403` | You can't write one of the two resources. Both ends are checked, so linking something *into* a data set needs write access on that data set too. |
+| `409` | The two are already connected that way. `(start, end, type)` is unique, one relation per pair per type. |
+
+:::note Edges into data sets and time series are validated
+The same two rules the [graph create](./resources#create-resources-and-relations) enforces
+apply here:
+
+- A relation **to a data set** must use the `BELONGS_TO` relationship type, that is the
+ relation data set membership is built from, and anything else is a `400`.
+- A **data set → time series** edge is accepted only when the series has no data set yet, or
+ already belongs to that very data set. A series in a *different* data set is a `400`: a
+ time series has one data set.
+:::
+
+## Look up {#look-up}
+
+`GET /edges/{id}` returns a single edge; an id that doesn't exist is a `404`.
+
+`POST /edges/byids` takes several ids and answers with a **graph**: the edges under
+`relations` and the resources at both ends under `nodes`, so you don't need a follow-up call
+to resolve endpoints. Unlike the single lookup, ids that match nothing are **silently
+omitted**, compare what comes back against what you asked for.
+
+Both lookups are gated by data set grants: reading an edge requires **read access to the data
+sets of both endpoints**, mirroring the write rule, because an edge reveals both ends. A denied
+edge behaves exactly like a missing one: `byids` omits it just as it omits an unknown id, and
+the single lookup answers `404`. An edge you may not read is therefore indistinguishable from
+one that does not exist, so don't infer that two resources are unlinked from a missing edge.
+The MCP `edge_get` tool follows the same rule.
+[Data set access control →](./datasets#access-control)
+
+
+
+
+```java
+DataWrapper one = client.edges().findById(341);
+
+GraphDataWrapper many = client.edges()
+ .byIds(List.of(IdCollection.createFromId(341), IdCollection.createFromId(342)));
+
+for (Resource endpoint : many.getNodes()) {
+ System.out.println(endpoint.getExternalId());
+}
+```
+
+
+
+
+```python
+one = client.edges.get(341) # EdgeProxy, or None if missing or not readable
+
+many = client.edges.by_ids([341, 342])
+for endpoint in many.nodes:
+ print(endpoint.external_id)
+```
+
+Edges have no external id, so `by_ids` and `delete` take numeric ids, or an `EdgeProxy` you
+already hold, which is accepted anywhere an id is.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let one = api.edges.get(341).await?;
+
+let many = api.edges.by_ids(&vec![IdAndExtId::from_id(341), IdAndExtId::from_id(342)]).await?;
+for endpoint in many.nodes().unwrap_or_default() {
+ println!("{}", endpoint.external_id());
+}
+```
+
+
+
+
+## Delete {#delete}
+
+`POST /edges/delete` (or `DELETE`, the endpoint takes both) removes relationships by id and
+answers `204` with no body. The resources at each end are untouched, this is how you
+disconnect two things without losing either. [Deleting a resource](./resources#delete) is the
+heavier move: it takes every relation the resource had with it.
+
+Deletion is **idempotent**: unknown ids are silently skipped, so a successful call is not
+evidence the edge existed.
+
+It can still be refused. Cutting an edge is rejected with a `409` `would-strand` if it would
+leave a surviving resource unreachable from a root, the same connectivity rule
+[deleting a resource](./resources#delete) is checked against, and the
+[problem document](./client#problem-documents) names the resources that would be stranded in
+`blockedBy`. An edge that is the only path from a subtree to the root is exactly the one you
+cannot cut: re-attach the subtree another way first, or delete it in the same call. A `403`
+means you lack write access to a data set one of the endpoints sits in.
+
+In Python, match `e.problem_slug == "would-strand"` and read `e.problem["blockedBy"]`; in Rust,
+`e.problem().map(|p| p.blocked_by())`. The check reads the graph, which settles shortly after a
+write, so deleting an edge immediately after creating it can be wrongly allowed and strand the
+node it held up.
+
+
+
+
+```java
+client.edges().delete(List.of(IdCollection.createFromId(341)));
+```
+
+
+
+
+```python
+client.edges.delete([341])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+api.edges.delete(&vec![IdAndExtId::from_id(341)]).await?;
+```
+
+
+
+
+## Relationship types {#types}
+
+Every edge carries a type, and the types are a per-tenant catalogue: `GET /edges/types` lists
+them, `POST /edges/types/create` registers names up front. Registering is optional, a type
+is created the first time an edge uses its name, so reach for it when you want the catalogue
+seeded before anyone writes, or a `description`/`i18nCode` attached to a type.
+
+A type is `{ id, name, description, i18nCode }`.
+
+
+
+
+```java
+DataWrapper catalog = client.edges().types();
+
+RelTypeForm form = new RelTypeForm();
+form.setName("Flows To"); // normalised to FLOWS_TO by the form, client-side
+client.edges().createTypes(List.of(form));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+catalog = client.edges.types()
+
+client.edges.create_types([
+ intellistream_datahub_sdk.RelTypeForm("FLOWS_TO", description="Flow direction"),
+])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::relations::RelTypeForm;
+
+let catalog = api.edges.types().await?;
+
+api.edges.create_types(&vec![RelTypeForm::new("FLOWS_TO")]).await?;
+```
+
+
+
+
+### What the server does to a name {#type-names}
+
+The two write paths do **not** normalise the same way, which is worth knowing before you
+create a type by accident.
+
+`POST /edges/types/create` snake-upper-cases the name as it reads the request, so `Flows To`,
+`flows to` and `FLOWS_TO` all land on the one type `FLOWS_TO`.
+
+`POST /edges/create` only **upper-cases** the `relationshipType` you give it. No underscores
+are inserted, so an edge created with `"relationshipType": "Flows To"` gets the type
+`FLOWS TO`, a *different* type from `FLOWS_TO`, silently created on the spot. Lookup is on the
+normalised name, so `flows_to` and `FLOWS_TO` are the same type either way.
+
+The practical rule: write the type name the way you want it stored, `FLOWS_TO`, and the two
+paths agree. In Java, `RelForm.setName("Flows To")` snake-upper-cases client-side and lines the
+create path up with the catalogue; `setRelationshipType` passes the string through. Python and
+Rust send what you give them.
+
+A name with no letter or digit in it (blank, or symbols only) is a `400`, and names registered
+through `types/create` are capped at 128 characters.
+
+:::caution A duplicate type name takes the whole batch with it
+`POST /edges/types/create` has no find-or-create: it saves a fresh type unconditionally, so a
+name that already exists (matched case-insensitively) collides on the unique name hash and
+comes back as a **`409`** naming the conflict. You cannot use it to look up the id of a type
+you did not just create; read `GET /edges/types` for that.
+
+Every form in a batch is saved in one transaction, so a single duplicate rolls the valid new
+types back alongside it. Treat the `409` as *"nothing in this batch was created"* rather than
+*"one of these already existed"*. Creating an edge with an unknown type name does not have
+this problem, that path is a proper find-or-create.
+:::
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Create relations | `edges().create` | `edges.create` | `edges.create` |
+| Get by id | `edges().findById` | `edges.get` | `edges.get` |
+| Look up several, with endpoints | `edges().byIds` | `edges.by_ids` | `edges.by_ids` |
+| Delete | `edges().delete` | `edges.delete` | `edges.delete` |
+| List types | `edges().types` | `edges.types` | `edges.types` |
+| Create types | `edges().createTypes` | `edges.create_types` | `edges.create_types` |
+
+All three clients cover the whole endpoint surface; what differs is how much wrapping survives.
+Java and Rust hand back the `DataWrapper`/`GraphDataWrapper` the API returns, so the items come
+out of `getItems()` / `get_items()`, or `nodes()` on the `GraphDataWrapper` Rust's `by_ids`
+returns. Python unwraps: `create` and `types` return plain lists,
+`get` returns an `EdgeProxy` or `None`, and `delete` returns nothing. Python's async client
+exposes the same six methods on `AsyncDataHubClient.edges`.
+
+The [MCP server](../mcp-server#relationships) covers all of these except `byids`, as
+`edge_create`, `edge_get`, `edge_delete`, `edge_list_types` and `edge_create_type`.
diff --git a/versioned_docs/version-1.0/reference/events.md b/versioned_docs/version-1.0/reference/events.md
new file mode 100644
index 0000000..535d3d3
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/events.md
@@ -0,0 +1,951 @@
+---
+sidebar_position: 6
+title: Events
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Events
+
+Record and query operational events.
+
+:::info An event's `externalId` is a correlation key, not an identity
+This is the opposite of what it means on a resource, and it is deliberate. An event's
+external id is the **source system's key for the subject** the event is about, an order, a
+permit, a batch, so **many events share one**. "Everything that happened to `PO-4500171`"
+is one indexed lookup, and that is what makes the log an audit trail.
+
+No uniqueness is enforced, and none ever will be. Per-event identity is the event `id`
+below. Naming policies do not apply to events either; only the
+[charset floor](./external-ids#the-charset-floor) does, so `21-PT-1234` is accepted on an
+event even when a `snake_case` policy is rejecting it on resources.
+[The two contracts →](./external-ids#the-two-contracts)
+:::
+
+## The event body {#body}
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `id` | UUID string | The event's identity. Time-ordered UUID v7, see the note under [Create](#create). |
+| `externalId` | string, 3–256 | **Required.** The subject's key in the source system. Shared across events on purpose. |
+| `eventTime` | [epoch millis, or ISO-8601 with an offset, on the way in](./client#timestamps) | **Required.** When it happened at the source. Never defaulted, see [Create](#create). |
+| `type` | string, 3–128 | Top-level categorization (`alarm`, `work_order`). |
+| `subType` | string, 3–128 | Refinement of `type` (`overpressure`). |
+| `status` | string, 3–128 | Free-form lifecycle marker (`OPEN`, `acknowledged`). No state machine is enforced. |
+| `source` | string, 2–128 | The system of record the event came from (`SAP`, a historian). |
+| `description` | string | Prose. [Full-text search](#search) reads it, along with `externalId` and the metadata values. |
+| `metadata` | map<string, string> | Flat key/value. An empty key is dropped rather than rejected. |
+| `dataSetId` | number | Optional. Platform-internal events (say, anomaly detection on a series outside any data set) legitimately have none. |
+| `relatedResources` | object[] | Resources the event is about. Each entry takes an `id`, an `externalId`, or both. |
+| `createdTime` | epoch millis | Server-set. When the platform stored it. |
+| `lastUpdatedTime` | epoch millis | Server-set. |
+
+`eventTime` and `createdTime` answer different questions and routinely differ by hours: a
+gateway that was offline over a weekend backfills Monday morning, so every event it sends
+carries a weekend `eventTime` and a Monday `createdTime`. Filter on `eventTime` to ask *when
+did it happen*, on `createdTime` to ask *when did we learn about it*.
+
+:::note A timestamp is ISO-8601 or epoch milliseconds
+Milliseconds is the only epoch unit, so `1767225600000` is `2026-01-01T00:00:00Z` while the
+10-digit seconds form `1767225600` is refused rather than converted. An ISO-8601 string keeps its
+own offset and **has to carry one**: `2024-06-17T12:34:56` with no zone is rejected, not read as
+UTC. That covers `eventTime` and the `min` / `max` bounds of every time filter below, for a bare
+JSON number and a quoted string alike. [Timestamps in full →](./timeseries#write-datapoints)
+:::
+
+:::note Numeric ids cross the wire as JSON strings
+`dataSetId` and the `id` of each `relatedResources` entry serialize as `"12"`, not `12`. Ids can
+exceed the 53-bit integer a JSON number is safe for in JavaScript, and a silently rounded id
+is worse than a quoted one. The clients parse them back to integers for you; a hand-rolled
+HTTP caller should expect the quotes.
+:::
+
+:::note One list, not two parallel ones
+Supply an `id`, an `externalId`, or both. The server resolves whichever side you left out and
+returns both, so a read always gives you the pair. Sending both when they name *different*
+resources is a `400` rather than a guess about which one you meant. The field names
+`relatedResourceIds` and `relatedResourceExternalIds` are unknown to the API and are refused
+with a `400`, like any other [unknown field](./client#unknown-fields).
+:::
+
+## Create {#create}
+
+Every event must carry an **event time**, the moment it occurred at the source (sensor,
+PLC, upstream system). The SDK deliberately does *not* default it to "now": an event
+without it is rejected rather than silently mis-timestamped.
+
+Send it as epoch milliseconds or ISO-8601 with an offset. Epoch seconds is refused with a
+`422` of `type: ".../errors/invalid-timestamp"` naming the mistake, where it used to be accepted
+and stored tens of thousands of years out. [Timestamps](./client#timestamps) has both forms and
+their bounds.
+
+
+
+
+```java
+EventModel event = new EventModel();
+event.setExternalId("door_open");
+event.setType("alarm");
+event.setEventTime(ZonedDateTime.now()); // required: when the event occurred
+
+client.events().create(List.of(event));
+```
+
+
+
+
+```python
+from datetime import datetime, timezone
+import intellistream_datahub_sdk
+
+event = intellistream_datahub_sdk.Event(
+ external_id="door_open",
+ type="alarm",
+ event_time=datetime.now(timezone.utc)) # required: when the event occurred
+
+client.events.create([event])
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::events::Event;
+
+let event = Event::new(
+ "door_open".into(),
+ "alarm".into(),
+ Utc::now()); // required: when the event occurred
+api.events.create(&vec![event]).await?;
+```
+
+
+
+
+Creating is **all-or-nothing**: if one event in the batch fails validation, none are
+written. Attaching the event to resources that do not exist is a `400`, as is a
+`dataSetId` naming no data set, so a typo surfaces at write time rather than as an event
+that quietly relates to nothing.
+
+Create, update and delete each take at most **10 000 events** per request, and one event
+carries at most 10 000 characters of `description`, 256 metadata entries and 100
+`relatedResources`. Past any of those is a `400`. [Limits & quotas](./limits) has the rest,
+including the daily and lifetime ceilings on how many events an organization may hold.
+
+:::note Event ids are time-ordered UUID v7
+The ingestion paths stamp every event that has no `id` with a **UUID v7** before sending,
+`create` in the Python and Rust clients, `ingest(...)` in Java (a plain Java `create` sends
+events as-is and lets the server assign ids). The server honors a client-supplied id, which is
+what makes retries idempotent: the events table is keyed by `id` and collapses rows that share
+one, so re-sending the same event (for example after a
+[buffered](./client#durable-ingest-buffering) outage) leaves one row instead of a duplicate.
+If you set the `id` yourself, use a time-ordered UUID v7, a random v4 scatters writes across
+that key and hurts insert and query performance. The created event (with its id) is returned
+from `create`, except when [durable buffering](./client#durable-ingest-buffering) spools the
+send: the Python and Rust clients then return no events (`[]` in Python, an empty result with
+status `202` in Rust).
+:::
+
+## Look up {#lookup}
+
+Fetch a single event by its UUID, or a batch by any mix of `id` and `externalId`. Ids that
+match nothing are **silently omitted**, compare what came back against what you asked for
+if a miss matters. A batch is capped at 10 000 ids.
+
+Because an external id is a correlation key, looking one up returns **every** event filed
+under it, not one event.
+
+
+
+
+```java
+UUIDAndExternalIdCollection byUuid = new UUIDAndExternalIdCollection();
+byUuid.setId(UUID.fromString("0195f3a2-4c1b-7f9e-9c3a-1b2d4e6f8a90"));
+
+DataWrapper events = client.events().byIds(List.of(
+ byUuid,
+ UUIDAndExternalIdCollection.createFromExternalId("PO-4500171"))); // every event about this order
+```
+
+
+
+
+```python
+import uuid
+
+events = client.events.by_ids(["PO-4500171"]) # a str selects by external id
+events = client.events.by_ids([uuid.UUID("0195f3a2-4c1b-7f9e-9c3a-1b2d4e6f8a90")])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::EventIdCollection;
+
+let events = api.events
+ .by_ids(&vec![EventIdCollection::from_external_id("PO-4500171")])
+ .await?;
+```
+
+
+
+
+`GET /events/{id}` fetches one event by UUID and returns `404` when there is none, the one
+place a missing event is an error rather than an omission. Python's `events.get` turns that
+`404` into `None`; Rust's `events.get` returns it as an `Err`.
+
+## Query
+
+
+
+
+```java
+EventRetreiver retriever = new EventRetreiver();
+retriever.setLimit(50);
+retriever.getFilter().setType(List.of("alarm"));
+DataWrapper events = client.events().filter(retriever);
+```
+
+
+
+
+```python
+events = client.events.filter(type="alarm", limit=50)
+```
+
+The criteria can also be built once as an `EventFilter` and passed as `filter=`. `limit`,
+`sort_by`, `sort_order` and `cursor` are always arguments of `filter()`, never fields of the
+`EventFilter`.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::{EventFilter, EventFilterForm};
+
+let mut filter = EventFilterForm::new(
+ EventFilter { r#type: Some(vec!["alarm".into()]), ..Default::default() });
+filter.set_limit(50);
+let events = api.events.filter(&filter).await?;
+```
+
+`EventFilter` is the criteria; `EventFilterForm` is the request body that wraps them with
+`limit`, `sort`, `cursor` and `advancedFilter`.
+
+
+
+
+`limit` defaults to **1 000** and is capped at **10 000**; a zero or negative value falls back
+to the default rather than returning nothing. The Python and Rust clients always send a
+`limit`, **1 000** unless you set one, so a call that says nothing about paging gets the same
+page size as the server's default. Whatever you ask for, the result is intersected with
+the data sets your token may read, a filter can never widen access, so an empty page can
+mean "no matches" or "none you may see", and the two are not distinguished.
+
+`GET /events?limit=` is the criteria-free read, `events.list` in all three clients. It runs the
+filter with an empty body, so it returns the **oldest** `limit` events (`eventTime` ascending),
+not the newest. `limit` defaults to 1 000, and above 10 000 is a `400` rather than a clamp. It
+never returns a `nextCursor`, so it is one page and no more: for "what just happened", filter with
+`sort` on `eventTime` descending. In Python it returns a plain list rather than a `Page`.
+
+### Filtering {#filtering}
+
+Every field you supply is combined with **AND**, an event must match all of them.
+
+| Field | Matching |
+| --- | --- |
+| `type`, `subType`, `status`, `source` | Pattern match. `*` and `%` are wildcards, `_` is literal. |
+| `externalId` | Pattern match, on the same rules. Literal entries are case-sensitive (see the note below). |
+| `metadata` | Every key/value pair given must be present on the event. |
+| `dataSetId` | Events belonging to these data sets. |
+| `relatedResources` | Events attached to these resources. |
+| `eventTime`, `createdTime`, `lastUpdatedTime` | `{ "min": …, "max": … }` bounds, see the note below. |
+
+Each field above takes **either a bare value or an array**, and the entries of an array are
+combined with **OR**. That is why they are named in the singular: asking for one thing is the
+common case and reads as `"type": "alarm"`, while asking for several reads as
+`"type": ["alarm", "warning"]`. The exceptions are `metadata` and `relatedResources`, whose entries
+must **all** match; they keep plural names for exactly that reason, because adding an entry there
+narrows the result where adding a `type` widens it.
+
+There is no `id` field. An event's id is a UUID string, so use [`byids`](#lookup) to fetch by id.
+
+`dataSetId` and `relatedResources` take the same shape: each entry is `{"id": …}` **or**
+`{"externalId": …}`, and the two can be mixed in one list:
+
+```json
+{
+ "filter": {
+ "type": ["alarm", "warning"],
+ "dataSetId": [{ "id": "43" }, { "externalId": "data_set_sap" }],
+ "relatedResources": [{ "externalId": "klp_pipe_ws_a1212_dl" }]
+ },
+ "limit": 50
+}
+```
+
+:::note Literal `externalId` entries are case-sensitive here
+Unlike the resource, data set and timeseries filters, an `externalId` entry **without** a wildcard
+is matched case-sensitively: every event writer hashes the external id verbatim, so the stored key
+for `shift_report_1` is not the key for `SHIFT_REPORT_1`. An entry **with** a wildcard is matched
+case-insensitively, so `"SHIFT_REPORT_1*"` is the case-insensitive way to ask the same question.
+:::
+
+**A parent data set stands in for its children.** Naming one covers everything beneath it in the
+`BELONGS_TO` hierarchy, the same expansion access control applies to a grant.
+
+An `externalId` that names no data set contributes nothing. That can only ever narrow the
+result, a typo gives you too few events, never events you should not see.
+
+:::caution Omitting `dataSetId` and sending `[]` are opposites
+Omit the field (or send `null`) for **no data set restriction**. An explicit empty list means
+**narrow to no data sets**, which matches nothing.
+
+The distinction matters if you build the filter programmatically. Code that collects data set
+references into a list and always sets the field silently returns zero events when that list
+comes back empty. For every other filter field the same code returns the unrestricted result.
+:::
+
+:::note Every time window is inclusive at both ends
+`eventTime`, `createdTime` and `lastUpdatedTime` are all matched as `min <= t <= max`, so an
+event landing exactly on `max` is returned. Back-to-back windows that share a boundary,
+Monday to Tuesday then Tuesday to Wednesday, both return an event stamped exactly at Tuesday
+midnight. To tile windows without double-counting, set each `max` one millisecond before the
+next window's `min`; the columns are stored to the millisecond.
+:::
+
+### Advanced filters {#advanced-filters}
+
+`advancedFilter` sits alongside `filter` and takes a **boolean expression**, written the way you
+would write a `WHERE` clause. Use it when flat AND is not enough: "type is alarm **or** the source
+is SAP", or "everything except the `test_` prefix".
+
+```json
+{
+ "filter": { "type": "alarm" },
+ "advancedFilter": "source = 'SAP' OR externalId LIKE 'PO-%'",
+ "limit": 200
+}
+```
+
+`filter` and `advancedFilter` are combined with **AND**, so the example above means "an alarm,
+and additionally either from SAP or with a `PO-` external id". Either may be used without the
+other.
+
+The dialect is **PostgreSQL-flavoured**. Function names, `::` casts, `ILIKE`, `<>` beside `!=`,
+`--` and `/* */` comments and single-quoted strings with `''` escaping all behave as they do in
+Postgres. A leading `WHERE` is accepted and ignored, so pasting a clause out of a Postgres query
+usually works.
+
+:::caution `AND` binds tighter than `OR`
+As in SQL, so `a OR b AND c` means `a OR (b AND c)`, which is often not what the parentheses in
+the rest of an expression suggest. Parenthesise when you mean otherwise; nothing warns you,
+because the expression is valid either way.
+:::
+
+#### What you can filter on
+
+| Field | Type |
+| --- | --- |
+| `id` | Event id (UUID) |
+| `externalId`, `type`, `subType`, `status`, `source`, `description` | Text |
+| `dataSetId` | Number |
+| `eventTime`, `createdTime`, `lastUpdatedTime` | Timestamp |
+| `metadata['key']` | Text, always. See below. |
+
+Names are the same ones `filter` uses, and matching is case-insensitive, so `subtype` and
+`subType` both work. Anything else is rejected by name; there is no way to reach a column that is
+not on this list.
+
+#### Operators
+
+`=` `!=` `<>` `<` `<=` `>` `>=`, `LIKE`, `ILIKE`, `IN (…)`, `BETWEEN … AND …`, `IS NULL`, and
+`NOT` before any of them. Combine with `AND`, `OR`, `NOT` and parentheses.
+
+```text
+type NOT LIKE 'pump%'
+status IN ('OPEN', 'IN_PROGRESS')
+dataSetId BETWEEN 1 AND 5
+eventTime > '2026-01-01' AND eventTime < '2026-02-01'
+subType IS NOT NULL
+```
+
+Timestamps accept `2026-01-01`, `2026-01-01 12:30` and `2026-01-01T12:30:00Z`. A value that is
+not a date is refused before the query runs, rather than failing inside the database.
+
+A quoted value may not contain a tab or a newline. Those cannot be sent as query parameters, so
+they are refused with a 400 rather than producing a confusing failure further down; match around
+them with `LIKE` instead.
+
+#### Functions
+
+| Function | Meaning |
+| --- | --- |
+| `to_timestamp(x)`, `to_date(x)` | Read text as a date or timestamp |
+| `to_int(x)`, `to_number(x)` | Read text as a whole number or a decimal |
+| `to_bool(x)` | Read text as a boolean: `true/false`, `t/f`, `yes/no`, `y/n`, `on/off`, `1/0` |
+| `date_part('year' \| 'month' \| 'day', x)` | Extract a part of a date, as a number |
+| `has_key('k')` | Whether the event's metadata has this key |
+| `lower(x)`, `upper(x)`, `length(x)` | As in Postgres |
+| `now()` | Current time |
+
+`::` is shorthand for the converters: `metadata['n']::int` is the same as `to_int(metadata['n'])`.
+Cast targets are `int`, `bigint`, `integer`, `float`, `numeric`, `boolean`, `bool`, `date` and
+`timestamp`.
+
+Any other function is rejected. This is an allow-list rather than a block-list, so nothing else in
+the underlying database is reachable, whatever it is called.
+
+#### Metadata is text, and you have to say what it is
+
+Every metadata value is stored as text, so comparing one as anything else needs a converter. This
+is deliberate: inferring the type from the other side of the comparison would mean one operand
+silently changing how the other is read.
+
+```text
+metadata['count'] > 5 ✗ 400, "wrap it: to_int(metadata['count'])"
+to_int(metadata['count']) > 5 ✓
+metadata['count']::int > 5 ✓ same thing
+metadata['site'] = 'bergen' ✓ text against text needs nothing
+```
+
+:::caution A missing metadata key reads as an empty string, not null
+So `metadata['nope'] IS NULL` would never match anything, and `metadata['nope'] = ''` matches
+every event that lacks the key. Ask with `has_key('nope')` instead. As a convenience,
+`metadata['k'] IS NULL` is answered as "this key is absent", because the literal reading is
+never useful.
+
+`subType` and `status` are genuinely nullable, so `IS NULL` on those two means what it says.
+:::
+
+#### When an expression is refused
+
+A rejected expression is a **400**, and no query runs: an expression the API cannot read is never
+partly applied, which would return everything you may see while quietly ignoring what you asked
+for. The problem detail carries enough to fix it:
+
+```json
+{
+ "type": "https://intellistream.ai/errors/filter-expression",
+ "title": "Invalid filter expression",
+ "detail": "'toDate' is the ClickHouse spelling. This filter uses PostgreSQL names, so use 'to_date'.",
+ "offset": 0,
+ "length": 6,
+ "suggestion": "to_date",
+ "suggestedQuery": "to_date(metadata['t']) = '2026-01-01'",
+ "help": "Function names follow PostgreSQL; ClickHouse spellings are mapped internally."
+}
+```
+
+`offset` and `length` point at the token at fault, so an editor can underline it, and
+`suggestedQuery` is your expression with the fix already applied. `suggestedQuery` is **absent
+when the repair is ambiguous**, so treat its presence as "there is one obvious fix" rather than
+"there is a fix". Common corrections are named outright rather than guessed at: database
+spellings such as `toDate` or `mapContains`, and physical column names such as `sub_type` or
+`event_time`, both map back to what this API calls them.
+
+#### Limits
+
+An expression may be 4 096 characters, nest 20 levels, hold 200 terms and call 20 functions.
+Subqueries, `SELECT`, `HAVING`, `UNION` and `JOIN` are not supported; those keywords are
+recognised so that using one tells you the feature is missing rather than that your column name
+is unknown.
+
+#### Moving from the old nested form
+
+Before this release `advancedFilter` took a nested `and`/`or`/`not` object with `equals`,
+`prefix` and `in` leaves. That form is now refused with a 400.
+
+| Old | Now |
+| --- | --- |
+| `{"equals": {"property": ["type"], "value": "alarm"}}` | `type = 'alarm'` |
+| `{"prefix": {"property": ["externalId"], "value": "PO-"}}` | `externalId LIKE 'PO-%'` |
+| `{"in": {"property": ["status"], "values": ["a","b"]}}` | `status IN ('a', 'b')` |
+| `{"and": [x, y]}` | `x AND y` |
+| `{"or": [x, y]}` | `x OR y` |
+| `{"not": x}` | `NOT x` |
+| `range`, `containsAny`, `containsAll`, `isSet` (SDK-side only) | See below |
+
+The last row is worth knowing if you used the Rust SDK: `range`, `containsAny` and `containsAll`
+had no server-side implementation and returned a 500, and `isSet` reached a code path that could
+not read the property it was given. They never worked, so there is nothing to migrate, and the
+equivalents are now `BETWEEN`, `IN`, repeated `AND`, and `has_key` or `IS NOT NULL`.
+
+Two other differences: values used to be compared as strings whatever the field, so `dataSetId`
+matched `"43"` rather than `43`, and the old `property` was a list whose first entry alone was
+read, with no way to reach an individual metadata key. Both are gone: types are real, and
+`metadata['key']` is addressable.
+
+### Ordering and paging {#paging}
+
+Events come back **`eventTime` ascending** unless you say otherwise, that is the order the
+cursor pages in, so paging does not change the order underneath you. Ask for another with
+`sort`, over `eventTime`, `createdTime`, `lastUpdatedTime`, `externalId`, `type`, `subType`,
+`status`, `source` or `dataSetId`:
+
+```json
+{ "filter": { "type": "alarm" },
+ "sort": { "property": ["eventTime"], "order": "desc" },
+ "limit": 200 }
+```
+
+Only the **first** sortable `property` is used, and `id` is appended behind it, a sort column alone is
+not a position unless it is unique, and a page boundary inside a run of equal values repeats or
+drops exactly those rows. A property that is not sortable is ignored rather than rejected, and
+any `order` that is not `desc` (compared case-insensitively) sorts ascending, a malformed sort degrades to the
+default instead of silently reversing your results. Null values sort last ascending, first
+descending.
+
+To walk past the first page, echo back the `nextCursor` the response carried:
+
+```json
+{ "filter": { "type": "alarm" },
+ "sort": { "property": ["eventTime"], "order": "desc" },
+ "cursor": "ZXZlbnRUaW1lfGRlc2N8MDE5NWYzYTItNGMxYi03ZjllLTljM2EtMWIyZDRlNmY4YTkwfHYxNzU0NDc2NTIyMTA0",
+ "limit": 200 }
+```
+
+The cursor is **opaque**, base64 of an encoding carrying the sort, the boundary value and the
+id, with no version tag, so do not build or parse one. A cursor that does not decode is refused with a
+`400` of `type: ".../errors/malformed-cursor"` rather than guessed at: half a position would
+silently skip or repeat the rows around the boundary.
+
+Send it with the **same** `sort` that produced it: a cursor is a position in one particular
+order. Continuing it under another is refused with the same `400`, which names both sorts.
+
+Prefer this to counting pages. Events are stored partitioned by event time, so resuming from
+a position lets whole partitions be skipped, where an offset re-reads everything ahead of it
+and gets slower the further you page. Page 400 costs what page 1 costs. The trade is that
+there is no random access: you walk forward from where you were and cannot jump to page 7.
+`nextCursor` is absent on a short page, so "keep going while it is present" is the whole loop,
+and since a full page may still be the last, a complete walk ends with one empty request.
+
+All three clients read the cursor off the response envelope rather than off the last event:
+
+
+
+
+```java
+DataWrapper page = client.events().filter(retriever);
+if (page.getNextCursor() != null) {
+ retriever.setCursor(page.getNextCursor()); // keep the same sort
+}
+```
+
+
+
+
+```python
+page = client.events.filter(type="alarm", limit=50)
+if page.next_cursor is not None:
+ page = client.events.filter(type="alarm", limit=50,
+ cursor=page.next_cursor) # keep the same sort_by / sort_order
+```
+
+
+
+
+```rust
+let page = api.events.filter(&filter).await?;
+if let Some(cursor) = page.next_cursor() {
+ filter.set_cursor(cursor); // keep the same sort
+}
+```
+
+
+
+
+## Policy findings {#policy-findings}
+
+A **policy finding**, a naming-policy violation that was allowed through and recorded for a
+steward, is an ordinary event. There is no findings endpoint and no findings client: they are
+written to the event store like anything else, so everything on this page already works on
+them.
+
+The encoding is a wire contract, so you can read findings without a policy-aware client:
+
+| Field | Holds |
+| --- | --- |
+| `type` | Always `policy_finding`. Matched exactly, never by prefix, this is the one filter separating findings from the tenant's real events. |
+| `subType` | Which policy fired, by its external id. |
+| `source` | `datahub_policy_`, truncated to 128 characters. |
+| `externalId` | `policy_finding__`, the correlation key every event in one finding's lifecycle shares. |
+| `status` | `OPEN` or `RESOLVED`, what *this event* asserts, not the finding's current state. |
+| `description` | What is wrong, in words. |
+| `relatedResources` | The entity the finding is about, by node id. |
+| `dataSetId` | That entity's data set. |
+| `metadata` | `offendingValue`, `suggestion` (when one could be derived), `raisedBy`. |
+
+Note the external id is keyed on the entity's **node id**, not its external id. The external
+id is the offending value here, the thing a steward is most likely to change, and keying on
+it would mean renaming a resource silently abandoned its finding and started a second stream.
+
+### Fetch the queue
+
+Filter on the type, ascending by `eventTime`, and page with [`cursor`](#paging):
+
+
+
+
+```java
+EventRetreiver retriever = new EventRetreiver();
+retriever.getFilter().setType(List.of("policy_finding"));
+retriever.getFilter().setSubType(List.of("naming_snake_case")); // one policy; omit for all
+retriever.setLimit(200);
+retriever.getSort().setProperty(List.of("eventTime"));
+retriever.getSort().setOrder("asc");
+
+DataWrapper findings = client.events().filter(retriever);
+```
+
+
+
+
+```python
+findings = client.events.filter(
+ type="policy_finding",
+ sub_type="naming_snake_case", # one policy; omit for all
+ limit=200)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::{EventFilter, EventFilterForm};
+
+let filter = EventFilterForm::default()
+ .set_filter(EventFilter {
+ r#type: Some(vec!["policy_finding".into()]),
+ sub_type: Some(vec!["naming_snake_case".into()]), // one policy; omit for all
+ ..Default::default()
+ })
+ .set_limit(200)
+ .build();
+
+let findings = api.events.filter(&filter).await?;
+```
+
+
+
+
+### Fold the stream
+
+A finding's current state is **not stored**, you derive it. Nothing is ever updated in place:
+raising appends an `OPEN`, resolving appends a `RESOLVED` carrying the same `externalId`. Group
+by external id, order by `eventTime` ascending, and the last event wins:
+
+
+
+
+```java
+Map current = new HashMap<>();
+findings.getItems().stream()
+ .sorted(Comparator.comparing(EventModel::getEventTime))
+ .forEach(e -> current.put(e.getExternalId(), e)); // last write wins
+
+List open = current.values().stream()
+ .filter(e -> "OPEN".equals(e.getStatus()))
+ .toList();
+```
+
+
+
+
+```python
+current = {}
+for e in sorted(findings, key=lambda e: e.event_time):
+ current[e.external_id] = e # last write wins
+
+open_findings = [e for e in current.values() if e.status == "OPEN"]
+```
+
+
+
+
+```rust
+use std::collections::HashMap;
+
+let mut events = findings.get_items().clone();
+events.sort_by_key(|e| e.event_time);
+
+let mut current: HashMap = HashMap::new();
+for e in events {
+ current.insert(e.external_id.clone(), e); // last write wins
+}
+
+let open: Vec<_> = current.values().filter(|e| e.status.as_deref() == Some("OPEN")).collect();
+```
+
+
+
+
+:::caution Do not filter on `status`
+A stored `OPEN` event means *this was raised*, not *this is outstanding*. Filtering the query
+on `status: "OPEN"` returns the raise of every finding that has since been resolved, the
+resolve is a separate, later event, and narrowing the query hides it. Open-ness is a
+conclusion drawn from the stream, not a fact the store holds, so fetch and fold.
+
+Order ascending for the same reason: replay out of order and a stale `OPEN` overwrites the
+`RESOLVED` that followed it. Keep folding across pages too, a `RESOLVED` on page 3 closes a
+finding whose `OPEN` arrived on page 1.
+:::
+
+:::caution A fold needs the *whole* stream, so a truncated page lies
+Folding is only correct if every event sharing an external id is in front of you. Get one
+page of a queue larger than your `limit` and an `OPEN` can arrive without the `RESOLVED` that
+closed it, the fold then reports a resolved finding as outstanding. It is a wrong answer,
+not an error, and nothing in the response marks it as partial.
+
+Page until short, a full page is a signal that there is more, never that you have it all.
+Narrowing by `subType` and `dataSetId` keeps the walk cheap, but it is paging, not narrowing,
+that makes the fold correct.
+:::
+
+Raising is idempotent: a raise event's id is derived from what it asserts, so re-evaluating an
+entity whose external id has not changed collapses onto the raise already stored. An entity
+written a thousand times contributes one `OPEN`, not a thousand. A raise for a *different*
+non-conforming value is a new fact and is appended, which is also all "reopening" is.
+
+For the steward's side of this, how to resolve a finding, what resolving means, and why
+findings are raised for resources but never for events, see
+[Findings](./external-ids#findings).
+
+## Full-text search {#search}
+
+`POST /events/search` is a case-insensitive **substring** match over `externalId`, `description`
+and the metadata values, returned newest first by `eventTime`.
+
+It is not word-aware, and it does not rank: events live in ClickHouse and have no full-text index,
+so `pump` finds `pump` and `pumps` but not `pumping`, and rows come back newest first rather than
+best-match first. The three node-backed searches do rank by relevance. It accepts the same `filter` block as `POST /events/filter`, which
+narrows the phrase's hits:
+
+```json
+{
+ "search": { "query": "overpressure" },
+ "filter": { "type": ["alarm"], "eventTime": { "min": 1745241600000 } },
+ "limit": 50
+}
+```
+
+`dataSetId` in that block covers everything beneath the data sets you name, exactly as it does on
+`/events/filter`.
+
+`query` must be 3 to 140 characters. There is no character restriction beyond that: punctuation,
+underscores and non-Latin scripts are all accepted, so an externalId or a Cyrillic asset name can
+be searched for directly. `limit` here is capped at **1 000**, lower than the 10 000 of `filter`.
+
+Reach for `filter` instead whenever the question is structured (a time range, an exact type, a
+related resource). It is faster and its results are predictable.
+
+## Distinct values {#distinct-values}
+
+Two families of endpoint answer "what values actually occur?", the material for a filter
+drop-down or a type-ahead, without scanning events yourself. Both are restricted to the data
+sets your token may read, so a UI built on them cannot offer a facet the user could not then
+query.
+
+| Endpoint | Returns |
+| --- | --- |
+| `GET /events/list/types` | Every distinct `type`, alphabetically. |
+| `GET /events/list/sub-types` | Every distinct `subType`. |
+| `GET /events/list/statuses` | Every distinct `status`. |
+| `GET /events/list/sources` | Every distinct `source`. |
+| `GET /events/search/type?q=` | Distinct `type` values containing `q`, case-insensitive. |
+| `GET /events/search/sub-type?q=` | The same for `subType`. |
+| `GET /events/search/status?q=` | The same for `status`. |
+| `GET /events/search/source?q=` | The same for `source`. |
+
+Both families take `limit` (default 1 000, clamped to 1–10 000). The `search/*` form requires
+`q` and returns `400` without it.
+
+The values come from small tables the write path maintains, not from a scan of the events, which
+is what makes them cheap. They are **eventually consistent** with the events: a new value appears
+once the write path has recorded it, and a value no event carries any more lingers until a
+periodic reconcile. Good for populating a picker, not proof that an event with that value exists
+right now.
+
+```
+GET /events/search/type?q=alarm&limit=20
+→ { "items": ["alarm", "alarm_cleared", "pre_alarm"] }
+```
+
+## Count {#count}
+
+`GET /events/count` returns `{ "count": 148392 }` for the tenant. It is a single cheap query,
+and it takes **no filters**, for a filtered count, run `POST /events/filter` with the `limit`
+you care about and measure the page.
+
+## Update {#update}
+
+`POST /events/update` changes fields on events that already exist. Identify each one by UUID
+`id` or by `externalId`, and name only the fields you want changed, anything you leave out
+keeps its current value.
+
+Each field is an object carrying a verb rather than a bare value, which is what lets "clear
+this" be expressed distinctly from "leave it alone":
+
+| Verb | Applies to | Effect |
+| --- | --- | --- |
+| `set` | every field | Replace the value. |
+| `setNull: true` | nullable fields only | Clear the value. `type` is not nullable, so asking to clear it is a `400`. |
+| `add` | `metadata`, `relatedResources` | Merge entries in, keeping the rest. |
+| `remove` | the same collections | Take entries out, keeping the rest. A `relatedResources` entry matches on either side, so you can remove by `id` or by `externalId` whichever you have. |
+
+```json
+{
+ "items": [
+ {
+ "externalId": "alarm_pipe_overpressure_2026_04_22_14_30",
+ "update": {
+ "status": { "set": "acknowledged" },
+ "metadata": { "add": { "acked_by": "olav" } }
+ }
+ }
+ ]
+}
+```
+
+Updatable fields are `description`, `type`, `subType`, `status`, `source`, `dataSetId`,
+`metadata` and `relatedResources`. Sending both `set` and `setNull` for one field is a
+`400`, the request is contradictory, so it is refused rather than resolved by precedence.
+
+`externalId` and `eventTime` are **fixed at creation** and are not update fields at all: an
+update naming either is a `400` that names the field, the same answer as for any field the
+form does not have. They are fixed for different reasons. The store partitions events by
+their time, and a row cannot move between partitions. The `externalId` is the correlation key
+that groups one subject's history (see [External ids](./external-ids)), so renaming one
+event would tear it out of its own trail. An event recorded against the wrong moment or the
+wrong key is deleted and written again, or corrected by a follow-up event, which the caution
+below recommends anyway.
+
+`setNull` is refused on `type`. Clearing it would leave the event unreadable by any client
+that models `type` as required, so the write is refused rather than the read failing later.
+`dataSetId` is the one field here that genuinely is nullable: `setNull` detaches the event
+from its data set, and naming a `dataSetId` that no data set has is a `400` rather than a
+stored dangling reference.
+
+:::caution Prefer a follow-up event to mutating one
+An event update runs a replace-and-cleanup on the stored record. While it is in flight, a
+concurrent read of the same event can briefly return the pre-update version *or* see it
+twice. Where the record matters for audit, write a new event that corrects the old one
+instead: that is what an append-only log is for, and it keeps the correction itself visible.
+
+`status` is the honourable exception, acknowledging an alarm in place is what the field is
+there for.
+:::
+
+## High-throughput ingestion
+
+
+
+
+`ingest` chunks, parallelises and retries events the same way as datapoints, returning
+the same [`IngestResult`](./timeseries.md#ingestresult) tuned with the same
+[`IngestOptions`](./timeseries.md#ingestoptions):
+
+```java
+IngestResult result = client.events().ingest(events,
+ IngestOptions.builder().batchSize(1_000).parallelism(8).build());
+```
+
+
+
+
+`create` accepts a whole batch:
+
+```python
+client.events.create(events) # list[Event]
+```
+
+
+
+
+`create` accepts a whole batch:
+
+```rust
+api.events.create(&events).await?; // Vec
+```
+
+
+
+
+## Delete
+
+Deletes are **idempotent**: removing an event that is already gone returns `200` and changes
+nothing, so a retried delete needs no bookkeeping.
+
+Remember that an external id names a *subject*, not an event. Deleting by external id removes
+**every event filed under it**, which is rarely what you want for a single mistaken record,
+delete that one by its UUID.
+
+:::caution A `200` means "accepted", not "gone"
+The delete is published to the ingestion pipeline and marked in the backend without waiting
+for the removal to land, a background job does the actual work. Until it has run, the event
+**can still come back from `filter` and `byids`**.
+
+So a test that deletes an event and immediately asserts it is gone will flake, and so will a
+UI that re-queries straight on the back of a delete. Poll until the event disappears rather
+than reading once, and treat its absence, not the `200`, as the signal. The same eventual
+consistency applies in the other direction: an event is not necessarily queryable the instant
+`create` returns.
+
+Once the background job has run the removal is permanent, and anything referencing the event
+by id stops resolving.
+:::
+
+
+
+
+```java
+client.events().delete(List.of(UUIDAndExternalIdCollection.createFromExternalId("door_open")));
+```
+
+
+
+
+```python
+client.events.delete(["door_open"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::events::EventIdCollection;
+
+api.events.delete(&vec![EventIdCollection::from_external_id("door_open")]).await?;
+```
+
+
+
+
+## What each client covers {#client-coverage}
+
+The three clients cover every event endpoint, the facet endpoints included.
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Create | `events().create` / `ingest` | `events.create` | `events.create` |
+| Get by id | `events().getById` | `events.get` | `events.get` |
+| List, oldest first (`GET /events`) | `events().list(limit)` | `events.list(limit=None)` | `events.list(limit)` |
+| Look up by id / external id | `events().byIds` | `events.by_ids` | `events.by_ids` |
+| Filter | `events().filter` | `events.filter` | `events.filter` |
+| Filter with `sort` | `EventRetreiver.sort` | `sort_by` / `sort_order` | `set_sort` |
+| Filter with paging | `EventRetreiver.cursor` | `cursor` | `set_cursor` |
+| [Advanced filter](#advanced-filters) | `EventRetreiver.setAdvancedFilter` | `advanced_filter` | `set_advanced_filter` |
+| Update | `events().update` | `events.update` | `events.update` |
+| Full-text search | `events().search` | `events.search` | `events.search` |
+| Count | `events().count` | `events.count` | `events.count` |
+| Delete | `events().delete` | `events.delete` | `events.delete` |
+| Distinct values | `events().listTypes` etc. | `events.list_types` etc. | `events.list_types` etc. |
+
+All three carry the same four pairs: `list_types` / `search_types` and the same for sub-types,
+statuses and sources (`listTypes` / `searchTypes` … in Java).
+
+Take the paging value from the response envelope, `getNextCursor()` in Java, `page.next_cursor`
+in Python, `page.next_cursor()` in Rust, and send it back unchanged. It is opaque; an
+undecodable cursor is refused with a `400`.
diff --git a/versioned_docs/version-1.0/reference/external-ids.md b/versioned_docs/version-1.0/reference/external-ids.md
new file mode 100644
index 0000000..36ca715
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/external-ids.md
@@ -0,0 +1,389 @@
+---
+sidebar_position: 2
+title: External ids & naming
+---
+
+# External ids & naming
+
+`externalId` is how your systems refer to something in DataHub without holding its numeric
+id. Two things about it repay reading once, because both are easy to get wrong from the
+outside: it **means different things on a resource and on an event**, and **two independent
+layers** decide whether a given value is accepted.
+
+## The two contracts
+
+Same field name, different jobs.
+
+| On | What `externalId` is | Unique? |
+| --- | --- | --- |
+| **Resources, data sets, time series** | The **identity** of one thing, your key for it, and what every integration matches on. | Yes, per tenant, compared without case |
+| **Events** | A **correlation key**, the source system's key for the *subject* the event is about. | **No, and deliberately never** |
+
+An order that is created, amended and then shipped produces three events all carrying
+`PO-4500171`. That is the point: the order's history is "every event with this external id,
+in time order", and the log is an audit trail because of it. Per-event identity comes from
+the platform's own event `id` (a time-ordered UUID v7, see [Events](./events)), never from
+the external id.
+
+:::danger Do not synthesise per-event external ids
+Making event external ids unique, `PO-4500171-1`, `PO-4500171-2`, throws away the only
+cheap way to ask for a subject's history, and pushes you toward updating events in place,
+which destroys the append-only record. If you need to de-duplicate a redelivered snapshot,
+set the event `id` yourself; retries then collapse to one row.
+
+The Python client cannot do that yet: `Event(...)` takes no `id` and its `id` is read-only, so
+each `create` gets a fresh one. In Rust, `Event.id` is a public field.
+:::
+
+## The two layers
+
+### Layer 1: the charset floor {#the-charset-floor}
+
+Platform-wide, not configurable, and it applies **everywhere an external id is accepted,
+events included**.
+
+| | |
+| --- | --- |
+| Accepted | `A-Z` `a-z` `0-9` and the separators `. _ : + = -` |
+| Rejected | whitespace, `/`, control characters, anything else |
+| Length | 3 to 256 characters |
+
+Whatever passes is stored **byte for byte as you sent it**. Nothing is lower-cased and no
+separator is rewritten, so a plant tag stays a plant tag:
+
+```text
+COM-99-PT-1034 ok, stored as sent
+=K1-M3+B02 ok — an IEC 81346 designation keeps its aspect prefixes
+21_PT_1234 ok
+Pump-A 01 rejected: contains a space
+line/3 rejected: contains a forward slash
+P1 rejected: shorter than 3 characters
+```
+
+Byte-for-byte storage is what lets you join on identifiers you already maintain. The
+historian, the maintenance system and DataHub hold the same string, so a match is a string
+comparison instead of a normalisation each integration has to reimplement identically.
+
+### Layer 2: the naming policy {#the-naming-policy}
+
+A convention an administrator configures on top of the floor. It applies to **every node type
+whose external id is an identity**: resources and assets, data sets, time series, functions and
+policies. Enforcement sits on the one write path they share, so it cannot cover one create
+endpoint and miss another.
+
+| Preset | Accepts |
+| --- | --- |
+| `qualified_tag` | The charset floor, plus at least 3 separator-delimited alphanumeric runs. `COM-99-PT-1034` and `=K1-M3+B02` pass; `pump-1234` and `P-101` do not. **The shipped default, in `warn` mode.** |
+| `verbatim_tag` | The charset floor and nothing more. |
+| `snake_case` | `[a-z0-9_]+` only. |
+| `pattern` | An administrator-supplied regular expression. |
+
+Each policy runs in one of two modes: **reject**, where the request fails with `400` and
+nothing is written, or **warn**, where the request succeeds and a finding is recorded for a
+data steward. A **near-duplicate guard** runs regardless of the preset and is described
+[below](#the-near-duplicate-guard).
+
+**Out of the box neither rule refuses anything.** Both ship in `warn` mode, so a two-part id
+such as `pump-1234` and a near duplicate of an existing id are each written and flagged. A
+client that never inspects the `warnings` array will therefore never notice the policy at all,
+which is the case worth designing for: on an import, treat a non-empty `warnings` array as work
+to do rather than noise.
+
+Resolution is tenant-first with a per-data-set override: a data set's own naming policy wins
+if it has one, otherwise the tenant's. The override **replaces** the tenant policy rather
+than merging with it.
+
+:::caution The naming policy never applies to events
+Only the charset floor does. An event external id is not a name someone chose, it is the
+source system's key for the subject. The platform does not impose a convention on data you do
+not own, and the policy's other rules would be meaningless there anyway, since events
+deliberately share external ids.
+
+With a `snake_case` policy active and set to reject, `client.resources().create(...)` with
+`21-PT-1234` fails and `client.events().create(...)` with `21-PT-1234` succeeds. That is
+correct behaviour, not a gap.
+:::
+
+## Case: compared without it, stored with it
+
+Uniqueness on resources, data sets and time series ignores case. Creating `com-99-pt-1034`
+when `COM-99-PT-1034` already exists is a duplicate and comes back as **`409`**, with a
+message naming the id it collides with, it is the ordinary "this external id already
+exists" path, not a naming-policy rejection.
+
+Lookups ignore case for the same reason, so uniqueness and lookup agree:
+
+```text
+GET /resources/VAL-01 ─┐
+ ├─ the same resource
+GET /resources/val-01 ─┘
+```
+
+Storage is still verbatim, so **what you read back is byte-identical to what you sent**, a
+lookup by `val-01` returns an entity whose `externalId` is `VAL-01`. Compare external ids
+case-insensitively in your own code if you compare them at all.
+
+## Batches are all-or-nothing
+
+Every endpoint that takes a list validates the **whole batch before writing anything**. One
+bad item in 500 creates nothing, and the error names every offending item rather than
+stopping at the first, so a rejected import is fixed in one pass.
+
+Items in the same request are also compared against each other, not just against stored data
+sending `PUMP-01` and `pump-01` together is rejected.
+
+## Warnings on the response
+
+When a policy is in warn mode, the write succeeds and the envelope carries a `warnings` array
+alongside `items`. It is **absent when empty**, so a client that has never looked at it sees
+no change:
+
+```json
+{
+ "items": [ ... ],
+ "warnings": [
+ {
+ "index": 3,
+ "externalId": "Pump-A 01",
+ "policy": "naming_snake_case",
+ "message": "Does not match naming policy 'snake_case'.",
+ "suggestion": "pump_a_01"
+ }
+ ]
+}
+```
+
+`index` is the item's position in the request you sent. Each warning is also persisted as a
+[finding](#findings) for a data steward, so ignoring the array does not make the problem go
+away quietly.
+
+## Rejections
+
+A naming-policy failure is an [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) problem
+document:
+
+```json
+{
+ "type": "https://intellistream.ai/errors/naming-policy",
+ "title": "Bad Request",
+ "status": 400,
+ "detail": "2 of 500 external ids violate naming policy 'snake_case'. Nothing was created.",
+ "violations": [
+ {
+ "index": 3,
+ "externalId": "Pump-A 01",
+ "policy": "naming_snake_case",
+ "reason": "contains a space",
+ "suggestion": "pump_a_01"
+ }
+ ]
+}
+```
+
+It is **`400`, not `403`**, malformed input, not an access decision. `detail` says outright
+that nothing was created, because that is the first thing you need to know before retrying.
+
+The body reaches you through the ordinary error path in each client: `DatahubApiException`
+in Java (`statusCode()` and `body()`), `DataHubException` in Python, `ResponseError` in Rust.
+No client has a typed accessor for `violations`, so read it as JSON: from `body()` in Java,
+`(e.problem or {}).get("violations", [])` in Python, and
+`e.problem().and_then(|p| p.extensions.get("violations").cloned())` in Rust. See
+[Results & errors](./client#results--errors).
+
+| Status | Means |
+| --- | --- |
+| `400` | Charset floor or naming policy rejected one or more ids. Nothing was written. |
+| `409` | The external id already exists, compared without case. |
+
+## The near-duplicate guard
+
+Always on, independent of the preset, and scoped to the **whole tenant** rather than one data
+set. It fires when a new id would land beside an existing one that differs only by case or by
+which separator it uses:
+
+```text
+existing: pump_a_01
+incoming: pump-a-01 → near duplicate
+incoming: PUMP.A.01 → near duplicate
+incoming: pump_a_02 → fine
+```
+
+Its mode is `warn` by default and can be set to `reject`, which is the setting to use once you
+are confident the two forms are never both wanted. Tenant scope is deliberate:
+uniqueness is already tenant-wide, and two spellings of one tag cannot both be *the*
+identifier for one asset even when they arrive through different data sets.
+
+## Preflight: check before you write
+
+```http
+POST /policies/naming/check
+```
+
+Takes candidate external ids and an optional data set id, and returns the same findings the
+write path would produce, **without writing anything**. It runs the same evaluator, so
+there is no second set of rules to keep in step.
+
+```json
+{
+ "externalIds": ["Pump-A 01", "pump_a_02"],
+ "names": ["Pump A 01", "Pump A 02"],
+ "dataSetId": 4471
+}
+```
+
+`names` is optional and pairs with `externalIds` **by position**. Supply it when you have it:
+suggestions are derived from the name first, so an entity called `Valve pressure sensors`
+gets offered `valve_pressure_sensors`, where deriving from a broken id could only manage
+`vps`. Either omit `names` entirely or send exactly as many as there are external ids, a
+partial list is rejected rather than paired up wrongly.
+
+Two things it is good for:
+
+- validating an id as a user types it, rather than on submit;
+- answering *"what would this policy do to the ids I already have?"* before an administrator
+ turns it on.
+
+### Suggestions {#suggestions}
+
+Every warning and every rejection carries a `suggestion` where one can be derived, a
+conforming external id you can offer as a one-click fix. It is **offered, never applied**:
+the server never rewrites an external id.
+
+Two guarantees make it safe to wire straight into a form:
+
+- **A suggestion always satisfies the policy it is offered for.** It is checked against the
+ charset floor, the length bounds and the active preset before being returned, so applying
+ one cannot bounce back with a second rejection.
+- **A suggestion is never an id that is already taken**: neither one that is stored nor one
+ claimed by an earlier item in the same batch.
+
+When nothing can be derived that satisfies both, `suggestion` is absent. That is deliberate:
+an honest omission beats a confident wrong answer. In particular a near-duplicate rejection
+usually has no suggestion, because every variant of the same name folds to the same taken
+value, the `reason` names the existing id instead, since the likeliest fix is to use it.
+
+## Findings {#findings}
+
+Every warning is recorded, so warn means *allowed and in the steward's queue*, not *allowed
+and forgotten*.
+
+**A finding is an event**, and more precisely, a *stream* of events. There is no findings
+endpoint: findings are stored, filtered and resolved as ordinary events, so everything you
+already use for events works on them.
+
+Nothing is ever updated in place. Raising a finding appends an `OPEN` event; resolving it
+appends a `RESOLVED` event carrying the **same `externalId`**. A finding's current state is not
+stored, you derive it: take every event sharing that external id, order by `eventTime`
+ascending, and the last one wins.
+
+Read the queue by filtering events on the finding type:
+
+```http
+POST /events/filter
+{
+ "filter": {
+ "type": "policy_finding",
+ "subType": "naming_snake_case", // which policy fired; omit for all
+ "dataSetId": [{ "id": 42 }]
+ },
+ "sort": { "property": ["eventTime"], "order": "asc" },
+ "limit": 200
+}
+```
+
+:::caution Do not filter on `status`
+A stored `OPEN` event means *this was raised*, not *this is outstanding*. Filtering the query on
+`status: "OPEN"` would return the raise of every finding that has since been resolved. Fetch the
+stream and fold it, that is what "the last event wins" means in practice.
+
+Order ascending for the same reason: replaying out of order lets a stale `OPEN` overwrite the
+`RESOLVED` that followed it.
+:::
+
+Each finding event carries:
+
+| Field | What it holds |
+|---|---|
+| `externalId` | The finding this event belongs to, the correlation key you fold on |
+| `subType` | The policy that fired, by external id |
+| `source` | `datahub_policy_` |
+| `description` | What is wrong, in words |
+| `relatedResources` | The entity the finding is about, by node id |
+| `dataSetId` | That entity's data set |
+| `eventTime` | When this happened |
+| `status` | `OPEN` or `RESOLVED`, what *this event* asserts |
+| `metadata.offendingValue` | The external id that tripped the policy |
+| `metadata.suggestion` | A conforming alternative, when one could be derived |
+| `metadata.raisedBy` | Subject of whoever wrote the offending value |
+
+Resolve one by appending a `RESOLVED` event that names the same finding:
+
+```http
+POST /events/create
+{
+ "items": [{
+ "externalId": "policy_finding_naming_snake_case_42", // the finding's externalId, unchanged
+ "type": "policy_finding",
+ "subType": "naming_snake_case",
+ "status": "RESOLVED",
+ "eventTime": "2026-08-06T11:02:00Z",
+ "relatedResources": [{ "id": 42 }],
+ "dataSetId": 7
+ }]
+}
+```
+
+Copy `dataSetId` and `relatedResources` from the raise. The resolve has to come back from the
+same filtered query the raise does, or a queue narrowed to one data set sees the complaint and
+misses the answer to it. Resolving therefore needs write access to that data set.
+
+Resolving is a judgement rather than a fix: the entity still breaks the policy, someone has
+decided that is acceptable. Because it is appended rather than edited, it does not erase the
+raise it answers, the finding's history stays readable.
+
+**Reopening needs no special rule.** If the external id later changes to another non-conforming
+value, the policy appends a fresh `OPEN` after the `RESOLVED`, and the replay says open again.
+
+**Raising is idempotent.** Re-evaluating an entity whose external id has not changed collapses
+onto the raise already stored, so an entity written a thousand times contributes one `OPEN`
+event, not a thousand.
+
+Findings are raised for every node type the policy covers: everything whose external id
+is a unique identity. Events raise none, whatever the policy says; a finding being an event
+does not change that.
+
+### Paging the queue {#findings-paging}
+
+Findings from a bulk import arrive in the thousands, so page with the cursor rather than an
+offset, and keep folding into the same state as pages arrive: a `RESOLVED` on page 3 closes a
+finding whose `OPEN` came on page 1. Sort by `eventTime` ascending, which is the order a fold
+needs, and echo each response's `nextCursor` back as `cursor` with the same `sort`:
+
+```http
+POST /events/filter
+{
+ "filter": { "type": "policy_finding" },
+ "sort": { "property": ["eventTime"], "order": "asc" },
+ "cursor": "ZXZlbnRUaW1lfGFzY3wwMTk4N2VmMy0yYTc4LTdjM2UtOWYxYS0yYjRjNmQ4ZTBhMTN8djE3NTQ0NzY1MjIxMDQ",
+ "limit": 200
+}
+```
+
+No `status` in the filter: narrowing to `OPEN` would drop the `RESOLVED` events that close the
+findings you are folding.
+
+The cursor is opaque and carries both the boundary event's time and its id, so an import that
+lands thousands of findings in the same millisecond pages cleanly. `nextCursor` is absent on
+the last page.
+
+[Fetching and folding the queue, with SDK examples →](./events#policy-findings)
+
+## Practical advice
+
+- **Send the identifier your source system already uses.** Pre-normalising to snake_case is
+ accepted but loses the byte-for-byte match with the source system.
+- **Never change an external id after the fact.** It is a promise other systems have written
+ down. The platform permits it; your integrations will not forgive it.
+- **Read `warnings` if you have a steward.** It is the difference between finding out now and
+ finding out from a search that quietly comes back short.
+- **On events, keep the subject's key.** One id, many events, in time order.
diff --git a/versioned_docs/version-1.0/reference/files.md b/versioned_docs/version-1.0/reference/files.md
new file mode 100644
index 0000000..d9d9987
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/files.md
@@ -0,0 +1,303 @@
+---
+sidebar_position: 8
+title: Files
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Files
+
+List directories, upload files, and download content. File metadata travels in
+`X-Datahub-*` headers, which the SDK percent-encodes for you.
+
+## List
+
+
+
+
+```java
+DataWrapper root = client.files().list();
+DataWrapper reports = client.files().list("/reports/2026");
+```
+
+
+
+
+```python
+roots = client.files.list_root_directory()
+listing = client.files.list_directory_by_path("/reports/2026")
+```
+
+
+
+
+```rust
+let roots = api.files.list_root_directory().await?;
+let listing = api.files.list_directory_by_path("/reports/2026").await?;
+```
+
+
+
+
+## Upload
+
+The server takes one path, `X-Datahub-Path`: the full path of the file including its name
+(`/reports/2026/q2.csv`), from which it splits the name off the last `/`. The Java client
+sends the `path` you give it as-is, so include the file name and a leading `/`. The Python
+and Rust clients take a `destination_path` folder plus a `name` (defaulting to the local
+file's name) and join the two into that same full path. The Java client uploads raw `content`
+bytes; the Python and Rust clients read a local file.
+They also default the external id to the file name as-is, the same default the server
+applies. External ids are stored verbatim and allow only letters, digits and `. _ : + = -`, so a
+file whose name has a space or another character outside that set needs an explicit external
+id, or the upload is a `400` naming `externalId`.
+
+
+
+
+```java
+byte[] content = Files.readAllBytes(Path.of("report.csv"));
+
+DataWrapper uploaded = client.files().upload(
+ FileUploadRequest.builder()
+ .path("/reports/2026/q2.csv")
+ .content(content)
+ .contentType("text/csv") // default: application/octet-stream
+ .externalId("report_2026_q2") // optional
+ .dataSetId(42L) // optional
+ .description("Q2 production") // optional
+ .build());
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+upload = intellistream_datahub_sdk.FileUpload(
+ path="report.csv", # local file
+ destination_path="/reports/2026/",
+ external_id="report_2026_q2",
+ name="q2.csv",
+ data_set_id=42,
+ description="Q2 production")
+
+uploaded = client.files.upload_file(upload) # -> list[INode]
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::files::FileUpload;
+
+// mime type is inferred from the file content
+let mut upload = FileUpload::new_with_destination_path("report.csv", "/reports/2026/")?;
+upload.set_external_id("report_2026_q2".into());
+upload.set_file_name("q2.csv".into());
+upload.set_data_set_id(42);
+upload.set_description("Q2 production".into());
+let uploaded = api.files.upload_file(upload).await?;
+```
+
+
+
+
+The upload echo carries the stored node, id included. Read it back from the folder listing:
+
+
+
+
+```java
+client.files().list("/reports/2026").getItems()
+ .forEach(n -> System.out.println(n.getId() + " " + n.getName() + " " + n.getSize()));
+```
+
+
+
+
+```python
+for node in client.files.list_directory_by_path("/reports/2026"):
+ print(node.id, node.name, node.size)
+```
+
+
+
+
+```rust
+for node in api.files.list_directory_by_path("/reports/2026").await?.get_items() {
+ println!("{:?} {} {}", node.id, node.name, node.size);
+}
+```
+
+
+
+
+### When it fails {#errors}
+
+| Status | Means |
+| --- | --- |
+| `403` | You lack write access to the data set named in `dataSetId`, or to the parent folder's data set. |
+| `404` | On download: no file with that id, or a file in a data set you may not read. The two are not distinguished, so a hidden file's existence is not leaked. |
+| `409` | A file with that path, or that `externalId`, already exists. Uniqueness is tenant-wide, not per data set. |
+
+There is no size cap on `PUT /files`: the upload streams to disk and is exempt from the
+[request-body limit](./limits#request-body-size).
+
+## Download {#download}
+
+All three clients download a file's raw bytes by id. The id is numeric: Python and Rust take
+it as an integer, Java as a string, because the endpoint also accepts an external id in that
+position (`download("report_2026_q2")` works in Java). Python and Rust add a streaming variant
+that writes straight to a path, so a large file never has to sit in memory whole.
+
+
+
+
+```java
+byte[] bytes = client.files().download("99");
+Files.write(Path.of("q2.csv"), bytes);
+```
+
+
+
+
+```python
+download = client.files.download(99)
+print(download.file_name, download.mime_type, len(download))
+Path("q2.csv").write_bytes(download.content)
+
+# or stream it to disk, which returns the byte count
+written = client.files.download_to_path(99, "q2.csv")
+```
+
+
+
+
+```rust
+let download = api.files.download(99).await?;
+std::fs::write("q2.csv", &download.bytes)?;
+
+// or stream it to disk, which returns the byte count
+let written = api.files.download_to_path(99, "q2.csv").await?;
+```
+
+
+
+
+### Partial and conditional reads
+
+`GET /files/download/{id}` takes a byte range and a conditional request. The clients do not wrap
+this yet, so it is for calling the endpoint over HTTP directly: resuming a large download that
+dropped, or reading a file's header without pulling the whole thing.
+
+| Request header | Response |
+| --- | --- |
+| `Range: bytes=0-1023` | `206` with `Content-Range`, carrying only those bytes |
+| `Range: bytes=-1024` | The last 1024 bytes |
+| `Range: bytes=1024-` | Everything from that offset onwards |
+| `If-None-Match: ""` | `304` while the file is unchanged |
+| `If-None-Match: "", ""` | The same `304` if either tag is current; `*` matches any |
+| `If-Range: ""` | The range while the file is unchanged, otherwise the whole file |
+
+```bash
+curl -H "Authorization: Bearer $TOKEN" -H "Range: bytes=0-1023" \
+ https://api.example.com/files/download/99
+```
+
+The `ETag` is the file's SHA-256, the same value `checksum` carries in its metadata. Send it back
+as `If-Range` when resuming: without it, a download that resumes against a file which changed in
+the meantime splices two different bodies together and reports success. `If-None-Match` ignores a
+`W/` prefix, as the standard requires, but `If-Range` does not, so resume with the exact tag you
+were given.
+
+A range starting past the end of the file gets a `416`. Several ranges in one header are answered
+with the whole file, since `multipart/byteranges` is not supported.
+
+## Find, update and restore (Java) {#find-update-restore}
+
+Five more `/files` endpoints that the Java client wraps. They are all node operations on the
+file index, so none of them moves bytes.
+
+| Java | Endpoint | What it does |
+| --- | --- | --- |
+| `files().getById(id)` | `GET /files?id=` | One file or folder by numeric id. `404` when there is none. |
+| `files().getByExternalId(extId)` | `GET /files?externalId=` | The same by external id. |
+| `files().search(q, limit)` | `GET /files/search` | Full-text over names, paths and metadata. Crosses folders, so it finds a file whose location you do not know. Pass `null` for `limit` to take the server default. |
+| `files().trash()` | `GET /files/trash` | The soft-deleted files you can read. |
+| `files().restore(ids)` | `POST /files/restore` | Puts them back at the path they were deleted from. |
+| `files().update(fileUpdate)` | `POST /files/update` | Rename, move, or edit metadata on one node. |
+
+```java
+import ai.intellistream.datahub.models.files.FileUpdate;
+
+DataWrapper hits = client.files().search("inspection report", 20);
+
+FileUpdate move = new FileUpdate();
+move.setExternalId("report_2026_q2");
+move.setPath("/reports/2026/archive"); // the target folder is created if missing
+client.files().update(move);
+
+DataWrapper deleted = client.files().trash();
+client.files().restore(List.of(IdCollection.createFromExternalId("report_2026_q1")));
+```
+
+`update` takes a bare `FileUpdate`, not a wrapper: the endpoint updates one node per call.
+Identify the node by `externalId` or `id`; every other field is optional and null means "leave
+unchanged". `metadata` and `relatedResources` **replace** rather than merge.
+
+A restore is not a force-overwrite. If something else already occupies the path, you get a
+`409`, so move or rename the occupant first. Names, paths and external ids in the trash listing
+are the pre-deletion values, and the deletion time is in the node's `deletedAt` field
+(`deleted_at` in Python and Rust).
+
+## Delete
+
+
+
+
+```java
+client.files().delete(List.of(IdCollection.createFromExternalId("report_2026_q2")));
+```
+
+
+
+
+```python
+client.files.delete(["report_2026_q2"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+
+api.files.delete(&DataWrapper::from(vec![IdAndExtId::from_external_id("report_2026_q2")])).await?;
+```
+
+
+
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| List root | `files().list()` | `files.list_root_directory` | `files.list_root_directory` |
+| List a path | `files().list(path)` | `files.list_directory_by_path` | `files.list_directory_by_path` |
+| Upload | `files().upload` | `files.upload_file` | `files.upload_file` |
+| Download | `files().download` | `files.download` / `files.download_to_path` | `files.download` / `files.download_to_path` |
+| Delete | `files().delete` | `files.delete` | `files.delete` |
+| Get by id (`GET /files?id=`) | HTTP | `files.get_by_id` | `files.get_by_id` |
+| Get by external id (`GET /files?externalId=`) | HTTP | `files.get_by_external_id` | `files.get_by_external_id` |
+| Search names and descriptions (`GET /files/search?q=`) | HTTP | `files.search(query)` | `files.search(query)` |
+| List the trash (`GET /files/trash`) | HTTP | `files.list_trash` | `files.list_trash` |
+| Restore from the trash (`POST /files/restore`) | HTTP | `files.restore` | `files.restore` |
+| Rename, move or edit (`POST /files/update`) | HTTP | `files.update(FileUpdate)` | `files.update(FileUpdate)` |
+
+Python and Rust add `download_to_path`, which streams to disk instead of holding the whole
+file in memory, see [Download](#download).
+
+Java adds six index operations: `getById`, `getByExternalId`, `search`, `trash`, `restore`
+and `update`. [Find, update and restore →](#find-update-restore)
diff --git a/versioned_docs/version-1.0/reference/filters.md b/versioned_docs/version-1.0/reference/filters.md
new file mode 100644
index 0000000..e531c58
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/filters.md
@@ -0,0 +1,346 @@
+---
+sidebar_position: 2.5
+title: Filters
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Filters
+
+Every collection read takes a filter object rather than a list of query parameters. The same
+few shapes recur across the node types, so this page describes the shared grammar once; the
+per-endpoint pages say which filter each call takes and what it returns.
+
+The examples here only build filters. Nothing on this page calls the API, so none of it needs
+a seeded tenant; [Events](./events#filtering) has the worked queries.
+
+## How a field is shaped {#field-shape}
+
+A filter field that ORs its entries is named in the **singular** and takes either a bare value
+or a list, so `"type": "Alarm"` and `"type": ["Alarm", "Warning"]` are both valid and mean
+"either". The two exceptions, `labels` and `relatedResources`, keep plural names because their
+entries **AND** together: a second entry narrows the result where a second `type` would widen
+it. `metadata` behaves the same way.
+
+Text entries are literal or wildcard:
+
+| Character | Meaning |
+| --- | --- |
+| `*` | Any run of characters |
+| `%` | The same, so a caller who thinks in SQL is not surprised |
+| `_` | **Literal.** External ids are built out of underscores, so `sap_work_orders` means that id and not `sapXwork_orders` |
+
+One list mixes both freely — `["sap_work_orders", "plant_*", "*_archive"]` is exact lookup,
+prefix search and contains search in a single call. Entries without a wildcard resolve through
+an index; only the patterns need a scan, so an exact lookup keeps its index however many
+patterns share the list. Every OR'd text list is capped at 1000 entries.
+
+Case sensitivity differs in one place, and it is worth knowing which:
+
+| Filter | Literal entry | Wildcard entry |
+| --- | --- | --- |
+| Node filters (`externalId`, `name`, `source`) | Case-**in**sensitive | Case-insensitive |
+| `EventFilter.externalId` | Case-**sensitive** | Case-insensitive, through `ILIKE` |
+
+Events are the odd one out because every event writer hashes the external id verbatim, so the
+stored key for `shift_report_1` is not the key for `SHIFT_REPORT_1`; nodes lowercase before
+hashing and match either. Writing the entry as a wildcard — `"SHIFT_REPORT_1*"` — is the way to
+ask an event question case-insensitively.
+
+## The node filter family {#node-filters}
+
+Every node filter inherits the same eight fields.
+
+| Field | Type | Combines | Meaning |
+| --- | --- | --- | --- |
+| `id` | list<long> | OR | Node ids. Serialised as strings on the wire, so a JavaScript client cannot round them through a double. |
+| `externalId` | list<string> | OR | Pattern list over external ids. |
+| `name` | list<string> | OR | Pattern list over names. |
+| `source` | list<string> | OR | Pattern list over originating systems. |
+| `labels` | list<string> | **AND** | Labels the node must carry, all of them. Names are canonicalised, so `"pump a"` finds the label stored as `PUMP_A`. A name matching no label matches no nodes. |
+| `metadata` | map<string, string> | **AND** | Entries that must all be present. A null value matches the key alone. |
+| `createdTime` | [`TimeFilter`](#timefilter) | AND | Window on record creation. |
+| `lastUpdatedTime` | [`TimeFilter`](#timefilter) | AND | Window on last update. |
+
+Two things build on that base:
+
+- **`DataSetFilter` adds nothing.** A data set *is* the scope other nodes are filtered by, so it
+ has no `dataSetId` of its own — the eight fields above are the whole filter.
+- **Resource, time-series and function filters add `dataSetId`**, a list of
+ [`IdCollection`](#idcollection) references. A data set stands in for everything beneath it in
+ the `BELONGS_TO` hierarchy, so naming a parent covers its children — the same expansion the
+ data set ACL applies. An external id naming no data set contributes nothing.
+
+
+
+
+```java
+DataSetFilter plants = new DataSetFilter();
+plants.setName(List.of("Plant A*")); // OR: a pattern list
+plants.setLabels(List.of("PRODUCTION")); // AND: every label must be carried
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+plants = intellistream_datahub_sdk.DatasetFilter(
+ name="Plant A*", # a bare string is a one-entry list
+ labels=["PRODUCTION"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::datasets::DatasetFilter;
+
+let mut plants = DatasetFilter::new();
+plants.set_name(vec!["Plant A*".to_string()])
+ .set_labels(vec!["PRODUCTION".to_string()]);
+let plants = plants.build();
+```
+
+
+
+
+There is no Python `NodeFilter` class: the shared fields are inlined into each filter, so
+`DatasetFilter`, `ResourceFilter`, `TimeSeriesFilter` and `FunctionFilter` each declare them directly.
+
+`dataSetId` is the one list where **null and empty mean opposite things**:
+
+| Sent | Meaning |
+| --- | --- |
+| Field omitted, or `null` | No data set restriction |
+| `[]` | Narrow to no data sets — matches nothing |
+
+The field was once a scalar `Long`, so the old shape still fails loudly rather than quietly:
+`"dataSetId": 43` is a bare number where an object is expected and the request is rejected.
+Send `{"id": "43"}` or an external id instead.
+
+## Event filters {#event-filters}
+
+An event query has two layers: an outer retriever carrying paging and ordering, and an inner
+`EventFilter` carrying the predicates.
+
+The inner filter shares `externalId`, `source`, `metadata`, `createdTime`, `lastUpdatedTime`
+and `dataSetId` with the node filters above — same names, same types, same semantics, pinned by
+a parity test since the compiler cannot enforce it. On top of those it adds:
+
+| Field | Type | Combines | Meaning |
+| --- | --- | --- | --- |
+| `type` | list<string> | OR | Pattern list over event types. |
+| `subType` | list<string> | OR | Pattern list over sub-types. |
+| `status` | list<string> | OR | Pattern list over lifecycle statuses. |
+| `eventTime` | [`TimeFilter`](#timefilter) | AND | Window on when the event occurred, as distinct from when the record was written. |
+| `relatedResources` | list<[`IdCollection`](#idcollection)> | **AND** | The event must relate to all of these. Each entry may carry an `id`, an `externalId`, or both. |
+
+There is **no `id` field on an event filter**. Look an event up by id through
+[`byIds`](./events#lookup) instead.
+
+
+
+
+```java
+EventFilter alarms = new EventFilter();
+alarms.setType(List.of("Alarm", "Warning")); // either
+alarms.setStatus(List.of("OPEN"));
+alarms.setRelatedResources(List.of(IdCollection.createFromExternalId("pump_1")));
+
+EventRetreiver retriever = new EventRetreiver();
+retriever.setFilter(alarms);
+retriever.setLimit(100);
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+alarms = intellistream_datahub_sdk.EventFilter(
+ type=["Alarm", "Warning"],
+ status="OPEN",
+ related_resources=[intellistream_datahub_sdk.IdCollection(external_id="pump_1")])
+
+# There is no outer form in Python: limit, sort_by, sort_order and cursor are
+# keyword arguments of events.filter(), which lets one filter be reused across calls.
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::{DataSort, EventFilter, EventFilterForm};
+
+let mut criteria = EventFilter::new();
+criteria.set_type(&["Alarm", "Warning"])
+ .set_status(&["OPEN"])
+ .set_related_resource_external_ids(&["pump_1"]);
+
+let mut form = EventFilterForm::new(criteria.build());
+form.set_limit(100).set_sort(DataSort::asc("eventTime"));
+```
+
+
+
+
+The outer layer has a different name in each client: `EventRetreiver` in Java, `EventFilterForm`
+in Rust, and in Python no type at all — `limit`, `sort_by`, `sort_order`, `cursor` and
+`advanced_filter` are keyword arguments of `events.filter()`.
+
+The outer retriever carries `limit`, `cursor`, `sort` and `advancedFilter` — a PostgreSQL-flavoured
+boolean expression that is AND-combined with the inner filter.
+[Events](./events#filtering) documents all four, with the query examples.
+
+## Retrieving datapoints {#retrievefilter}
+
+Datapoint retrieval is not a node filter: it selects points within one series by time window
+rather than nodes by their columns. One filter is one series, so reading several means passing
+several filters.
+
+| Field | Type | Meaning |
+| --- | --- | --- |
+| `externalId` | string | The series, by external id. |
+| `id` | long | The series, by numeric id. Serialised as a string. |
+| `start` / `end` | timestamp | The window. **At least one is required.** |
+| `limit` | integer | Maximum datapoints, 0 to 100 000. Defaults to `0`; an explicit null becomes 100. |
+| `aggregates` | list<string> | Aggregates to compute instead of raw points. |
+| `granularity` | string | Bucket size for `aggregates`, e.g. `1h`. |
+| `includeOutsidePoints` | boolean | Include the points bracketing the window. Defaults to `false`. |
+| `mergeDuplicates` | boolean | Collapse points sharing a timestamp. Defaults to `false`. |
+| `cursor` | string | Continuation from a previous page, see [paging](#paging). |
+
+
+
+
+```java
+RetrieveFilter series = new RetrieveFilter();
+series.setExternalId("engine_temperature");
+series.setStart(ZonedDateTime.now().minusHours(1));
+series.setEnd(ZonedDateTime.now());
+series.setLimit(1000);
+```
+
+
+
+
+```python
+import datetime
+import intellistream_datahub_sdk
+
+now = datetime.datetime.now(datetime.timezone.utc)
+series = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature", # id or external id, positional and required
+ start=now - datetime.timedelta(hours=1),
+ end=now,
+ limit=1000)
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::generic::RetrieveFilter;
+
+// The constructor and setters are crate-private, so build the struct directly.
+let series = RetrieveFilter {
+ external_id: Some("engine_temperature".to_string()),
+ start: Some(Utc::now() - chrono::Duration::hours(1)),
+ end: Some(Utc::now()),
+ limit: Some(1000),
+ ..Default::default()
+};
+```
+
+
+
+
+Python names the series `ts` and takes it first; it accepts an id, an external id or a
+`TimeSeries`. Java and Rust carry `id` and `externalId` as separate fields.
+
+## Shared pieces {#shared}
+
+### IdCollection {#idcollection}
+
+Wherever a filter references another entity, it does so by id or by external id, in the same
+shape:
+
+```json
+[{"id": "43"}, {"externalId": "data_set_sap"}]
+```
+
+`id` travels as a JSON string even though it is a number, for the precision reason above.
+
+The type is named differently per client: `IdCollection` in Java and Python, `IdAndExtId` in
+Rust. Java and Rust offer `createFromId`/`createFromExternalId` and `from_id`/`from_external_id`;
+Python constructs it with keywords and refuses one carrying neither. Most Python call sites also
+accept a bare `int` or external-id `str` where a reference is expected.
+
+Events are keyed by UUID rather than a numeric id, so they use a separate `EventIdCollection`.
+
+### TimeFilter {#timefilter}
+
+A window on the wire, both bounds optional and **both inclusive** — subtract a millisecond to
+exclude the top:
+
+```json
+{"min": 1754476522104, "max": 1754480122104}
+```
+
+The clients spell it differently: Java sets `min` and `max`, Python takes `start` and `end` and
+refuses a filter with neither, and Rust models it as `Between`, `After` or `Before`. All three
+send `min`/`max`.
+
+### Sorting {#sorting}
+
+`sort` carries a `property` list and an `order`. There is deliberately **no `nulls` option** on
+a filter sort — where nulls sit follows the direction. A `nulls` field did exist and was removed,
+so sending one now is an unknown field and a `400`.
+
+Nodes default to `createdTime` descending and events to `eventTime` ascending. An unsortable
+property falls back to the default rather than failing, and any `order` that is not `desc`
+(compared case-insensitively) sorts ascending.
+
+Subscriptions take the same sort, without `nulls`, over `id`, `externalId`, `name`,
+`createdTime` and `lastUpdatedTime`, newest created first by default. In Python every `filter()`,
+`subscriptions.filter` included, takes `sort_by` and `sort_order` and builds the sort internally.
+
+### Default sizes {#defaults}
+
+| | Value |
+| --- | --- |
+| Server default, when no limit is sent | 1000 |
+| Maximum page size | 10 000 |
+
+Asking for more than the maximum is a `400`. The cap is on one page, not on how much you can
+read — walk past it with a cursor.
+
+When you do not set a `limit`, the Python and Rust clients either omit it or send 1000, so a
+call that says nothing about paging returns up to 1000 rows either way.
+
+## Paging with a cursor {#paging}
+
+A listing that has more to give returns a cursor on the envelope. It is **opaque**: base64, no
+version tag, and its contents are ours to change. Echo back exactly what you were handed rather
+than assembling one.
+
+A cursor is only meaningful against the order it was produced in, so it must travel with the
+same `sort`. Three things are refused with a `400`
+[`.../errors/malformed-cursor`](./client#problem-documents):
+
+- a value that is not readable as a cursor,
+- one whose row reference is not a valid id,
+- one produced by a different sort than the request asks for.
+
+None of them restarts the walk silently, which would loop a paging client or skip rows.
+
+Sorting by `subType` or `status` pages like any other sort, although both are nullable: the
+null block is a position in the order, so the walk continues through it rather than stopping
+after the first page.
+
+A full page does not mean there is more — a walk ends with one request that comes back empty.
+[Events](./events#paging) has the walk.
diff --git a/versioned_docs/version-1.0/reference/labels.md b/versioned_docs/version-1.0/reference/labels.md
new file mode 100644
index 0000000..e4a0d31
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/labels.md
@@ -0,0 +1,92 @@
+---
+sidebar_position: 5.5
+title: Labels
+---
+
+# Labels
+
+Labels are the tenant-wide vocabulary that resources and time series are categorised by. Every
+resource carries at least one. Creating a resource with a label name that does not exist yet
+creates the label.
+
+## The label object {#body}
+
+`LabelForm` is the request body, and what the Java client reads responses back into:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `id` | number | Server-assigned. |
+| `name` | string, 3 to 128 | Normalised to `SNAKE_UPPER_CASE`. Unique per tenant. |
+| `description` | string | Prose. |
+| `i18nCode` | string | Translation key for a localised UI (`nifi.function`). |
+| `color` | string, at most 7 | Hex, `#3A9F2E`. |
+
+## Read
+
+```java
+import ai.intellistream.datahub.label.LabelForm;
+
+DataWrapper all = client.labels().list();
+DataWrapper one = client.labels().getById(5677892L);
+```
+
+`list()` takes no limit and no cursor: it returns the whole vocabulary.
+
+## Create and update
+
+```java
+LabelForm critical = new LabelForm();
+critical.setName("critical"); // stored as CRITICAL
+critical.setDescription("Needs an operator response within the hour");
+critical.setColor("#cc11cc");
+
+client.labels().create(List.of(critical));
+```
+
+A name that already exists is a `409`.
+
+`update` identifies each label by `id` or by `name`, and the fields you send replace what is
+stored. `id` wins when both are given. An item that carries neither is a `400`. Renaming is the
+one case that needs the `id`, since a name used as the lookup key cannot also be the new name.
+
+```java
+LabelForm recolour = new LabelForm();
+recolour.setId(5677892L);
+recolour.setColor("#a11");
+client.labels().update(List.of(recolour));
+```
+
+By name instead, for a caller that never saw the synthetic id:
+
+```java
+LabelForm recolour = new LabelForm();
+recolour.setName("critical");
+recolour.setColor("#a11");
+client.labels().update(List.of(recolour));
+```
+
+## Delete
+
+```java
+client.labels().delete(List.of(IdCollection.createFromId(5677892L)));
+```
+
+The endpoint answers `204` with no body. Two cases are refused with a `400`:
+
+| Refusal | Detail |
+| --- | --- |
+| The label is still carried by a node | The error names what is blocking it. Clear it with `resources().update` and `labels.remove`, then delete. |
+| The label is a type-label | `ASSET`, `DATASET`, `POLICY`, `TIMESERIES` and `FUNCTION` are reserved, whether attached or not. They cannot be renamed either, and an ordinary label cannot be renamed onto one. |
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Get by id (`GET /labels/{id}`) | `labels().getById` | `labels.get` | `labels.get` |
+| List all (`GET /labels`) | `labels().list` | `labels.list` | `labels.list` |
+| Create (`POST /labels/create`) | `labels().create` | `labels.create` | `labels.create` |
+| Update (`POST /labels/update`) | `labels().update` | `labels.update` | `labels.update` |
+| Delete (`POST` or `DELETE /labels/delete`) | `labels().delete` | `labels.delete` | `labels.delete` |
+
+The per-resource label caps (64 labels, each at most 512 characters) are in
+[Limits & quotas](./limits#field-caps).
diff --git a/versioned_docs/version-1.0/reference/limits.md b/versioned_docs/version-1.0/reference/limits.md
new file mode 100644
index 0000000..6e329de
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/limits.md
@@ -0,0 +1,337 @@
+---
+sidebar_position: 11
+title: Limits & quotas
+description: The size, rate and volume ceilings the API enforces, the status code each one answers with, and which of them a client should retry.
+---
+
+# Limits & quotas
+
+The API enforces six kinds of ceiling, and the status code says which one you hit and what to
+do about it. That is the whole design: **what clears by waiting answers `429` and carries a
+`Retry-After`, and what does not answers something else.** Retry the first kind, fix the
+second.
+
+| Limit | Status | Problem `type` | How it clears |
+| --- | --- | --- | --- |
+| [Field caps](#field-caps) | `400` / `422` | the usual validation body | Shorten the field |
+| [Batch caps](#batch-caps) | `400` / `422` | the usual validation body | Split the batch |
+| [Request body size](#request-body-size) | `413` | `.../errors/request-too-large` | Split the batch |
+| [Binary frame caps](#binary-frames) | `400` / `413` | `.../errors/invalid-frame` on the `400`, `.../errors/request-too-large` on the `413` | Split the frame or fix the producer |
+| [Rate limit](#rate-limits) | `429` + `Retry-After` | `.../errors/rate-limit-exceeded` | Wait the seconds it names |
+| [Daily ingest quota](#daily-ingest-quotas) | `429` + `Retry-After` | `.../errors/ingest-quota-exceeded` | Wait until 00:00 UTC |
+| [Lifetime ceiling](#lifetime-ceilings) | `403`, no `Retry-After` | `.../errors/tenant-limit-reached` | Ask for it to be raised |
+| [Graph transfer](#graph-transfer) | `400` on export, `413` on import | an `error` body, not a problem document | It does not: the component is too large to move as one file |
+
+Every `type` above is prefixed `https://intellistream.ai/errors/`, and the `429` and `413`
+bodies are [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) problem documents served as
+`application/problem+json`. They reach you through the ordinary error path in each client
+(`DatahubApiException` in Java, `DataHubException` in Python, `ResponseError` in Rust); see
+[Results & errors](./client#results--errors).
+
+:::note The numbers are defaults, not guarantees
+Rate limits, quotas, ceilings and WebSocket caps are deployment policy: an operator sets
+them, and a tenant can be given its own. [Lifetime ceilings](#lifetime-ceilings) are off
+altogether unless a deployment turns them on. Read the `limit` field off the response rather
+than hard-coding the value. The field, batch and binary frame caps below are the wire contract
+and do not vary.
+:::
+
+## Field caps {#field-caps}
+
+Each of these bounds one entity. They apply on **create and on update alike**: the update
+forms enforce the same cap on `set` and on `add`, so you cannot grow past a cap one append at
+a time.
+
+| Field | Cap |
+| --- | --- |
+| `description` | 10 000 characters |
+| `metadata` | 256 entries, keys 128 characters, values 1 024 characters |
+| `labels` | 64 labels, each at most 512 characters |
+| `relatedResources` | 100 entries |
+| `geoLocation`, as raw GeoJSON | 65 536 characters |
+| Datapoint `value` | 64 characters |
+
+`description` is capped everywhere it appears: events, resources, time series, data sets,
+policies, functions and relationships.
+
+## Batch caps {#batch-caps}
+
+| Payload | Cap |
+| --- | --- |
+| `items` in a `DataWrapper` | 10 000 |
+| Nodes and relations in `POST /resources/create` and `/resources/update` | 1 000 each |
+| `datapoints` in one `DatapointsCollection`, numeric series | 100 000 |
+| `datapoints` in one `DatapointsCollection`, `TEXT` or `MIXED` series | 10 000 |
+
+The `items` cap covers every endpoint taking the standard envelope: `/events/create`,
+`/events/update`, `/events/delete`, `/timeseries/data`, `/timeseries/create`, `/edges/create`,
+`/datasets/create`, `/labels/create`, `/policies/create` and `/functions/create`.
+
+The tighter [`TEXT`/`MIXED`](./timeseries#value-types) cap is checked in the service, once the
+series' value type has been resolved, so it comes back naming the series type rather than the
+field. Split a text series into collections of 10 000 points or fewer.
+
+The `items` cap is enforced wherever the handler validates the body. `/events/update` and
+`/events/delete` do not, so an oversized update or delete batch is bounded only by the
+[request body size](#request-body-size).
+
+## Request body size {#request-body-size}
+
+| Endpoint | Cap |
+| --- | --- |
+| `POST /timeseries/data` | 16 MiB |
+| `POST /timeseries/data/binary` | 64 MiB compressed, `datahub.limits.max-body-bytes-datapoints-binary` |
+| Everything else | 4 MiB |
+| `PUT /files` and `GET /files/download/**` | exempt, they stream |
+| `POST /resources/import` and `GET /resources/export/{id}` | exempt, they stream; the [file format](#graph-transfer) has its own ceilings |
+
+```json
+{
+ "type": "https://intellistream.ai/errors/request-too-large",
+ "title": "Request body too large",
+ "status": 413,
+ "detail": "The request body exceeds the 4194304 byte limit for this endpoint.",
+ "limitBytes": 4194304
+}
+```
+
+A `413` is **terminal**. The same request will never become acceptable by being sent again,
+so split the batch instead of retrying it.
+
+`POST /timeseries/data/binary` answers a body over its cap the same way, `request-too-large`
+with `limitBytes` and no `reason`, before any frame is read. A body under the cap can still
+break a [frame cap](#binary-frames): the same `413` and `type`, plus a `reason`.
+
+## Binary frame caps {#binary-frames}
+
+[`POST /timeseries/data/binary`](./binary-datapoints) carries datapoints as frames, and the
+frame format fixes its own caps. Every refusal is a problem document carrying the `type` its
+status calls for, `.../errors/invalid-frame` on a `400`, `.../errors/request-too-large` on a
+`413` and `.../errors/too-many-in-flight` on the `429`, each with a stable `reason` beside it,
+and nothing of the request was inserted:
+
+| Cap | Value | Answered with |
+| --- | --- | --- |
+| Rows per frame, numeric series | 100 000 | `400`, `row-count-mismatch` |
+| Rows per frame, `TEXT` or `MIXED` series | 10 000 | `400`, `row-count-mismatch` |
+| Series per frame | 10 000 | `400`, `directory-invalid` |
+| Text value | 64 characters and 256 bytes | `400`, `value-invalid` |
+| Decompressed payload per frame | 4 MiB | `413`, `frame-too-large` |
+| Frames per request | 32 | `413`, `too-many-frames` |
+| Decompressed total per request | 64 MiB | `413`, `request-too-large` |
+| Binary requests validated at once, per API instance | deployment policy | `429`, `too-many-in-flight`, `Retry-After: 1` |
+
+The `400`s mean the producer is wrong and the `413`s mean split; only the `429` clears by
+waiting. The full list of reasons, and the retry rules, are on
+[Binary datapoint frames](./binary-datapoints#responses).
+
+## Graph transfer {#graph-transfer}
+
+[Export and import](./resources#graph-transfer) of a graph component have ceilings of their
+own, fixed in the file format rather than set by the deployment:
+
+| Limit | Cap | Answered with |
+| --- | --- | --- |
+| Nodes in one component or file | 2 000 000 | `400` on export, `413` on import |
+| Relationships in one component or file | 2 000 000 | `400` on export, `413` on import |
+| Compressed file size on import | 512 MB | `413` |
+
+These answer with the `{ "error": { "code", "message" } }` body the resource endpoints use,
+not a problem document, and none of them clears by waiting: a component over the cap cannot
+be exported as one file at all.
+
+## Rate limits {#rate-limits}
+
+Counted per organization and per user in a fixed one-minute window, with separate budgets for
+reads and writes.
+
+Which budget a request spends follows what it **does**, not which method it uses. A `GET` is a
+read and a `PUT`, `PATCH` or `DELETE` is a write, but a `POST` that only reads because it
+carries a filter body is charged as a read: `/events/filter`, `/resources/search`,
+`/timeseries/data/list`, `/timeseries/byids` and every other endpoint whose last path segment
+is `filter`, `search`, `byids`, `list`, `count`, `check`, `fetch-related`, `fetch-nearest`,
+`aggregate` or `latest`. So a poller sitting on a filter endpoint budgets against the read
+allowance, which is the larger of the two.
+
+| Scope | Writes / min | Reads / min |
+| --- | --- | --- |
+| Organization | 2 000 | 6 000 |
+| User | 600 | 1 200 |
+
+```json
+{
+ "type": "https://intellistream.ai/errors/rate-limit-exceeded",
+ "title": "Too many requests",
+ "status": 429,
+ "detail": "This tenant has used its 2000 requests per minute. Retry in 37 seconds.",
+ "scope": "tenant",
+ "limit": 2000,
+ "retryAfter": 37
+}
+```
+
+`scope` is `tenant` or `user`, which tells you whether the noisy neighbour is you or your
+colleagues. `Retry-After` carries the same seconds as `retryAfter`, and never exceeds the 60
+seconds left in the window.
+
+The [MCP tools](/mcp-server) at `/mcp` spend the **same budget** as REST: an agent and your
+ingest job share one allowance.
+
+## Daily ingest quotas {#daily-ingest-quotas}
+
+Per organization, per UTC day, reset at 00:00 UTC. `Retry-After` points at that reset, so it
+can be hours.
+
+| `metric` | Default per day |
+| --- | --- |
+| `events` | 100 000 |
+| `nodes` | 50 000 |
+| `relationships` | 100 000 |
+| `data points` | 10 000 000 |
+| `ingested bytes` | 1 GiB of write-request body |
+
+`nodes` is the shared count of resources, time series, data sets, labels, policies and
+functions: they are one population, not five. On the [binary datapoint path](./binary-datapoints)
+`ingested bytes` counts the **decompressed** size of the frames, so compressing harder does not
+stretch the allowance, and every quota is charged after validation, so a refused request costs
+nothing.
+
+```json
+{
+ "type": "https://intellistream.ai/errors/ingest-quota-exceeded",
+ "title": "Ingest quota exceeded",
+ "status": 429,
+ "detail": "Daily events ingest quota (100000) is spent; it resets at 00:00 UTC.",
+ "metric": "events",
+ "limit": 100000,
+ "retryAfter": 43200
+}
+```
+
+## Lifetime ceilings {#lifetime-ceilings}
+
+Totals, not rates: how large an organization may grow. Unlike everything above, these are
+**off unless a deployment turns them on**, and the numbers size a free or trial organization.
+Handle the `403`, but do not plan your data model around these figures: ask whoever runs your
+deployment what applies to you.
+
+| `metric` | Ceiling where they are on |
+| --- | --- |
+| `objects` (resources, time series, data sets, labels and policies share it) | 1 000 |
+| `events` | 25 000 |
+| `data points` | 1 000 000 000 |
+| `text data points`, on `TEXT` / `MIXED` series | 100 000 |
+
+```json
+{
+ "type": "https://intellistream.ai/errors/tenant-limit-reached",
+ "title": "Tenant limit reached",
+ "status": 403,
+ "detail": "This tenant has reached its limit of 25000 events. Contact IntelliStream to have it raised.",
+ "metric": "events",
+ "limit": 25000
+}
+```
+
+There is deliberately **no `Retry-After`**: waiting does not clear a ceiling, and the status
+is `403` rather than `429` so no client mistakes it for one that does. The ceiling moves when
+someone raises it, which is a conversation with whoever operates the deployment, not a retry.
+
+Whether deleting helps depends on the metric:
+
+| Metric | Counted | Does deleting free room? |
+| --- | --- | --- |
+| `objects` | Live rows | **Yes** |
+| `events`, `data points` | Cumulative | **No** |
+
+## WebSocket caps {#websockets}
+
+Both endpoints, `/timeseries/datapoints/subscription/listen/**` and
+`/timeseries/datapoints/listen`:
+
+| Cap | Default |
+| --- | --- |
+| Concurrent connections per organization | 10 |
+| Concurrent connections per user | 10 |
+| Subscriptions multiplexed over one socket | 10 |
+
+**Over the connection cap**, the server sends one error frame naming the limit and then
+closes with **1008** (policy violation), so you get a reason instead of a bare close. Both
+sockets send the same shape, so one parser reads either:
+
+```json
+{"error":true,"reason":"websocket-limit-reached","scope":"tenant","limit":10,"message":"..."}
+```
+
+**Over the per-socket subscription cap**, the socket **stays open**: the subscriptions
+already attached to it are still valid, and only the one that would not fit is refused.
+
+```json
+{"error":true,"subscriptionExternalId":"engine_temps","reason":"subscription-limit-reached"}
+```
+
+It arrives on the same path as the other per-subscription refusals
+(`forbidden`, `not-found`), so a client that already handles those handles this one. See
+[Subscriptions](./subscriptions#live-delivery).
+
+## What the SDKs do about it {#sdk-behaviour}
+
+The split above lines up with what the ingest paths already do, so most of this needs no code
+from you:
+
+| Response | Retried in process | Spooled when [buffering](./client#durable-ingest-buffering) is on |
+| --- | --- | --- |
+| `429` (rate limit, daily quota), `5xx`, network failure | Java: **yes**, with backoff, except a `5xx` the API explains as `needs-operator` (a `500 internal`). Rust and Python: on the binary path, see below | Yes, and Rust and Python send it again on the next ingest call |
+| `401`, and `403` on a grant | No | Yes, until the credential is fixed |
+| `403` on a [lifetime ceiling](#lifetime-ceilings) | No | Java: **no**, surfaced to you. Rust and Python: **yes**, like any `403` |
+| `400` / `422` (validation), `413` (body too large) | No | No, surfaced to you |
+| `404 unknown-timeseries` / `422 external-id-mismatch` on the [binary path](./timeseries#binary-ingest) | **Once**, after re-resolving the series | No, the binary path has no spool |
+
+Java's `ingest` reads the `retry` member of the
+[problem document](./client#problem-documents) where the answer carries one, and falls back to
+the status where it does not (a network failure, a proxy's HTML `502`). It backs off and replays a `429`, a
+`502`/`503`/`504`, a network failure and anything else marked `same-request`, up to `maxRetries`
+times; a `500` the API marks `needs-operator` is surfaced instead. Its backoff is floored at the
+`Retry-After` the response asked for, and a `Retry-After` over 30 seconds fails the batch at once
+rather than sleeping, so a spent daily quota reaches your code, or the spool, instead of holding
+a thread until 00:00 UTC.
+
+Rust and Python retry JSON ingest through the spool instead: with buffering on, a `408`, `429`, `5xx`
+or network failure is written to disk and the next ingest call sends it again, oldest first, before
+its own data; with it off, the error reaches your code. Their binary
+path, `insert_datapoints_binary`, retries `429`, `5xx` and network failures up to three times
+by default, 1, 2 and 3 seconds apart. Neither waits the `Retry-After`, so a rate limit that
+outlasts those retries, or a daily quota, ends in the spool or in your code the same way.
+A `413` or a validation failure reaches your code, which is the right place for it, since
+neither is fixed by trying again.
+
+In Java, a lifetime ceiling is the one `403` that does **not** spool: it surfaces on the call
+that hit it, in `errors()` on the [`IngestResult`](./timeseries#ingestresult). Why Java treats
+it differently from the other `403`s is under
+[durable ingest buffering](./client#durable-ingest-buffering). Rust and Python spool it like
+any other `403` when buffering is on, so the call returns without an error and the data waits
+in the spool until the time window or size cap drops it. With buffering off, the `403` reaches
+your code.
+
+Things to check in your own configuration:
+
+- **Java: `batchSize` defaults to `10 000`**, exactly the `items` cap. If you raised it, lower
+ it back to 10 000 or below. See [IngestOptions](./timeseries#ingestoptions). Rust and Python
+ have no batch size to set: `insert_datapoints` cuts requests at 100 000 datapoints, the
+ numeric per-collection cap.
+- **A `TEXT` or `MIXED` series batch must stay at or under 10 000 points per collection**,
+ whatever `batchSize` says. The Rust and Python chunking does not look at the value type
+ either, so split a text series yourself. A numeric batch of 10 000 points is roughly 500 KB
+ of JSON, comfortably inside the 16 MiB datapoint body cap.
+- **Rust and Python: `DATAPOINT_INSERT_PARALLELISM` sets how many datapoint requests are in
+ flight at once**, default `4`. Rust also takes `set_datapoint_insert_parallelism` on
+ `DataHubConfig`; Python reads it only from the environment, through `from_env()` or
+ `from_envfile(path)`. It applies with buffering off, since the buffered path sends one
+ request at a time. Each request can carry 100 000 points that the API holds in memory while
+ it parses them, so raise it with care. Java's counterpart is `parallelism` on
+ [IngestOptions](./timeseries#ingestoptions).
+- **The binary path has no size knob to get wrong**: `ingestBinary` (Java) and
+ `insert_datapoints_binary` (Rust, Python) cut frames and requests under the
+ [binary frame caps](#binary-frames) themselves. Rust tunes it with `BinaryIngestOptions`:
+ `zstd_level` (`9`), `max_retries` (`3`) and `request_concurrency` (`4`). Python takes only
+ `zstd_level=`.
diff --git a/versioned_docs/version-1.0/reference/policies.md b/versioned_docs/version-1.0/reference/policies.md
new file mode 100644
index 0000000..6c43f2a
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/policies.md
@@ -0,0 +1,143 @@
+---
+sidebar_position: 5.7
+title: Policies & governance
+---
+
+# Policies & governance
+
+A **policy** is a rule a data set is held to: read-only, field masking, a naming convention.
+Each one is an instance of a policy **type**, and each is a node in the same graph as
+resources.
+
+A **governance template** describes compliance rules (retention, access restrictions, required
+metadata). Templates are read-only over the API; a policy names one through `templateId`, and
+the template's metadata is merged into the policy.
+
+The Java client reaches both through `client.policies()` and `client.governance()`.
+[`datasets().policies()`](./datasets#client-coverage) lists the same catalogue from the data
+set side, in all three clients.
+
+Python and Rust have no policy service. A policy is a node, so they reach one through
+[`resources`](./resources#typed-reads): a read narrowed to node type `policy` comes back as `Policy`,
+and `resources.create` takes a `Policy` node, under the same `403` rule as any
+[`POLICY`-labelled create](./resources#body).
+
+## Policy types {#types}
+
+`GET /policies/types` lists the types a policy can instantiate. Each entry carries the type's
+`name`, `type` and `description`.
+
+```java
+import ai.intellistream.datahub.models.Policy;
+
+DataWrapper types = client.policies().listTypes();
+```
+
+| Type | Scope | What it does |
+| --- | --- | --- |
+| `SECURITY_POLICY` | One data set | Restricts data access or actions for security compliance. |
+| `ENCRYPTION_POLICY` | One data set | Encryption requirements for stored or transmitted data. |
+| `MASKING_POLICY` | One data set | Hides or obfuscates sensitive fields. |
+| `IS_WRITE_PROTECTED` | One data set | Marks the data set read-only; blocks writes except delete. |
+| `IS_READ_PROTECTED` | One data set | Restricts read access to authorised users. |
+| `HAS_REQUIREMENT` | One data set | Marks a data set that must meet a compliance requirement. |
+| `NAMING_CONVENTION` | Tenant-wide, overridable per data set | Enforces the [external-id naming convention](./external-ids#the-naming-policy) at write time. Applies to resources and data sets; events are exempt. |
+
+Attaching a policy where its type does not allow is refused on create, over the whole batch,
+rather than accepted and quietly enforcing nothing.
+
+## Read, create, update, delete
+
+```java
+DataWrapper some = client.policies().list(100);
+DataWrapper one = client.policies().getById(5677892L);
+
+Policy readOnly = new Policy();
+readOnly.setName("IS_WRITE_PROTECTED"); // the type it instantiates
+readOnly.setExternalId("plant_oslo_read_only"); // optional
+client.policies().create(List.of(readOnly));
+```
+
+A policy's `type` is read back from its `name`, so name it after the type. An `externalId` you
+leave out is generated. `templateId` names a [governance template](#governance-templates).
+
+`update` is partial, on the same field verbs as
+[`resources().update`](./resources#update): `set`, `setNull`, and `add` / `remove` for
+`metadata`. Only the fields named in each entry's `update` block change; a field you leave out
+keeps its stored value, `deactivated` included.
+
+```java
+import ai.intellistream.datahub.models.forms.UpdatePolicyForm;
+
+UpdatePolicyForm off = new UpdatePolicyForm();
+off.setExternalId("plant_oslo_read_only");
+off.getUpdate().getDeactivated().set(true);
+client.policies().update(List.of(off));
+```
+
+`delete` takes ids and answers `204` with no body:
+
+```java
+client.policies().delete(List.of(IdCollection.createFromExternalId("plant_oslo_read_only")));
+```
+
+## Dry-run a name against the naming policy {#naming-check}
+
+`POST /policies/naming/check` runs the naming policy over candidate external ids and reports
+what it would do. It writes nothing.
+
+```java
+import ai.intellistream.datahub.models.policy.NamingCheckForm;
+import ai.intellistream.datahub.models.policy.PolicyFinding;
+
+NamingCheckForm form = new NamingCheckForm();
+form.setExternalIds(List.of("COM-99-PT-1034", "vps"));
+form.setNames(List.of("Valve 21 PT 1034", "Valve pressure sensors"));
+form.setDataSetId(12L); // omit for the tenant policy
+
+List findings = client.policies().checkNaming(form);
+for (PolicyFinding finding : findings) {
+ System.out.println(finding.externalId() + ": " + finding.message()
+ + " (try " + finding.suggestion() + ")");
+}
+```
+
+| Field | Notes |
+| --- | --- |
+| `externalIds` | Required, at most 1000. |
+| `names` | Optional, aligned by position. Either omit it or supply exactly as many as there are ids; a length mismatch is rejected. |
+| `dataSetId` | Check against the policy governing this data set. Omit for the tenant policy. `403` when you cannot read it. |
+
+`checkNaming` unwraps the `findings` envelope and hands back the list. Only non-conforming ids
+appear, so an empty list means every id is fine. `PolicyFinding` is a record: `index()` (the item's position in
+your batch), `externalId()`, `decision()` (`OK`, `WARNING` or `NOT_OK`), `policyExternalId()`
+(`policy` on the wire), `message()` and `suggestion()`, which is null when none can be derived.
+
+Violations that were allowed through and recorded are events, read with `events().filter` on
+`type = "policy_finding"`.
+
+## Governance templates {#governance-templates}
+
+```java
+import ai.intellistream.datahub.models.GovernanceTemplateDTO;
+
+DataWrapper all = client.governance().listTemplates();
+DataWrapper one = client.governance().getTemplateById(12L);
+```
+
+`GovernanceTemplateDTO` is a record of `id()`, `externalId()`, `name()`, `description()` and
+`metadata()`.
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| List policies (`GET /policies?limit=`) | `policies().list` | — | — |
+| List policy types (`GET /policies/types`) | `policies().listTypes` | — | — |
+| Get by id (`GET /policies/{policyNodeId}`) | `policies().getById` | — | — |
+| Create (`POST /policies/create`) | `policies().create` | — | — |
+| Update (`POST /policies/update`) | `policies().update` | — | — |
+| Delete (`POST` or `DELETE /policies/delete`) | `policies().delete` | — | — |
+| [Naming dry-run](#naming-check) (`POST /policies/naming/check`) | `policies().checkNaming` | — | — |
+| List governance templates (`GET /governance/templates`) | `governance().listTemplates` | — | — |
+| Get a governance template (`GET /governance/templates/{id}`) | `governance().getTemplateById` | — | — |
diff --git a/versioned_docs/version-1.0/reference/resources.md b/versioned_docs/version-1.0/reference/resources.md
new file mode 100644
index 0000000..d2328e9
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/resources.md
@@ -0,0 +1,1160 @@
+---
+sidebar_position: 3
+title: Resources
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Resources
+
+Hierarchical, asset-like entities and the relationships between them. Create resources
+and the edges between them in one call; the server returns the persisted graph.
+
+A resource's `externalId` is its **identity**: unique per tenant, stored exactly as you send
+it, and compared without case. Mirror the tag your operation already maintains,
+`COM-99-PT-1034` is stored as `COM-99-PT-1034`, not rewritten.
+[External ids & naming →](./external-ids)
+
+## The resource body {#body}
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `id` | number | Server-assigned. Crosses the wire as a JSON string, see the note below. |
+| `externalId` | string, 3–256 | **Required.** Unique per tenant, stored verbatim, matched case-insensitively. |
+| `name` | string, 3–512 | **Required.** What a human calls it. This is the field [search](#search) reads. |
+| `labels` | string[] | **Required, at least one.** The type tags (`Pump`, `Plant`). Upper-cased by the server. |
+| `description` | string | Prose. |
+| `metadata` | map<string, string> | Flat key/value, filterable by exact match. |
+| `source` | string, 2–128 | The upstream system of record this came from (`SAP`, a historian, a file drop). |
+| `dataSetId` | number | The data set the resource belongs to. |
+| `geoLocation` | GeoJSON geometry | `Point`, `Polygon`, … Validated on write; stored verbatim. Returned only on [assets](#typed-reads). |
+| `isRoot` | boolean | Whether the resource is a navigation root. Deletes are checked against reachability from a root, see [Delete](#delete). Returned only on resources and assets. |
+| `relatedResources` | object[] | Read-only view of the graph: `{ id, externalId, relationshipType, direction, edgeId }` per connected node. Populated on the create echo and on `fetch-related` / `fetch-nearest`; empty on `/resources/{id}`, `byids`, `filter` and `search`. |
+| `createdTime`, `lastUpdatedTime` | epoch millis | Server-set. A create body may carry them, but they are ignored: the stored values are the server's. |
+
+Labels are how the platform types a node. The type-label (`ASSET`, `TIMESERIES`, `DATASET`,
+`POLICY`, `FUNCTION`) is what the create pipeline reads to decide which kind of entity to
+build, and free-form labels ride alongside it. That is also why one `/resources/create` call
+can hold a mix of node types, a time series next to an asset, rather than needing one
+endpoint per type. The same label types what a read returns: see
+[Reads come back typed](#typed-reads).
+
+Two of those type-labels are privileged. A create carrying `DATASET` or `POLICY` builds a
+data set or a policy, and managing those is stricter than writing data: it requires the
+`/datasets/*/write` grant or `DATAHUB_ADMIN`, whichever endpoint the request arrives
+through. Without it the call is a `403`, even when you can write the data set named in
+`dataSetId`. The same rule guards updating or deleting such a node via `/resources`, and
+edges onto a data set node. See [Access control](./datasets#access-control).
+
+:::note Numeric ids cross the wire as JSON strings
+`id` and `dataSetId` serialize as `"5677892"`, not `5677892`, ids can exceed the 53-bit
+integer a JSON number is safe for in JavaScript. The clients parse them back for you. The same
+holds for the ids on an [edge](./edges#body), `start` and `end` included.
+:::
+
+## Reads come back typed {#typed-reads}
+
+The read endpoints (`/resources/{id}`, `byids`, `filter`, `search`, `fetch-related`,
+`fetch-nearest`) return each node in the shape of its kind, and the type-label inside
+`labels` is the discriminator. There is deliberately no separate type property on the wire:
+an element whose labels contain `TIMESERIES` *is* the time series shape.
+
+| Type-label present | Shape returned |
+| --- | --- |
+| `ASSET` | An asset: the body above, `geoLocation` included. |
+| `TIMESERIES` | A [time series](./timeseries): `unit`, `unitExternalId`, `valueType`. |
+| `DATASET` | A [data set](./datasets). |
+| `POLICY` | A policy: `type` and `deactivated`. A read never carries its `value`, `templateId` or `dataSetId`. |
+| `FUNCTION` | A function. |
+| none | A plain resource, the body above. |
+
+Three rules govern which fields appear where:
+
+- A time series carries its **full label set**, not only `["TIMESERIES"]`.
+- `isRoot` belongs to resources and assets; `geoLocation` belongs to assets. A flat resource
+ body naming a `geoLocation` is a `400`: a plain resource has nowhere to store one, so it is
+ refused rather than accepted and dropped. Send an `ASSET`-labelled body instead.
+- A policy carries no `nodeType` field. The `POLICY` label is the type.
+
+
+
+
+These calls return `DataWrapper`, and the concrete class of each item is the
+subtype, so pattern-match to reach type-specific fields. `fetchRelated`/`fetchNearest` still
+return a `ResourceNetwork`; its `nodes` are `NodeModel` too.
+
+```java
+for (NodeModel node : client.resources().filter(retriever).getItems()) {
+ if (node instanceof Timeseries ts) {
+ System.out.println(ts.getExternalId() + " in " + ts.getUnit());
+ }
+}
+```
+
+
+
+
+Each item is the same class the type's own endpoint returns, so `isinstance` works and a
+time series from `resources.filter()` behaves exactly like one from `timeseries.by_ids()`.
+`Asset` and `Policy` are in the set.
+
+```python
+from intellistream_datahub_sdk import TimeSeries
+
+for node in client.resources.filter(external_id="pump_*"):
+ if isinstance(node, TimeSeries):
+ print(node.external_id, node.unit)
+```
+
+When you are dispatching from data rather than branching, every node class also carries
+`node_type`, one of `asset`, `timeseries`, `function`, `resource`, `dataset`, `policy`:
+
+```python
+by_type = {}
+for node in client.resources.filter(external_id="pump_*"):
+ by_type.setdefault(node.node_type, []).append(node)
+```
+
+
+
+
+Reads return `DataWrapper` (or `GraphDataWrapper`), where `Node` is an enum with
+one variant per type. Match it, or use the accessors for the fields every node shares.
+
+```rust
+use intellistream_datahub_sdk::Node;
+use intellistream_datahub_sdk::filters::NodeFilter;
+use intellistream_datahub_sdk::resources::{ResourceFilter, ResourceFilterForm};
+
+let form = ResourceFilterForm::new(ResourceFilter {
+ node: NodeFilter { external_id: Some(vec!["pump_*".into()]), ..Default::default() },
+ ..Default::default()
+});
+
+for node in api.resources.filter(&form).await?.get_items() {
+ match node {
+ Node::TimeSeries(ts) => println!("{} in {:?}", ts.external_id, ts.unit),
+ other => println!("{} ({:?})", other.external_id(), other.kind()),
+ }
+}
+```
+
+`Node` is `#[non_exhaustive]`, so a node type added later is not a breaking change for a
+`match` that already has a catch-all arm.
+
+
+
+
+Create and update echoes are typed the same way, so an asset updated through
+`/resources/update` comes back as an asset carrying its `geoLocation`. One difference on the
+update echo: `relatedResources` is left empty on purpose. The request touched only some of the
+node's edges, and answering with those alone would be indistinguishable from answering with all
+of them. A delete has no echo at all, being a `204` with no body.
+
+The Python and Rust clients read the update echo typed as well: Python's `GraphResult.nodes`
+holds each node as its own class, and Rust's `update` returns `GraphDataWrapper`, so a
+time series updated there comes back with its `unit` and an asset with its `geoLocation`.
+
+## Look up
+
+Fetch by numeric id or external id (you can mix them). Lookup ignores case, so `pump_1` and
+`PUMP_1` resolve to the same resource; what comes back keeps the spelling it was created
+with. Identifiers that match nothing are **silently omitted** rather than erroring, so
+compare the returned items against what you asked for when a miss matters.
+
+
+
+
+```java
+import ai.intellistream.datahub.models.IdCollection;
+
+NodeModel pump = client.resources().getById(5677892).getItems().get(0);
+
+DataWrapper some = client.resources().byIds(List.of(
+ IdCollection.createFromExternalId("pump_1"),
+ IdCollection.createFromId(5677892)));
+```
+
+
+
+
+```python
+# pass entity objects, external-id strings, or numeric ids
+resources = client.resources.by_ids(["pump_1", 5677892])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let resources = api.resources.by_ids(&vec![
+ IdAndExtId::from_external_id("pump_1"),
+ IdAndExtId::from_id(5677892),
+]).await?;
+```
+
+
+
+
+## Create resources and relations {#create-resources-and-relations}
+
+Pass the resource forms (nodes) and the relation forms (edges); the call returns the
+created graph, nodes plus server-assigned edges. Each resource needs **at least one
+label** (a type tag such as `Plant` or `Pump`), a node with none is rejected with
+`400 resource.needs.at.least.one.label`. Labels and relationship types are both
+upper-cased by the server. External ids are not: they are stored verbatim.
+
+The call is **all-or-nothing**. Every external id in the batch is validated before anything
+is written, so one item rejected by the
+[naming policy](./external-ids#the-naming-policy) means nothing is created and the `400`
+names every offending item, not just the first. If the policy is set to warn instead, the
+response carries a [`warnings` array](./external-ids#warnings-on-the-response) next to
+`items`.
+
+One request carries at most **1 000 nodes and 1 000 relations**, and one node at most 10 000
+characters of `description`, 256 metadata entries, 64 labels and 64 KiB of raw GeoJSON in
+`geoLocation`. Past any of those is a `400` before anything is written; the same caps apply on
+[update](#update), to `set` and `add` alike. See [Limits & quotas](./limits#field-caps).
+
+A relation may reference a node being created in the same request by its `externalId`, or
+point at one that already exists. An edge whose endpoint is neither is a `400` naming the
+endpoint it could not resolve.
+
+Two more checks run over the whole batch before anything is written, on **every** node create
+endpoint: `/resources`, `/assets`, `/datasets`, `/functions`, `/policies` and `/timeseries`
+alike. Both refuse the whole request, so a rejected batch creates nothing.
+
+| Refused | Status | Named in |
+| --- | --- | --- |
+| An `externalId` already taken in the tenant, or repeated within the same batch. Compared without case. | `409` `duplicate` | `duplicated`, one entry per offending id |
+| A `dataSetId` that does not exist, or that resolves to a node which is not a data set. | `400` `bad-request` | `fields`, one entry per offending id |
+
+Both are [problem documents](./client#problem-documents), so branch on the `type` and read the
+entries from the extension member. `requestId` and `retry` ride along as on any refusal:
+
+```json
+{
+ "type": "https://intellistream.ai/errors/duplicate",
+ "title": "Conflict",
+ "status": 409,
+ "detail": "A node with that externalId already exists.",
+ "instance": "/resources/create",
+ "duplicated": [{ "externalId": "pump_1" }]
+}
+```
+
+Use [update](#update) to change an existing resource rather than re-creating it. Access is
+decided before either check, so a caller who may not write the data set is told that (`403`)
+instead of being handed a `400` about an id they were never allowed to name.
+
+:::note Relationships into data sets and time series are validated
+Two endpoint rules apply to every relationship, on create and on update (an update can
+retarget a relationship or change its type):
+
+- A relationship **to a data set** must use the `BELONGS_TO` relationship type, that is the
+ relationship the data set hierarchy and membership are built from, and anything else is
+ rejected with a `400`.
+- A **data set → time series** relationship is accepted only when the series has no data set
+ yet, or already belongs to that very data set (creating a series inside a data set produces
+ exactly that membership relationship). A series in a *different* data set is rejected with a
+ `400`, a time series has one data set.
+:::
+
+
+
+
+```java
+Resource plant = new Resource();
+plant.setExternalId("plant_oslo");
+plant.setName("Oslo Plant");
+plant.setLabels(List.of("Plant"));
+
+Resource pump = new Resource();
+pump.setExternalId("pump_1");
+pump.setName("Pump 1");
+pump.setLabels(List.of("Pump"));
+
+RelForm contains = new RelForm();
+contains.setName("contains");
+contains.setFromExternalId("plant_oslo");
+contains.setToExternalId("pump_1");
+
+GraphDataWrapper created = client.resources()
+ .create(List.of(plant, pump), List.of(contains));
+
+System.out.println(created.getNodes().size() + " resources, "
+ + created.getRelations().size() + " relations");
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+plant = intellistream_datahub_sdk.Resource(external_id="plant_oslo", name="Oslo Plant", labels=["Plant"])
+pump = intellistream_datahub_sdk.Resource(external_id="pump_1", name="Pump 1", labels=["Pump"])
+contains = intellistream_datahub_sdk.RelForm.by_external_ids("plant_oslo", "pump_1", "contains")
+
+result = client.resources.create([plant, pump], [contains])
+print(len(result.nodes), "resources,", len(result.relations), "relations")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::Resource;
+use intellistream_datahub_sdk::relations::RelForm;
+
+let mut plant = Resource::new();
+plant.external_id = "plant_oslo".into();
+plant.name = "Oslo Plant".into();
+plant.labels = Some(vec!["Plant".into()]);
+
+let mut pump = Resource::new();
+pump.external_id = "pump_1".into();
+pump.name = "Pump 1".into();
+pump.labels = Some(vec!["Pump".into()]);
+
+let contains = RelForm::by_external_ids("plant_oslo", "pump_1", "contains");
+
+let created = api.resources.create(vec![plant, pump], vec![contains]).await?;
+
+println!("{} resources, {} relations",
+ created.nodes().map_or(0, |n| n.len()),
+ created.relations().map_or(0, |r| r.len()));
+```
+
+
+
+
+An edge comes back as a `Relation`, `{ id, start, end, type, description, metadata }`,
+where `start` and `end` are the ids of the two nodes (as JSON strings, like every other id).
+That is why you send `fromExternalId`/`toExternalId` but read `start`/`end`: the write side
+speaks in your identifiers, the read side in the graph's.
+
+Relations are directional. `from` → `to` is the direction you will see when you
+[traverse](#traverse-the-graph), so `plant contains pump` and `pump contains plant`
+describe different graphs.
+
+### Relations without the nodes {#create-relations}
+
+There are two ways to create a relation and they produce the same edge. The call above sends
+nodes and relations together, in one transaction. `POST /edges/create` sends the relations by
+themselves, for when both ends already exist and repeating them would be noise, same fields,
+same rules, same edges back.
+
+That endpoint, and the rest of the `/edges` surface (reading a relationship back, deleting one
+without touching its endpoints, the relationship-type catalogue), has its own page.
+[Edges →](./edges)
+
+To disconnect two resources without touching either of them, [delete the edge](./edges#delete).
+[Deleting a resource](#delete) is the heavier move: it takes every relation the resource had
+with it.
+
+## List
+
+`GET /resources?limit=` returns the newest `limit` resources you may read, with no body and no
+criteria. It is the cheap "what have I got" read, and **every node type answers it the same way**:
+
+| Endpoint | Returns |
+| --- | --- |
+| `GET /resources?limit=` | The newest resources you may read. |
+| `GET /assets?limit=` | The same, restricted to assets. |
+| `GET /datasets?limit=` | Data sets. |
+| `GET /timeseries?limit=` | Time-series. It also takes `dataSetId=` to scope to one data set and everything beneath it. |
+| `GET /events?limit=` | Events. |
+| `GET /policies?limit=` | Policies. |
+| `GET /functions?limit=` | Functions. |
+
+`limit` defaults to **1000** and is capped at **10000**; above that is a `400` rather than a
+silent clamp, so a short page always means you ran out of rows. Absent, zero or negative all mean
+"you decide" and give you the default. These are the same numbers `POST //filter`
+uses, so which one you reach for cannot change the page size you get.
+
+The listing is the **first page only**: it never returns a `nextCursor`. A walk needs a `sort` and
+a cursor to continue it, and both belong in a request body, so paging lives on `/filter` below.
+
+```bash
+curl -H "Authorization: Bearer $TOKEN" "$API/resources?limit=50"
+```
+
+## Filter
+
+`POST /resources/filter` finds resources by structured criteria. Everything you supply is
+combined with **AND**.
+
+| Field | Matching |
+| --- | --- |
+| `name` | Pattern, case-insensitive. `*` and `%` are wildcards, `_` is literal. |
+| `source` | Pattern, on the same rules. |
+| `externalId` | Pattern, on the same rules. |
+| `id` | Exact numeric id. |
+| `nodeType` | Restrict to these node types. Omit for every type. |
+| `isRoot` | `true` or `false`. |
+| `labels` | Resources carrying **all** of these labels. |
+| `dataSetId` | Resources in any of these data sets. |
+| `metadata` | Every key/value given must be present on the resource. |
+| `createdTime`, `lastUpdatedTime` | `{ "min": …, "max": … }`, [ISO-8601 or epoch millis](./client#timestamps), both bounds inclusive. |
+
+Each field above except `isRoot`, `labels` and `metadata` takes **either a bare value or an
+array**, and the entries of an array are combined with **OR**. That is why they are named in the
+singular: `"name": "pipe%"` is the common case, and `"name": ["pipe%", "valve%"]` asks for either.
+`labels` and `metadata` are the exceptions, requiring **all** entries to match, and they keep
+plural names because adding an entry there narrows the result where adding a `name` widens it.
+
+```json
+{
+ "limit": 100,
+ "filter": {
+ "name": "pipe%",
+ "dataSetId": [{ "id": 12 }, { "externalId": "data_set_sap" }],
+ "metadata": { "work_order": "wo-sap-12344" },
+ "createdTime": { "min": "2026-01-01T00:00:00Z" }
+ }
+}
+```
+
+`limit` defaults to **1 000** and is capped at **10 000**; a zero, negative or null value
+falls back to the default rather than returning nothing. Results come newest created first
+unless ordered otherwise, and page with a cursor, the same contract as
+[timeseries](./timeseries#sorting-and-paging), over the same sortable properties.
+
+:::caution A pattern-less value matches exactly, not as a substring
+`"name": "pipe"` matches a resource named exactly `pipe`, not every name containing it. Add a
+wildcard for the loose match you probably want: `"pipe*"` for a prefix, `"*pipe*"` for a contains
+search. The same holds for `source` and `externalId`.
+:::
+
+:::note Omitting `dataSetId` and sending `[]` are opposites
+Omit the field (or send `null`) for **no data set restriction**. An explicit empty list means
+**narrow to no data sets**, which matches nothing. Every other list field treats empty as "no
+restriction", so this is the one to watch when you build the filter programmatically.
+:::
+
+
+
+
+```java
+ResourceRetreiver retriever = new ResourceRetreiver();
+retriever.setLimit(100);
+retriever.getFilter().setName(List.of("pipe%"));
+retriever.getFilter().setMetadata(Map.of("work_order", "wo-sap-12344"));
+retriever.getFilter().setDataSetId(List.of(IdCollection.createFromId(12L)));
+
+DataWrapper matches = client.resources().filter(retriever);
+```
+
+
+
+
+```python
+matches = client.resources.filter(
+ name="pipe%",
+ metadata={"work_order": "wo-sap-12344"},
+ data_set_id=[12],
+ limit=100)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::NodeFilter;
+use intellistream_datahub_sdk::generic::IdAndExtId;
+use intellistream_datahub_sdk::resources::{ResourceFilter, ResourceFilterForm};
+
+// The criteria every node type shares are a flattened `NodeFilter`, so they nest in Rust even
+// though they sit alongside the resource's own fields on the wire.
+let form = ResourceFilterForm::new(ResourceFilter {
+ node: NodeFilter {
+ name: Some(vec!["pipe%".into()]),
+ metadata: Some([("work_order".into(), Some("wo-sap-12344".into()))].into()),
+ ..Default::default()
+ },
+ data_set_id: Some(vec![IdAndExtId::from_id(12)]),
+ ..Default::default()
+}).with_limit(100);
+
+let matches = api.resources.filter(&form).await?;
+```
+
+
+
+
+## Search {#search}
+
+Free-text search across **every node type** (assets, timeseries, functions, resources, data sets
+and policies), the same breadth as `POST /resources/filter`. The phrase is matched against `name`,
+`externalId` and `description`. Matching is fuzzy and word-aware: search `pipe` and you also get
+`pipes`, `piping`, and multi-word names containing the term.
+
+Results are **ranked by relevance** (`ts_rank`), strongest match first, with `id` as a tie-break so
+equal-scoring rows keep a stable order and repeated identical searches agree. Ranking means the
+database scores and sorts every match before applying `limit`, so a very broad phrase costs more
+than a narrow one.
+
+`limit` is capped at **1 000** here, lower than the 10 000 of `filter`, and `query` must be
+3 to 140 characters. `limit` applies across all node types, and `POLICY` nodes are searched.
+
+### Narrowing with `filter` {#search-filter}
+
+`filter` is optional and takes the same criteria as `POST /resources/filter`. It only ever removes
+matches: the phrase decides what the candidates are. `nodeType` and `dataSetId` are applied by the
+search query itself, everything else is applied to the hits afterwards.
+
+```json
+{
+ "search": { "query": "pump" },
+ "filter": { "nodeType": ["timeseries"], "dataSetId": [{ "id": "12" }] },
+ "limit": 50
+}
+```
+
+
+
+
+```java
+SearchBody search = new SearchBody<>();
+search.setLimit(10);
+search.getSearch().setQuery("pump");
+DataWrapper matches = client.resources().search(search);
+```
+
+
+
+
+```python
+matches = client.resources.search("pump", limit=10)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::SearchAndFilterForm;
+use intellistream_datahub_sdk::resources::ResourceFilter;
+
+let form = SearchAndFilterForm::::new("pump").with_limit(10);
+let matches = api.resources.search(&form).await?;
+```
+
+
+
+
+Reach for `filter` instead whenever the question is structured, an exact external id, a
+metadata value, a data set, a time range. It is faster and its results are predictable.
+
+## Update {#update}
+
+`POST /resources/update` changes fields on resources and relations that already exist.
+Identify each node by `id` or `externalId`, each relation by `id`, and name only what you
+want changed, anything you leave out keeps its current value.
+
+Each field is an object carrying a verb rather than a bare value, which is what lets "clear
+this" be said distinctly from "leave it alone":
+
+| Verb | Applies to | Effect |
+| --- | --- | --- |
+| `set` | every field | Replace the value. |
+| `setNull: true` | nullable fields only | Clear the value. `name` and `externalId` are not nullable, so asking to clear either is a `400`. |
+| `add` | `metadata`, `labels` | Merge entries in, keeping the rest. |
+| `remove` | `metadata`, `labels` | Take entries out, keeping the rest. |
+
+```json
+{
+ "nodes": [
+ {
+ "externalId": "klp_pipe_ws_a1212_dl",
+ "update": {
+ "name": { "set": "klp pipe ws-a1212-dl (renamed)" },
+ "metadata": { "add": { "inspected_by": "olav" } },
+ "labels": { "add": ["CRITICAL"] }
+ }
+ }
+ ],
+ "relations": []
+}
+```
+
+Updatable node fields are `externalId`, `name`, `description`, `source`, `dataSetId`,
+`metadata`, `labels` and `geoLocation`. On a relation they are `start`, `end`,
+`fromExternalId`, `toExternalId`, `relationship`, `relationshipId`, `description` and
+`metadata`, so an edge can be retargeted or retyped in place, subject to the same
+[endpoint rules](#create-resources-and-relations) as a create.
+
+Sending both `set` and `setNull` for one field is a `400`: the request is contradictory, so
+it is refused rather than resolved by precedence. `setNull` against `name` or `externalId`
+is also a `400`, for the same reason a create cannot omit them: every resource has to have
+both. Rename with `set` instead.
+Changing `externalId` runs it past the
+[naming policy](./external-ids#the-naming-policy), which reports violations per item in an
+RFC 9457 problem response. The whole batch is **all-or-nothing**.
+
+:::caution A `409` means someone else got there first
+Updates are guarded by optimistic locking. If another request changed or deleted the
+resource while yours was in flight, you get a `409` of type `optimistic-lock` and
+**nothing was written**, no partial application to unpick. Re-read the resource with
+`byids` and retry the update against fresh state.
+
+This is worth designing for rather than retrying blindly: two writers doing
+`metadata: { add: … }` can both succeed after a re-read, whereas two doing
+`metadata: { set: … }` will keep clobbering each other however many times you retry.
+:::
+
+All three clients wrap this: `resources().update(nodes, relations)` in Java,
+`resources.update([...])` in Python, and `resources.update(&updates)` in Rust, each taking the
+per-entry update forms above. The Python and Rust forms (`ResourceUpdate`) cover nodes only:
+neither client has a relation update form, so retarget or retype an edge from Java or over HTTP.
+
+## Delete {#delete}
+
+Delete by id or external id; unknown identifiers are silently skipped. A successful delete
+returns `204` with no body, and deleting something already gone is a no-op, so a retried
+delete needs no bookkeeping.
+
+Deleting a resource takes **all** of its relationships with it, inbound and outbound. That is
+where the one real constraint comes from:
+
+:::caution The graph must stay connected
+A delete is rejected with `409` `would-strand` if it would leave any surviving resource
+unreachable from a root resource, that is, if it would strand part of the graph. The problem
+document names the resources that would be stranded in `blockedBy`
+(`[{ "externalId": "pump_1" }]`), so the fix is either to include them in the same delete or
+to re-attach them through another path first.
+
+Delete a mid-level node in a hierarchy and this is what you will hit: removing a plant that
+holds twenty pumps takes the edges to those pumps with it, stranding all twenty. The check
+is what stops a routine cleanup from quietly orphaning half a site.
+:::
+
+A single safety-check failure rolls the whole batch back, nothing is deleted unless
+everything can be. As with update, a concurrent modification surfaces as a `409` with
+nothing removed.
+
+
+
+
+```java
+client.resources().delete(List.of(IdCollection.createFromExternalId("pump_1")));
+```
+
+
+
+
+```python
+client.resources.delete(["pump_1"])
+```
+
+
+
+
+```rust
+api.resources.delete(&vec![IdAndExtId::from_external_id("pump_1")]).await?;
+```
+
+
+
+
+## Traverse the graph
+
+`fetch-related` walks the graph outward from a starting resource and returns the
+connected sub-graph, a `ResourceNetwork` of `nodes`, the `edges` between them, and
+their `labels`. Traversal is **undirected** and bounded by `depth` (`-1` = the whole
+connected component), optionally filtered to specific relationship types. Use it for
+relationship reasoning, root-cause correlation, blast radius, that a flat lookup
+can't do. See [Correlate alarms with the graph](/guides/correlate-alarms).
+
+| Field | Default | Meaning |
+| --- | --- | --- |
+| `id` / `externalId` | none | Where to start. Supply exactly one. |
+| `depth` | `-1` | Hops to follow. `-1` loads the entire connected component. |
+| `relationshipTypes` | all | Which edge types the walk may follow. |
+| `excludedLabels` | none | Labels the walk neither passes through nor returns, e.g. `["POLICY"]` to keep governance nodes out of an asset view. |
+| `limit` | `5000` | Safety cap on nodes loaded. When the component is bigger, the nearest `limit` nodes come back. |
+
+That `limit` is the one to watch: it is a silent truncation, not an error. On a densely
+connected site an unbounded `depth` will hit 5 000 nodes long before it runs out of graph,
+and what you get back is a *neighbourhood*, not the component you asked for. Bound `depth`
+to 1–3 unless you know the graph is sparse.
+
+Nodes from `fetch-related` and `fetch-nearest` come back [typed by label](#typed-reads) and
+carry the fields the graph mirror holds: `id`, `externalId`, `name`, `description`, `source`,
+`dataSetId`, `labels`, `metadata`, `createdTime` and `lastUpdatedTime`, plus `relatedResources`
+built from the edges of the network you fetched. By type: `isRoot` on resources and assets,
+`geoLocation` on assets (a point only; other geometries are not carried), `unit`, `unitExternalId` and `valueType` on time series, and
+`deactivated` on policies. Fetch by id when you need a field outside that list.
+
+
+
+
+```java
+// convenience: within `depth` hops of an external id
+ResourceNetwork net = client.resources().fetchRelated("sensor_a", 5);
+
+// or the full form, filtering which relationship types to follow
+RelatedResourcesForm form = new RelatedResourcesForm();
+form.setExternalId("sensor_a");
+form.setDepth(5);
+form.setRelationshipTypes(List.of("PART_OF"));
+ResourceNetwork filtered = client.resources().fetchRelated(form);
+
+net.nodes().forEach(n -> System.out.println(n.getExternalId()));
+```
+
+
+
+
+```python
+net = client.resources.fetch_related(
+ external_id="sensor_a", depth=5, relationship_types=["PART_OF"])
+
+for node in net.nodes:
+ print(node.external_id)
+for edge in net.edges:
+ print(edge.start, "->", edge.end, edge.relationship_type)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::RelatedResourcesForm;
+
+let net = api.resources.fetch_related(
+ &RelatedResourcesForm::from_external_id("sensor_a")
+ .with_depth(5)
+ .with_relationship_types(vec!["PART_OF".into()])).await?;
+
+for node in net.nodes() {
+ println!("{}", node.external_id());
+}
+```
+
+
+
+
+### The nearest N of a kind {#fetch-nearest}
+
+`POST /resources/fetch-nearest` answers a question `fetch-related` cannot: *the ten nearest
+time series to this pump*. It walks breadth-first and caps on the number of **matching
+end-nodes**, not on hops or total nodes, so "the 10 nearest `TIMESERIES`" is exactly ten
+however many intermediate nodes lie between them. You get those nodes plus the sub-graph
+connecting them back to the start.
+
+| Field | Default | Meaning |
+| --- | --- | --- |
+| `id` / `externalId` | none | Where to start. Supply exactly one, as for `fetch-related`. |
+| `endLabels` | none | Labels that qualify as a match, e.g. `["TIMESERIES"]`. The walk continues past them. |
+| `limit` | `10` | How many matching end-nodes to return. |
+| `relationshipTypes` | all | Which edge types the walk may follow. |
+| `excludedLabels` | none | Labels never traversed or returned. |
+
+That is the difference worth internalising: with `fetch-related` you pick a radius and find
+out what is inside it, which on an unfamiliar graph is a guess. With `fetch-nearest` you name
+what you are looking for and how many you want, and the radius follows.
+
+The endpoint resolves an `externalId` for you, and so does the Java form. The Python and Rust
+clients build the request from a numeric `id` only, so there resolve an external id with
+`by_ids` first when that is all you have.
+
+
+
+
+```java
+FetchNearestResourcesForm form = new FetchNearestResourcesForm();
+form.setExternalId("pump_1"); // or form.setId(5677892L)
+form.setEndLabels(List.of("TIMESERIES"));
+form.setLimit(10);
+form.setExcludedLabels(List.of("POLICY"));
+
+ResourceNetwork nearest = client.resources().fetchNearest(form);
+```
+
+
+
+
+```python
+nearest = client.resources.fetch_nearest(
+ 5677892, # the Python client takes the numeric id
+ end_labels=["TIMESERIES"],
+ limit=10,
+ excluded_labels=["POLICY"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::resources::FetchNearestResourcesForm;
+
+let nearest = api.resources.fetch_nearest(
+ &FetchNearestResourcesForm::from_id(5677892) // the Rust client takes the numeric id
+ .with_end_labels(vec!["TIMESERIES".into()])
+ .with_limit(10)
+ .with_excluded_labels(vec!["POLICY".into()])).await?;
+```
+
+
+
+
+## Export and import a graph {#graph-transfer}
+
+Two endpoints move a whole connected component between tenants or environments as one file.
+Export starts from one resource, walks outward with no depth limit, and writes every reachable
+node and every relationship between them; import recreates them somewhere else. The file
+references everything by `externalId` and never by numeric id, which is what makes it
+portable: numeric ids are database identities and do not survive the transfer.
+
+| Endpoint | Body | Returns |
+| --- | --- | --- |
+| `GET /resources/export/{id}` | none; `id` is the numeric id of the resource to start from | The file, `application/octet-stream`, as an attachment named `.dhgraph` |
+| `POST /resources/import` | the file, verbatim, as `application/octet-stream` | A JSON summary of what was created and what was skipped |
+
+The file is a gzip-compressed binary, streamed on the way out and decoded incrementally on the
+way in, so neither side holds it whole in memory. Treat it as opaque: the format is versioned
+by the server and is not part of the API contract.
+
+**What the file carries.** Per node: `externalId`, `name`, `description`, `source`, `isRoot`,
+`labels`, `metadata`, the data set it belongs to (by that data set's `externalId`) and, for an
+`ASSET`, a point `geoLocation`; other geometries are not carried. Per relationship: both
+endpoints by `externalId`, the type, `description`, `metadata` and the data set. A data set
+node inside the component is exported as a node like any other, ahead of everything that
+references it.
+
+**What it does not carry.** A time-series's `unit` and `valueType`, so a time-series node in
+the file cannot be created on import and is reported instead; datapoints; events; files.
+
+### Export {#graph-export}
+
+Export needs read access to the data set of the starting resource, and nothing more: the walk
+is [gated on the starting node only](./datasets#access-control), so the file holds every node
+the component reaches. A component of more than **2 000 000 nodes** or **2 000 000
+relationships** is refused with a `400` naming the limit, and nothing is exported partially.
+
+| Status | Meaning |
+| --- | --- |
+| `200` | The file. |
+| `404` | No such resource, or the caller may not read it. |
+| `400` | The component is over the export limit. |
+
+### Import {#graph-import}
+
+Import replays the file through the same pipeline as [create](#create-resources-and-relations),
+so everything a create does, an import does: the [naming policy](./external-ids#the-naming-policy)
+is applied, the data set ACLs are checked, and each committed segment is published to the
+message bus and mirrored into the graph. The caller needs write access to every data set a
+node or a relationship lands in, and the all-data-sets grant for a node that arrives with no
+data set and for any `DATASET` node in the file. A denial is a `403`.
+
+**Skipped, not refused.** A node whose `externalId` already exists in the tenant is left as it
+is, and so is a relationship already present between the same two endpoints with the same
+type. Importing a file back into the tenant it came from is therefore a no-op, and
+re-uploading after a failure is safe. Time-series nodes are skipped and listed by
+`externalId`, and the relationships touching them are skipped with them; to keep those, create
+the series through [`/timeseries`](./timeseries#create-a-series) first and import the same file
+again. A data set reference is resolved by `externalId` against the data sets in the file and
+those already in the tenant; one that resolves nowhere is dropped, and the node is created
+without a data set.
+
+**Segments.** The upload is committed as it streams in, one transaction per 50 000 objects,
+nodes first and relationships after, so memory stays flat however large the file. Each segment
+is atomic on its own: a failure keeps the segments already committed and rejects the rest.
+Because import skips what already exists, re-upload the same file once the cause is fixed and
+it fast-forwards through the committed segments and resumes where it stopped. The response
+counts the segments committed.
+
+```json
+{
+ "nodesCreated": 4210,
+ "relationsCreated": 4209,
+ "nodesSkippedExisting": 3,
+ "nodesSkippedTimeseries": ["pump_1_vibration", "pump_1_temperature"],
+ "relationsSkipped": 2,
+ "dataSetReferencesDropped": 0,
+ "segments": 1,
+ "warnings": []
+}
+```
+
+`warnings` carries [naming-policy warnings](./external-ids#the-naming-policy) exactly as a
+create does, and a naming-policy refusal is the same `400` problem body a create returns.
+
+| Status | Meaning |
+| --- | --- |
+| `200` | The summary above, also when everything was skipped. |
+| `400` | Not a readable graph file, a naming-policy refusal, or a value that failed validation. |
+| `403` | A data set the caller may not write to. |
+| `413` | Over a transfer limit: more than 2 000 000 nodes or relationships in the file, or a file larger than 512 MB. Nothing is imported. |
+
+Like a file upload, the import is exempt from the general
+[request body size](./limits#request-body-size) cap and from the daily ingest byte quota: it
+is consumed as it arrives, and the 512 MB ceiling is checked on the bytes as they stream in.
+The nodes and relationships it creates count against their own daily quotas through the
+ordinary create path. A reverse proxy in front of the API has to pass the upload through
+unbuffered and uncapped, as the shipped nginx examples do for `/resources/import`.
+
+Every client wraps the pair. Export hands back the file as bytes and import takes the same
+bytes, so moving a sub-graph between two tenants is two calls and no temporary file unless you
+want one. Python and Rust also stream either end to or from a path, for a file too large to hold
+in memory.
+
+
+
+
+```java
+import ai.intellistream.datahub.api.graphtransfer.GraphImportResult;
+
+byte[] graph = source.resources().export(5677892L);
+
+GraphImportResult result = target.resources().importGraph(graph);
+System.out.println(result.nodesCreated() + " created, "
+ + result.nodesSkippedExisting() + " already there, "
+ + result.nodesSkippedTimeseries() + " to create as series first");
+```
+
+`GraphImportResult` is a record, so its fields read as `result.nodesCreated()` and so on, one
+per key of the JSON above.
+
+
+
+
+```python
+graph = source.resources.export_graph(5677892) # bytes
+
+result = target.resources.import_graph(graph)
+print(result.nodes_created, "created,",
+ result.nodes_skipped_existing, "already there,",
+ result.nodes_skipped_timeseries, "to create as series first")
+
+# Or through a file, never held whole in memory:
+source.resources.export_graph_to_path(5677892, "plant_oslo.dhgraph")
+target.resources.import_graph_from_path("plant_oslo.dhgraph")
+```
+
+
+
+
+```rust
+let graph: Vec = source.resources.export_graph(5677892).await?;
+
+let result = target.resources.import_graph(graph).await?;
+println!("{} created, {} already there, {:?} to create as series first",
+ result.nodes_created, result.nodes_skipped_existing, result.nodes_skipped_timeseries);
+
+// Or through a file, never held whole in memory:
+source.resources.export_graph_to_path(5677892, "plant_oslo.dhgraph").await?;
+target.resources.import_graph_from_path("plant_oslo.dhgraph").await?;
+```
+
+
+
+
+The export walks the graph, which settles a moment after a write. A component created and
+exported straight away can come back short; read it with `fetch_related` first when that
+matters.
+
+Over plain HTTP, send the bearer token the client already holds:
+
+```bash
+curl -fsS -H "Authorization: Bearer $TOKEN" \
+ -o plant_oslo.dhgraph "$API/resources/export/5677892"
+
+curl -fsS -H "Authorization: Bearer $TOKEN" \
+ -H "Content-Type: application/octet-stream" \
+ --data-binary @plant_oslo.dhgraph "$API/resources/import"
+```
+
+Against the [rate-limit](./limits#rate-limits) budget, export is a read and import a write.
+
+## The `/assets` endpoints {#assets}
+
+An **asset** is the node type that can be a navigation root and the only one that carries a
+`geoLocation`. It has its own endpoint family, and every call in it is the pipeline above with
+the `ASSET` type pinned: the same ACLs, the same [naming policy](./external-ids#the-naming-policy),
+the same [create checks](#create-resources-and-relations), the same status codes. Reach for it
+when a call should only ever see assets, and for `/resources` when one call carries or returns
+several node types.
+
+| Endpoint | Behaves as | Worth knowing |
+| --- | --- | --- |
+| `POST /assets/create` | [create](#create-resources-and-relations) | Nodes only, no `relations` array. `201`, and the echo is asset-shaped. `labels` may be omitted: `ASSET` is added for you. |
+| `GET /assets/{id}` | [look up](#look-up) | One asset, wrapped in `items` like every other read. |
+| `POST /assets/byids` | [look up](#look-up) | Ids that are missing, are not assets, or are not readable are omitted rather than failing the call. |
+| `GET /assets?limit=` | [filter](#filter) | The first `limit` assets, newest first, no criteria. Never returns a `nextCursor`: it is the first page and nothing more. `limit` defaults to 1000 and caps at 10 000. |
+| `POST /assets/filter` | [filter](#filter) | The same criteria, the same paging. A `nodeType` in the body is replaced, see below. |
+| `POST /assets/search` | [search](#search) | Same replacement, and the `filter` block is applied exactly as on [`/resources/search`](#search-filter). |
+| `POST /assets/update` | [update](#update) | Takes `nodes` and `relations` exactly as `/resources/update` does. |
+| `POST` or `DELETE /assets/delete` | [delete](#delete) | `204`, and the same [connectivity check](#delete). |
+
+```http
+POST /assets/create
+{
+ "items": [
+ {
+ "externalId": "plant_oslo",
+ "name": "Oslo Plant",
+ "labels": ["Plant"],
+ "isRoot": true,
+ "geoLocation": { "type": "Point", "coordinates": [10.75, 59.91] }
+ }
+ ]
+}
+```
+
+:::caution `nodeType` in the body is replaced, not merged
+A `nodeType` you send to `/assets/filter` or `/assets/search` is overwritten with `asset`.
+`nodeType` entries are combined with **OR**, so honouring a supplied `["timeseries"]` would
+*widen* a request made to `/assets` into a mixed query instead of narrowing it. Ask
+`/resources/filter` when you want a mixed set.
+:::
+
+:::note A `404` here answers three questions at once
+`GET /assets/{id}` replies the same way to an id that does not exist, an id belonging to a node
+of some other type, and an asset in a data set you may not read. That is deliberate: a
+distinguishable `403` would confirm that an id exists.
+:::
+
+The Java client has `assets()`, whose reads come back as `Asset` rather than the polymorphic
+`NodeModel` that `/resources` returns, so `isRoot` and `geoLocation` are there without a cast:
+
+```java
+import ai.intellistream.datahub.models.Asset;
+
+Asset plant = new Asset();
+plant.setExternalId("plant_oslo");
+plant.setName("Oslo Plant");
+plant.setIsRoot(true);
+client.assets().create(List.of(plant));
+
+DataWrapper newest = client.assets().list(100);
+DataWrapper one = client.assets().getById(5677892L);
+DataWrapper some = client.assets().byIds(List.of(
+ IdCollection.createFromExternalId("plant_oslo")));
+```
+
+`filter`, `search`, `update` and `delete` take the same forms as their `resources()`
+counterparts. The update echo is a `GraphDataWrapper`, not an asset one,
+because an update may touch a relation whose other end is not an asset.
+
+Python and Rust have the same service as `assets` (`client.assets`, `api.assets`), with
+`create`, `get_by_id`, `by_ids`, `list`, `filter`, `search`, `update` and `delete`. Reads come
+back as `Asset`, and `update` echoes typed nodes as on `/resources`: a `GraphResult` in Python,
+a `GraphDataWrapper` in Rust.
+
+## The `/functions` endpoints {#functions}
+
+A **function** is a plain node distinguished by its `FUNCTION` label. Like
+[`/assets`](#assets), every call is the shared pipeline with the `FUNCTION` type pinned: the same
+ACLs, naming policy and status codes. The type is fixed in the query itself, so nothing in the
+body can widen a call to other node types.
+
+| Endpoint | Behaves as | Worth knowing |
+| --- | --- | --- |
+| `POST /functions/create` | [create](#create-resources-and-relations) | Nodes only, no `relations` array. `201`, and the echo is function-shaped. |
+| `GET /functions/{id}` | [look up](#look-up) | One function, wrapped in `items`. |
+| `POST /functions/byids` | [look up](#look-up) | Ids that are missing, are not functions, or are not readable are omitted rather than failing the call. |
+| `GET /functions?limit=` | [filter](#filter) | The first `limit` functions, newest first, no criteria and no `nextCursor`. `limit` defaults to 1000 and caps at 10 000. |
+| `POST /functions/filter` | [filter](#filter) | The criteria every node type shares plus `dataSetId` ([`FunctionFilter`](./filters)), with `sort`, `limit` and `cursor` paging. |
+| `POST /functions/search` | [search](#search) | Free text, with an optional `filter` block of the same criteria ANDed on. |
+| `POST /functions/update` | [update](#update) | `nodes` and `relations`. An entry naming a node of another type is a `404`, and nothing in the batch is written. |
+| `POST` or `DELETE /functions/delete` | [delete](#delete) | `204`. Ids that are not functions are skipped, not deleted. |
+
+```http
+POST /functions/filter
+{
+ "limit": 100,
+ "filter": {
+ "name": ["rolling*"],
+ "labels": ["AGGREGATION"],
+ "dataSetId": [{ "externalId": "plant_a" }]
+ }
+}
+```
+
+`GET /functions/{id}` reports a function you may not read as missing (`404`) rather than
+forbidden, exactly as `GET /assets/{id}` does.
+
+Each client has a typed `functions` service whose reads come back as `Function`:
+
+
+
+
+```java
+import ai.intellistream.datahub.function.Function;
+import ai.intellistream.datahub.models.datafilters.FunctionFilter;
+
+DataWrapper some = client.functions().byIds(List.of(
+ IdCollection.createFromExternalId("fn_rolling_average")));
+
+FunctionFilter criteria = new FunctionFilter();
+criteria.setLabels(List.of("AGGREGATION"));
+DataWrapper page = client.functions().filter(criteria);
+```
+
+`filter` also takes a `FunctionRetreiver` for `limit`, `sort` and `cursor`, and `search` takes a
+`SearchBody`, the same forms as their `assets()` counterparts.
+
+
+
+
+```python
+# entity objects, external-id strings, or numeric ids
+some = client.functions.by_ids(["fn_rolling_average"])
+
+page = client.functions.filter(labels=["AGGREGATION"]) # a Page; page.next_cursor pages on
+hits = client.functions.search("rolling average")
+```
+
+`filter` takes the criteria as keywords or a prepared `FunctionFilter` as `filter=`, never both,
+plus `limit`, `sort_by`, `sort_order` and `cursor`. `search` takes `query`, an optional
+`FunctionFilter` and `limit`.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::NodeFilter;
+use intellistream_datahub_sdk::functions::{FunctionFilter, FunctionFilterForm};
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let some = api.functions.by_ids(&vec![
+ IdAndExtId::from_external_id("fn_rolling_average"),
+]).await?;
+
+let criteria = FunctionFilter {
+ node: NodeFilter { labels: Some(vec!["AGGREGATION".into()]), ..Default::default() },
+ data_set_id: None,
+};
+let page = api.functions.filter(&FunctionFilterForm::new(criteria)).await?;
+let hits = api.functions.search_by_query("rolling average").await?;
+```
+
+`FunctionFilterForm` carries `limit` and a `PageRequest` for `sort` and `cursor`; `search` takes a
+`SearchAndFilterForm`, and `search_by_query` is the shorthand for a bare phrase.
+
+
+
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Get by numeric id | `resources().getById` | `resources.get_by_id` | `resources.get_by_id` |
+| Look up by id / external id | `resources().byIds` | `resources.by_ids` | `resources.by_ids` |
+| List, newest first (`GET /resources`) | `resources().list` | `resources.list` | `resources.list` |
+| Create | `resources().create` | `resources.create` | `resources.create` |
+| Update | `resources().update` | `resources.update` | `resources.update` |
+| Delete | `resources().delete` | `resources.delete` | `resources.delete` |
+| Search | `resources().search` | `resources.search` | `resources.search` |
+| Filter | `resources().filter` | `resources.filter` | `resources.filter` |
+| Traverse (`fetch-related`) | `resources().fetchRelated` | `resources.fetch_related` | `resources.fetch_related` |
+| Nearest N (`fetch-nearest`) | `resources().fetchNearest` | `resources.fetch_nearest` | `resources.fetch_nearest` |
+| [Export / import a graph](#graph-transfer) | `resources().export` / `importGraph` | `resources.export_graph` / `import_graph`, plus `_to_path` / `_from_path` | `resources.export_graph` / `import_graph`, plus `_to_path` / `_from_path` |
+
+
+Relations have their own client surface in all three clients, `edges()` in Java, `edges` in
+Python and Rust. [Edges → client coverage](./edges#client-coverage)
diff --git a/versioned_docs/version-1.0/reference/subscriptions.md b/versioned_docs/version-1.0/reference/subscriptions.md
new file mode 100644
index 0000000..0e0f950
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/subscriptions.md
@@ -0,0 +1,326 @@
+---
+sidebar_position: 9
+title: Subscriptions
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Subscriptions
+
+Durable, fan-out subscriptions over time series, plus **live delivery over a WebSocket**.
+
+## Manage subscriptions
+
+
+
+
+```java
+Subscription sub = new Subscription();
+sub.setExternalId("engine_temps");
+sub.setName("Engine temps");
+sub.setTimeseries(List.of(IdCollection.createFromExternalId("engine_temperature")));
+client.subscriptions().create(List.of(sub));
+
+DataWrapper all = client.subscriptions().filter(new SubscriptionRetriever());
+
+client.subscriptions().delete(List.of(IdCollection.createFromExternalId("engine_temps")));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+sub = intellistream_datahub_sdk.Subscription(
+ external_id="engine_temps",
+ name="Engine temps",
+ timeseries=["engine_temperature"])
+client.subscriptions.create([sub])
+
+all_subs = client.subscriptions.filter()
+
+client.subscriptions.delete(["engine_temps"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::subscriptions::{Subscription, SubscriptionFilterForm};
+use intellistream_datahub_sdk::generic::IdAndExtId;
+
+let sub = Subscription::new(
+ "engine_temps".into(), "Engine temps".into(),
+ vec![IdAndExtId::from_external_id("engine_temperature")]);
+api.subscriptions.create(&sub).await?;
+
+let all = api.subscriptions.filter(&SubscriptionFilterForm::default()).await?;
+
+api.subscriptions.delete(&vec![IdAndExtId::from_external_id("engine_temps")]).await?;
+```
+
+
+
+
+:::note Data set access control
+Creating a subscription requires **read access to every bound series' data set**. If you
+lack read access to any of them, `create` fails with **HTTP 403** and nothing is persisted.
+Access is granted through Keycloak **organization groups**: `/datasets//read` for one
+data set (and everything beneath it), or the wildcard `/datasets/*/read` for all of them.
+[Data set access control →](./datasets#access-control)
+:::
+
+## Find subscriptions {#find}
+
+`GET /subscriptions?limit=` returns the newest `limit` subscriptions your token may read, no
+body required and no cursor. For anything narrower, or to page, `filter` posts
+`POST /subscriptions/filter`, the same envelope every other collection's filter takes, with
+criteria combined by **AND**:
+
+| Criterion | Matching |
+| --- | --- |
+| `id` | Exact numeric ids. |
+| `externalId`, `name` | Patterns, case-insensitive. `*` and `%` are wildcards, `_` is literal, and an entry with no wildcard matches exactly. |
+| `timeseries` | Subscriptions bound to **any** of these time-series, each named by `id`, `externalId`, or both. |
+| `createdTime`, `lastUpdatedTime` | `{ "min": …, "max": … }` bounds. |
+
+Each field above except `createdTime` and `lastUpdatedTime` takes **either a bare value or an
+array**, and the entries of an array are combined with **OR**, exactly as on the other
+collections' filters.
+
+Both endpoints return only subscriptions you can read in full: every timeseries a subscription
+binds must sit in a data set you have read access to, the same rule `create` above and
+[live delivery](#live-delivery) enforce. One ungranted timeseries hides the whole subscription.
+
+The page can be ordered and walked exactly as [timeseries](./timeseries#sorting-and-paging)
+can: `sort` takes `id`, `externalId`, `name`, `createdTime` or `lastUpdatedTime`, and the
+response carries `nextCursor` while more remain.
+
+```json
+{
+ "limit": 100,
+ "filter": {
+ "externalId": ["engine_*"],
+ "timeseries": [{ "externalId": "engine_temperature" }]
+ },
+ "sort": { "property": ["createdTime"], "order": "asc" }
+}
+```
+
+All three clients send the whole body:
+
+| | Java | Python and Rust |
+| --- | --- | --- |
+| Criteria | all of them, on `SubscriptionRetriever` | all of them, on `SubscriptionFilter` |
+| `cursor` | sent | sent |
+| `limit` you did not set | the server's default, 1000 | the server's default, 1000 |
+
+`limit` is capped at 10 000 everywhere. In Python, pass the criteria as keywords,
+`client.subscriptions.filter(timeseries=["engine_temperature"], limit=100)`, or a prepared
+`SubscriptionFilter` as `filter=`, with `sort_by`, `sort_order` and `cursor` as keywords of the
+call; the result is a `Page` carrying `next_cursor`. Rust takes a `SubscriptionFilterForm`, whose
+`paging` carries `sort` and `cursor`.
+
+This replaced `POST /subscriptions/list`, which took a `filter` argument in a shape nothing else
+in the API used: its own default page size, a sort that was not validated, and no cursor, so a
+tenant past the first page could not reach the rest.
+
+## Live delivery
+
+`listen` opens a WebSocket to `/timeseries/datapoints/subscription/listen//...`,
+one path segment per subscription, and authenticates the upgrade request with the same
+`Authorization: Bearer ` header as any REST call. Drive a loop (or, in Java, stream
+messages to a handler), and **ack** the messages you've processed, anything left unacked is
+redelivered on reconnect.
+
+
+
+
+`stream` delivers each message on a dedicated virtual thread and auto-acks once the
+handler returns (throw to nack); the returned handle is `AutoCloseable`:
+
+```java
+import ai.intellistream.datahub.sdk.subscriptions.SubscriptionMessage;
+
+try (var stream = client.subscriptions().listen(List.of("engine_temps"))
+ .stream((SubscriptionMessage msg) -> process(msg.payload()))) {
+ awaitShutdown();
+}
+```
+
+Or drive `poll` yourself, a blocking queue hand-off (not network polling) that returns
+`null` on timeout. Reach for `poll`, or `stream(handler, AckMode.MANUAL)`, when you need
+to ack on your own schedule:
+
+```java
+import ai.intellistream.datahub.sdk.client.SubscriptionListener;
+import java.time.Duration;
+
+try (SubscriptionListener listener = client.subscriptions().listen(List.of("engine_temps"))) {
+ while (running) {
+ SubscriptionMessage msg = listener.poll(Duration.ofSeconds(5));
+ if (msg == null) continue;
+ process(msg.payload());
+ listener.ack(msg.messageId());
+ }
+}
+```
+
+
+
+
+The listener is iterable and a context manager:
+
+```python
+with client.subscriptions.listen(["engine_temps"]) as listener:
+ for msg in listener:
+ process(msg.payload)
+ listener.ack([msg.message_id])
+```
+
+The iterator never ends on its own: a dropped or closed socket is reconnected for you. An error
+is raised from it instead, which ends the `for` loop, but the listener stays usable, so catch it
+and keep iterating. `next_message()` receives one message at a time and raises the same way. On
+an `AsyncDataHubClient`, `listen` returns a `SubscriptionListenerAsync`, driven with
+`async with` and `async for`.
+
+
+
+
+`next().await` yields `Some(Ok(msg))` or `Some(Err(..))` and never `None`: a dropped or closed
+socket is reconnected for you, with a fresh token and the same subscriptions. An `Err` is a
+refused subscription while the socket stays open, a connection refused at the WebSocket cap
+(`ListenError::ConnectionLimit`), a frame that could not be decoded, or a reconnect that ran out
+of retries, and calling `next` again resumes reconnecting:
+
+```rust
+let mut listener = api.subscriptions.listen(&["engine_temps"]).await?;
+while let Some(result) = listener.next().await {
+ match result {
+ Ok(msg) => {
+ process(&msg.payload);
+ listener.ack(&[msg.message_id.as_str()]).await?;
+ }
+ Err(e) => eprintln!("listen error: {}", e),
+ }
+}
+```
+
+
+
+
+Every listener has `ack`/`nack`, `subscribe`/`unsubscribe` to change the live interest set at
+runtime, and `close`. Java receives through `stream` or `poll`, with refusals on `pollError`.
+Python receives by iterating or with `next_message`, Rust with `next`, and both add
+`set_subscriptions`, which replaces the whole interest set.
+
+A frame on the wire carries one subscription's messages:
+
+| Field | Type | |
+| --- | --- | --- |
+| `subscriptionExternalId` | string | The subscription the batch came from. |
+| `messages[].messageId` | string | Opaque. Echo it back in an ack or nack. |
+| `messages[].payload.eventAction` | `CREATE`, `UPDATE`, `DELETE` or `RENAME` | What happened. |
+| `messages[].payload.eventObject` | `DATAPOINTS` | What it happened to. |
+| `messages[].payload.items[]` | object[] | One entry per series: `id` (a JSON string), `externalId`, `valueType`, `datapoints[]`, and optionally `inclusiveBegin` and `exclusiveEnd`. |
+| `messages[].payload.items[].datapoints[]` | `{ timestamp, value }` | `timestamp` is an ISO-8601 UTC string (`2026-08-30T22:00:00Z`); `value` is a string. |
+
+Ack and nack are `{"action": "ack", "messageIds": [...]}` and the same with `"nack"`. The
+clients unpack each entry of `messages` into one message, whose `payload` is the object above.
+
+:::note Refused subscriptions surface as errors
+Live delivery enforces the same data set ACL: to attach a subscription you must be able to read
+**all** of its bound series. A subscription you can't read (`reason: "forbidden"`) or one that
+doesn't exist (`reason: "not-found"`) is refused per-subscription, the connection stays open for
+the subscriptions that did attach. The refusal is surfaced, not swallowed: a `SubscriptionError`
+via `pollError` in Java, an `Err(ListenError::Subscription { .. })` from `next().await` in Rust, and
+an exception raised from the iterator in Python. A refused subscription is therefore visible
+instead of looking like an indefinitely silent stream.
+:::
+
+:::note Sockets and subscriptions are capped
+Ten concurrent connections per organization, ten per user, and ten subscriptions multiplexed
+over one socket, by default. The two refusals behave differently, on purpose:
+
+| Over the cap on | The server | The socket |
+| --- | --- | --- |
+| Connections | Sends `reason: "websocket-limit-reached"`, then closes with **1008** | Closed |
+| Subscriptions on one socket | Sends `reason: "subscription-limit-reached"` for the one that did not fit | **Stays open** |
+
+The second arrives on the same per-subscription error path as `forbidden` and `not-found`
+above, so a client that handles those handles it. Frame shapes and the configurable numbers
+are in [Limits & quotas](./limits#websockets).
+:::
+
+:::tip Acking is at-least-once
+Ack a message only after you've durably handled it. If your process dies before the ack,
+the server redelivers it, so make your handler idempotent.
+:::
+
+## A live tail without a subscription {#live-tail}
+
+`/timeseries/datapoints/listen` streams the datapoints written to a set of series with no
+subscription behind it: nothing to create first, nothing to ack, nothing left behind when the
+socket closes. It is the lighter tool and the weaker one. It starts at the latest point, keeps no
+cursor, and does not replay what was written while the socket was down, so a reader that must
+see every point uses [live delivery](#live-delivery) instead.
+
+The token travels as a WebSocket subprotocol, not a header, because a browser cannot set one on
+the handshake: offer `datahub.bearer.` and `datahub.v1`, and the server echoes `datahub.v1`.
+An `externalIds=a,b,c` query parameter seeds the set, and
+`{"action": "set" | "subscribe" | "unsubscribe", "externalIds": [...]}` changes it. Every change
+is narrowed to the series whose data set you may read, silently: an id you cannot read, or that
+does not exist, is dropped rather than refused. A frame is
+`{"datapoints": [{"externalId", "valueType", "timestamp", "value"}, ...]}`, `value` a string.
+
+A bad token or a missing role is refused after the upgrade, with a **1008** close; a connection
+over the [WebSocket cap](./limits#websockets) gets the error frame described there first.
+
+
+
+
+The Java client does not wrap the tail. Use [live delivery](#live-delivery).
+
+
+
+
+```python
+with client.timeseries.listen_datapoints(["engine_temperature"]) as tail:
+ for point in tail:
+ print(point.external_id, point.timestamp, point.value)
+```
+
+`subscribe`, `unsubscribe` and `set_timeseries` change the set while it runs. On an
+`AsyncDataHubClient` it returns a `DatapointListenerAsync`, driven with `async for`.
+
+
+
+
+```rust
+let mut tail = api.time_series.listen_datapoints(&["engine_temperature"]).await?;
+while let Some(point) = tail.next().await {
+ let point = point?;
+ println!("{} {} {}", point.external_id, point.timestamp, point.value);
+}
+```
+
+`subscribe`, `unsubscribe` and `set_timeseries` change the set while it runs. A dropped
+connection is reopened with a fresh token and the current set; a refusal is returned as an
+error instead of retried.
+
+
+
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Create | `subscriptions().create` | `subscriptions.create` | `subscriptions.create` |
+| List `subscriptions().list` | HTTP | `subscriptions.list(limit=None)` | `subscriptions.list(limit)` |
+| Filter | `subscriptions().filter` | `subscriptions.filter(*, filter=, id=, external_id=, name=, timeseries=, created_time=, last_updated_time=, limit=, sort_by=, sort_order=, cursor=)`, returning a `Page` | `subscriptions.filter` |
+| Delete | `subscriptions().delete` | `subscriptions.delete` | `subscriptions.delete` |
+| Live delivery | `subscriptions().listen` | `subscriptions.listen` (`SubscriptionListenerAsync` with `async for` on the async client) | `subscriptions.listen` |
+| [Live tail](#live-tail), no subscription | — | `timeseries.listen_datapoints` | `time_series.listen_datapoints` |
+
+
diff --git a/versioned_docs/version-1.0/reference/tenant.md b/versioned_docs/version-1.0/reference/tenant.md
new file mode 100644
index 0000000..100c0ac
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/tenant.md
@@ -0,0 +1,198 @@
+---
+sidebar_position: 12
+title: Tenant
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Tenant
+
+Which optional features the tenant your token belongs to may use, and the settings your
+organization administers for itself: `client.tenant()` in Java, `client.tenant` in Python and
+`api.tenant` in Rust. Every answer is a plain object, not an `items` list.
+
+## Feature flags {#features}
+
+`GET /tenant/features` says which optional features the tenant may use. A disabled feature's
+endpoints answer `404` or `403`.
+
+
+
+
+```java
+import ai.intellistream.datahub.tenant.TenantFeatures;
+
+TenantFeatures features = client.tenant().features();
+if (features.isFilesEnabled()) {
+ // show the file browser
+}
+```
+
+
+
+
+```python
+features = client.tenant.features()
+if features.files:
+ ... # show the file browser
+```
+
+
+
+
+```rust
+let features = api.tenant.features().await?;
+if features.files {
+ // show the file browser
+}
+```
+
+
+
+
+| Wire field | Java | Python and Rust | Feature |
+| --- | --- | --- | --- |
+| `files` | `isFilesEnabled()` | `files` | The [file storage](./files) endpoints. |
+| `policy` | `isPolicyFeatureEnabled()` | `policy` | [Policies and governance](./policies). |
+| `streaming` | `isStreamingFeatureEnabled()` | `streaming` | Streaming ingest. |
+| `chat` | `isChatFeatureEnabled()` | `chat` | The AI assistant, configured below. |
+
+`policy`, `streaming` and `chat` fall back to the deployment default when the tenant's
+configuration does not carry them; `files` reads absent as off. Every client reads each flag as
+a plain boolean.
+
+## What you may change {#settings-permissions}
+
+Settings are granted per **scope**, through Keycloak organization groups:
+`/settings//read` and `/settings//write`, or `/settings/*/read` and
+`/settings/*/write` for every scope. Read and write are separate grants, and write does not
+imply read. The one scope is `llm`.
+
+`GET /tenant/settings/permissions` says which grants you hold, with wildcards resolved, so
+every scope the platform knows is listed by name. The endpoint is ungated.
+
+
+
+
+```java
+import ai.intellistream.datahub.models.tenant.SettingsPermission;
+
+Map grants = client.tenant().settingsPermissions();
+SettingsPermission llm = grants.get("llm");
+llm.read();
+llm.write();
+```
+
+
+
+
+```python
+grants = client.tenant.settings_permissions() # dict[str, SettingsPermission]
+grants["llm"].read
+grants["llm"].write
+```
+
+
+
+
+```rust
+let grants = api.tenant.settings_permissions().await?; // HashMap
+grants["llm"].read;
+grants["llm"].write;
+```
+
+
+
+
+## Model configuration {#llm-settings}
+
+`GET /tenant/settings/llm` and `PUT /tenant/settings/llm` read and replace the model your
+organization's assistant runs on. The API key is never returned: `apiKeySet` says whether one
+is stored, and `configured` says whether the settings amount to a model that can be called.
+
+
+
+
+```java
+import ai.intellistream.datahub.models.tenant.TenantLlmSettings;
+import ai.intellistream.datahub.models.tenant.TenantLlmSettingsForm;
+
+TenantLlmSettings current = client.tenant().llmSettings();
+current.apiKeySet();
+current.configured();
+
+TenantLlmSettings saved = client.tenant().updateLlmSettings(new TenantLlmSettingsForm(
+ current.provider(), "the-model-id", null /* apiKey: keep the stored one */,
+ current.baseUrl(), current.reasoningEffort(), "high", current.turnTimeout(),
+ current.maxOutputTokens(), current.maxIterations(), current.instructions()));
+```
+
+`TenantLlmSettings` and `TenantLlmSettingsForm` are records, so the fields read as
+`current.provider()` and so on.
+
+
+
+
+```python
+current = client.tenant.llm_settings()
+current.api_key_set
+current.configured
+
+# Keywords rather than a form. Every one you leave out is sent unset, so carry the rest over;
+# api_key left out keeps the stored credential.
+saved = client.tenant.update_llm_settings(
+ provider=current.provider, model="the-model-id", base_url=current.base_url,
+ reasoning_effort=current.reasoning_effort, effort="high",
+ turn_timeout=current.turn_timeout, max_output_tokens=current.max_output_tokens,
+ max_iterations=current.max_iterations, instructions=current.instructions,
+)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::tenant::TenantLlmSettingsForm;
+
+let current = api.tenant.llm_settings().await?;
+current.api_key_set;
+current.configured;
+
+// A form built from what is stored saves it unchanged, `api_key: None` keeping the credential;
+// change only what you mean to.
+let form = TenantLlmSettingsForm {
+ model: Some("the-model-id".into()),
+ effort: Some("high".into()),
+ ..TenantLlmSettingsForm::from(¤t)
+};
+let saved = api.tenant.update_llm_settings(&form).await?;
+```
+
+
+
+
+`apiKey` is write-only: it is carried on the form and never returned by a read.
+
+The `PUT` replaces the configuration with what you send, absent meaning unset, except for
+`apiKey`: absent or blank leaves the stored credential alone, and only a non-blank value
+overwrites it.
+
+A field you leave unset falls back to the deployment default. The fields that identify the
+model have no default: a `provider`, a `model`, and an `apiKey` or a `baseUrl` depending on the
+provider.
+
+`effort` accepts `low`, `medium`, `high`, `xhigh` or `max`, weakest first, and the list is on
+the record as `TenantLlmSettings.EFFORT_LEVELS`.
+
+The API instance that served the write refreshes immediately. Other instances and services
+cache the tenant registry and pick the change up within five minutes.
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Feature flags (`GET /tenant/features`) | `tenant().features` | `tenant.features` | `tenant.features` |
+| Settings permissions (`GET /tenant/settings/permissions`) | `tenant().settingsPermissions` | `tenant.settings_permissions` | `tenant.settings_permissions` |
+| Read model configuration (`GET /tenant/settings/llm`) | `tenant().llmSettings` | `tenant.llm_settings` | `tenant.llm_settings` |
+| Change model configuration (`PUT /tenant/settings/llm`) | `tenant().updateLlmSettings` | `tenant.update_llm_settings` | `tenant.update_llm_settings` |
diff --git a/versioned_docs/version-1.0/reference/timeseries.md b/versioned_docs/version-1.0/reference/timeseries.md
new file mode 100644
index 0000000..9cdd395
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/timeseries.md
@@ -0,0 +1,990 @@
+---
+sidebar_position: 4
+title: Time series
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Time series
+
+Time series metadata, datapoint retrieval, and datapoint ingestion (single-request or
+high-throughput).
+
+A series' `externalId` identifies it: unique per tenant, compared without case, and stored
+exactly as you send it. [External ids & naming →](./external-ids)
+
+## Create a series
+
+
+
+
+```java
+Timeseries series = new Timeseries();
+series.setExternalId("engine_temperature");
+series.setName("Engine temperature");
+series.setUnit("celsius");
+
+client.timeseries().create(List.of(series));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+ts = intellistream_datahub_sdk.TimeSeries(
+ external_id="engine_temperature",
+ name="Engine temperature",
+ unit="celsius",
+ value_type="float") # Python's default is bigint
+
+client.timeseries.create([ts])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+let mut ts = TimeSeries::new("engine_temperature", "Engine temperature");
+ts.unit = Some("celsius".into());
+ts.set_value_type("float"); // decimal readings
+api.time_series.create_one(&ts).await?;
+```
+
+
+
+
+A create needs a `unit` (free text), a `unitExternalId` from the [unit catalogue](./units), or
+both. Given only `unitExternalId`, the series gets its `unit` from the catalogue entry; a
+`unitExternalId` the catalogue does not have is a `400` on that field.
+[Tag a series with a unit →](../guides/work-with-units#tag-a-series-with-a-unit)
+
+## Value types
+
+Every series has a **value type** that decides how its datapoints are stored. A create that
+sends no value type gets floating-point (`float32`), right for most sensor readings. Whether
+yours sends one depends on the client:
+
+- **Java** sends none unless you set it.
+- **Python**'s `TimeSeries` always sends one and defaults to `bigint`, which refuses decimal
+ values. Pass `value_type="float"` for readings.
+- **Rust**'s `TimeSeries::new` sets `float`.
+
+So the create above gets `float32` from Java and `float` from Python and Rust, and each accepts
+decimal values as-is. Set the type explicitly when you need something else:
+
+| Value type | Use it for |
+| --- | --- |
+| `float32` *(default)* | Sensor readings, 32-bit precision is plenty. |
+| `float` | Double-precision floating point. |
+| `numeric` / `decimal32` | **Exact decimals**, money, lab values, stored without floating-point rounding. Send the value's string form, see below. |
+| `bigint` | Whole numbers (counts, integer statuses). |
+| `text` | Non-numeric string values. |
+| `mixed` | Heterogeneous values in one series. |
+
+A float written to a `bigint` series is [rejected](#write-datapoints), so pick the type that
+matches the data.
+
+Python can set only three of these: `TimeSeries` accepts `bigint`, `float` (or its alias
+`decimal`) and `text`, and raises `ValueError` for `float32`, `numeric`, `decimal32` and
+`mixed`. Create a series of those types from Java, Rust or `POST /timeseries/create`; Python
+reads and writes it like any other.
+
+### Ask which type suits a unit {#value-type-hint}
+
+`GET /timeseries/recommend-value-type/{unitExternalId}` suggests the `valueType` that gives the
+best storage compression for a unit while still representing it faithfully. The mapping is a
+heuristic: small low-precision ranges map to `DECIMAL32`, wide-magnitude analog values to
+`FLOAT32`, and a unit it does not know gets a compact default with `recognized` false. Use it to
+pre-select the type in a create form; it is advice, not a constraint.
+
+
+
+
+```java
+import ai.intellistream.datahub.api.responses.ValueTypeRecommendation;
+
+ValueTypeRecommendation hint = client.timeseries().recommendValueType("temperature_deg_c");
+hint.getRecommendedValueType(); // the suggested type, upper case
+hint.getReason(); // why, in a sentence you can show a user
+hint.isRecognized(); // false means you got the generic default
+```
+
+
+
+
+```python
+hint = client.timeseries.recommend_value_type("temperature_deg_c")
+hint.recommended_value_type # the suggested type, upper case
+hint.reason # why, in a sentence you can show a user
+hint.recognized # False means you got the generic default
+```
+
+
+
+
+```rust
+let hint = api.time_series.recommend_value_type("temperature_deg_c").await?;
+hint.recommended_value_type; // the suggested type, upper case
+hint.reason; // why, in a sentence you can show a user
+hint.recognized; // false means you got the generic default
+```
+
+
+
+
+Every value crosses the wire as a string, whatever the type. For an exact decimal, send the
+string form rather than a float: `Datapoint.of(ts, "12.34")` in Java, `DatapointString(ts,
+"12.34")` with `insert_datapoints` in Python (`insert_from_lists` takes floats and converts
+them), and `DatapointString` in Rust.
+
+`text` and `mixed` also carry a tighter write cap than the numeric types: **10 000 datapoints
+per collection** rather than 100 000, and a lifetime ceiling of their own. The check runs once
+the series' value type is resolved, so it names the series type rather than a field. See
+[Limits & quotas](./limits#batch-caps).
+
+For a value that must reconcile exactly, use `numeric`:
+
+
+
+
+```java
+Timeseries price = new Timeseries();
+price.setExternalId("book_value_usd");
+price.setName("Book value (USD)");
+price.setUnit("usd");
+price.setValueType("numeric"); // exact decimals, no float rounding
+
+client.timeseries().create(List.of(price));
+```
+
+
+
+
+Python cannot create a `numeric` series: `value_type="numeric"` raises `ValueError` (see
+above). Create it from Java or Rust, then write to it from Python with the value's string form:
+
+```python
+import pandas as pd
+
+client.timeseries.insert_datapoints([intellistream_datahub_sdk.DatapointsCollectionString(
+ [intellistream_datahub_sdk.DatapointString(pd.Timestamp.now(tz="UTC"), "12.34")],
+ ts="book_value_usd")])
+```
+
+
+
+
+```rust
+let mut price = TimeSeries::new("book_value_usd", "Book value (USD)");
+price.unit = Some("usd".into());
+price.set_value_type("numeric"); // exact decimals, no float rounding
+api.time_series.create_one(&price).await?;
+```
+
+
+
+
+## Filter series
+
+`POST /timeseries/filter` finds series by structured criteria. Everything you supply is
+combined with AND, a series must match every criterion to be included.
+
+| Criterion | Matching |
+| --- | --- |
+| `dataSetId` | The data set **and every data set beneath it** in the data set hierarchy, so a series attached to a child (or grandchild, …) data set matches too. Each entry is `{"id": …}` or `{"externalId": …}`. |
+| `unit` | Pattern, case-insensitive. `*` and `%` are wildcards, `_` is literal (`"cel%"`). |
+| `unitExternalId` | Pattern on the unit-catalogue external id (e.g. `temperature_deg_c`), on the same rules. |
+| `valueType` | Exact, case-insensitive, against the closed catalogue: `BIGINT`, `FLOAT`, `FLOAT32`, `NUMERIC`, `DECIMAL32`, `TEXT`, `MIXED`. Not a pattern. |
+| `id` | The shared node criterion. Exact numeric ids, not patterns. |
+| `externalId`, `name`, `source` | The shared node criteria. Patterns, on the same rules as `unit`. |
+| `labels` | Series carrying **all** of these labels. |
+| `metadata` | Every key/value pair given must be present. **A null value matches the key alone**, whatever it holds. |
+| `createdTime`, `lastUpdatedTime` | `{ "min": …, "max": … }` bounds. |
+
+Each field above except `labels`, `metadata` and the two time windows takes **either a bare value or an array**, and the
+entries of an array are combined with **OR**. That is why they are named in the singular:
+`"unit": "celsius"` is the common case, and `"unit": ["celsius", "kelvin"]` asks for either.
+`labels` and `metadata` require **all** entries to match and keep their plural names for that
+reason.
+
+Results come newest first unless you ask for another order, see
+[sorting and paging](#sorting-and-paging): capped by `limit` (default 1000, max 10000; a value
+`<= 0` falls back to the default, and above the ceiling is a 400). Series in data sets you lack
+read access to are silently omitted, the result is what your token may see, not an error. For
+free-text lookups use `POST /timeseries/search` instead.
+
+
+
+
+```java
+import ai.intellistream.datahub.models.datafilters.TimeseriesFilter;
+
+TimeseriesFilter criteria = new TimeseriesFilter();
+criteria.setDataSetId(List.of(IdCollection.createFromId(12L))); // and every data set beneath it
+criteria.setUnit(List.of("celsius"));
+
+DataWrapper series = client.timeseries().filter(criteria);
+```
+
+Pass a `TimeseriesRetreiver` instead of the bare criteria to set an explicit `limit`.
+
+
+
+
+```python
+series = client.timeseries.filter(
+ data_set_id=[12], # this data set and every data set beneath it
+ unit="celsius", # a pattern field also takes a bare value
+ limit=100)
+```
+
+The criteria are keywords, or a prepared `filter=intellistream_datahub_sdk.TimeSeriesFilter(...)`
+(passing both is a `TypeError`). `limit`, `sort_by`, `sort_order` and `cursor` are always
+arguments of the call, so one `TimeSeriesFilter` can be reused across `filter()` and `search()`.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::IdAndExtId;
+use intellistream_datahub_sdk::{TimeSeriesFilter, TimeSeriesFilterForm};
+
+let criteria = TimeSeriesFilter {
+ data_set_id: Some(vec![IdAndExtId::from_id(12)]), // and every data set beneath it
+ unit: Some(vec!["celsius".into()]),
+ ..Default::default()
+};
+let series = api.time_series.filter(&TimeSeriesFilterForm::new(criteria, Some(100))).await?;
+```
+
+
+
+
+The hierarchy expansion is what makes "master" data sets useful: filter on the top-level
+data set of a site or project and you get the series of the whole family beneath it, without
+knowing (or maintaining a list of) the sub-data sets.
+
+## Search series {#search}
+
+`POST /timeseries/search` is a free-text search over series. The phrase is matched against `name`,
+`externalId` and `description`, fuzzily and word-aware, so `temp` also finds `temperature` and
+`tempered`. `limit` defaults to 100 and caps at **1 000**, lower than the 10 000 of
+[`filter`](#filter-series), and `query` must be 3 to 140 characters.
+
+Results are **ranked by relevance** (`ts_rank`), strongest match first, with `id` as a tie-break so
+equal-scoring rows keep a stable order and repeated identical searches agree. Ranking means the
+database scores and sorts every match before applying `limit`, so a very broad phrase costs more
+than a narrow one.
+
+`filter` is optional and takes the same `TimeseriesFilter` as `POST /timeseries/filter`. It only
+ever removes matches: the phrase decides what the candidates are. `dataSetId` is applied by the
+search query itself, everything else is applied to the hits afterwards.
+
+```json
+{
+ "search": { "query": "temperature" },
+ "filter": { "unit": ["deg_*"], "valueType": ["FLOAT"] },
+ "limit": 50
+}
+```
+
+:::caution `search.name` and `search.description` are rejected
+They are unknown fields, refused with a `400`; use `filter.name` and `query`.
+:::
+
+## Sorting and paging {#sorting-and-paging}
+
+The three node filters, `/timeseries/filter`, `/resources/filter` and `/datasets/filter`,
+share this contract. (`/events/filter` works the same way over its own columns; see
+[events](./events#paging).)
+
+Order a page with `sort`, over `id`, `externalId`, `name`, `source`, `description`,
+`createdTime`, `lastUpdatedTime` or `dataSetId`. The default is `createdTime` descending,
+newest created first.
+
+```json
+{ "filter": { "unit": "celsius" },
+ "sort": { "property": ["name"], "order": "asc" },
+ "limit": 100 }
+```
+
+Only the **first** recognised `property` is used, and `id` is appended behind it: a sort column alone is not
+a position unless it is unique, and a page boundary inside a run of equal values repeats or drops
+exactly those rows. An unrecognised property falls back to the default rather than being
+rejected, and any `order` that is not `desc`, compared without case, sorts ascending. Nulls sort last ascending,
+first descending, most of these columns are nullable, since every node type shares one table.
+
+A page that has a successor carries a `nextCursor`. Echo it back as `cursor` to continue:
+
+```json
+{ "filter": { "unit": "celsius" },
+ "sort": { "property": ["name"], "order": "asc" },
+ "cursor": "bmFtZXxhc2N8N3x2YQ",
+ "limit": 100 }
+```
+
+The cursor is **opaque**, base64 of an encoding carrying the sort, the boundary value
+and the id, so do not build or parse one. Send it with the **same** sort that produced it; a
+cursor is a position in one particular order, and continuing it under another is refused. One
+that does not decode is refused with a `400` of `type: ".../errors/malformed-cursor"`.
+
+`nextCursor` is absent on a short page, so "keep going while it is present" is the whole loop. A
+full page may still be the last, so a complete walk ends with one empty request.
+
+
+
+
+```java
+TimeseriesRetreiver retriever = new TimeseriesRetreiver();
+retriever.getSort().setProperty(List.of("name"));
+retriever.getSort().setOrder("asc");
+
+DataWrapper page = client.timeseries().filter(retriever);
+while (page.getNextCursor() != null) {
+ retriever.setCursor(page.getNextCursor());
+ page = client.timeseries().filter(retriever);
+}
+```
+
+
+
+
+```python
+cursor = None
+while True:
+ page = client.timeseries.filter(
+ unit="celsius", limit=100, sort_by="name", sort_order="asc", cursor=cursor)
+ for ts in page:
+ ...
+ cursor = page.next_cursor
+ if cursor is None:
+ break
+```
+
+`filter()` returns a `Page`: iterable, indexable and sized like a list, carrying `.next_cursor`
+and `.items` (a real `list`). It is not a `list` subclass, so `isinstance(page, list)` is `False`.
+
+
+
+
+```rust
+use intellistream_datahub_sdk::filters::PageRequest;
+use intellistream_datahub_sdk::{TimeSeriesFilter, TimeSeriesFilterForm};
+
+let mut paging = PageRequest::asc("name");
+loop {
+ let form = TimeSeriesFilterForm::new(TimeSeriesFilter::default(), Some(100))
+ .with_paging(paging.clone());
+ let page = api.time_series.filter(&form).await?;
+ // ... use page.get_items()
+ match page.next_cursor() {
+ Some(cursor) => paging = PageRequest::asc("name").after(cursor),
+ None => break,
+ }
+}
+```
+
+
+
+
+## List and update series {#list-and-update}
+
+`GET /timeseries/{id}` reads one series by numeric id and answers `404` when there is none, or
+none you may read; `timeseries().getById` in Java, `timeseries.get_by_id` in Python and
+`time_series.get_by_id` in Rust. [Look up](#filter-series) by external id, or several at once,
+with `by_ids`, which leaves out what it cannot find rather than failing.
+
+`GET /timeseries?limit=` is the cheap "what have I got" read: the first `limit` series, newest
+created first, no criteria. Add `dataSetId` (an id or an external id) to restrict it to one data
+set and everything beneath it in the `BELONGS_TO` hierarchy. Python's and Rust's `list` take `limit`
+alone. Anything narrower belongs in
+[filter](#filter-series).
+
+`POST /timeseries/update` changes fields on series that already exist. It is a partial update:
+only the fields named in each entry's `update` block change, and the series is identified by
+`id` or `externalId`. Changing `valueType` on a series that already holds data is refused,
+because the stored points would no longer parse; create a new series instead. Setting
+`unitExternalId` to a blank value or one the [unit catalogue](./units) does not have is a `400`
+on `unitExternalId`, before anything is written; `setNull` still clears it.
+
+```java
+import ai.intellistream.datahub.timeseries.UpdateTimeseries;
+
+DataWrapper newest = client.timeseries().list(100);
+
+// Java takes the numeric dataSetId, so look the data set up by external id first.
+long engineData = client.datasets().byIds(List.of(
+ IdCollection.createFromExternalId("engine_data"))).getItems().get(0).getId();
+DataWrapper inSet = client.timeseries().list(100, engineData);
+
+UpdateTimeseries change = new UpdateTimeseries().setExternalId("engine_temperature");
+change.getUpdate().getDescription().set("Engine block temperature, port side");
+client.timeseries().update(List.of(change));
+```
+
+The Python and Rust method names are in the [coverage table](#client-coverage).
+
+## Delete a series
+
+Deletes the series and its datapoints. Remove any referencing subscriptions (and edges) first, or
+the backend responds 409.
+
+The definition is gone when the call returns; the datapoint purge is handed off and completes
+shortly after. Nothing can read those datapoints in the meantime, because every read resolves the
+series first.
+
+
+
+
+```java
+import ai.intellistream.datahub.models.IdCollection;
+
+client.timeseries().delete(List.of(IdCollection.createFromExternalId("engine_temperature")));
+```
+
+
+
+
+```python
+client.timeseries.delete(["engine_temperature"])
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+
+api.time_series
+ .delete(&DataWrapper::from_vec(vec![IdAndExtId::from_external_id("engine_temperature")]))
+ .await?;
+```
+
+
+
+
+## Write datapoints
+
+A datapoint is a `(timestamp, value)` pair grouped under a series' external id. A `value` is
+capped at **64 characters** on the wire, which fits any number and any status code, and one
+collection holds at most **100 000** datapoints (10 000 for a `text` or `mixed` series).
+
+A `timestamp` is a millisecond epoch or an ISO-8601 string. It carries
+no zone of its own, so it is UTC, and the accepted range is 12 to 14 digits. This is to stop people from accidentally sending epoch seconds.
+
+An ISO-8601 string keeps whatever offset or zone it carries, the offset is not optional.
+
+`POST /timeseries/data` refuses a write it cannot store:
+
+| Refused | Answered with |
+| --- | --- |
+| A malformed body: no `datapoints`, a datapoint with no `timestamp`, a blank `value` | `400`, `fields` naming each offender |
+| A `timestamp` in neither [accepted form](./client#timestamps) | `422` of `type: ".../errors/invalid-timestamp"`, `fields` naming `timestamp` and the series' `externalId` |
+| A `value` that does not parse as the series' [value type](#value-types): a float or text to a `bigint` series, `NaN`, `Infinity` or a hex literal to a `numeric` one | `422` of `type: ".../errors/invalid-datapoint"` |
+
+Both `422`s carry `retry: change-request` and used to be a `500`, which the ingest paths retried. A
+`422` is [surfaced to you](./limits#sdk-behaviour) instead.
+
+
+
+
+Timestamps are **epoch milliseconds** as strings:
+
+```java
+DatapointsCollection collection = new DatapointsCollection();
+collection.setExternalId("engine_temperature");
+collection.setDatapoints(List.of(
+ new DatapointString(String.valueOf(System.currentTimeMillis()), "92.4")));
+
+client.timeseries().insertDatapoints(List.of(collection));
+```
+
+
+
+
+Pass timezone-aware timestamps (pandas or `datetime`); the SDK converts to UTC:
+
+```python
+import pandas as pd
+
+client.timeseries.insert_from_lists(
+ timestamps=pd.date_range("2026-01-01", periods=3, freq="h", tz="UTC"),
+ values=[92.4, 92.6, 92.1],
+ ts=ts)
+```
+
+
+
+
+```rust
+use chrono::Utc;
+
+api.time_series
+ .insert_datapoint(None, Some("engine_temperature".into()), Utc::now(), "92.4".into())
+ .await?;
+```
+
+
+
+
+## High-throughput ingestion
+
+For large or unbounded volumes the SDK chunks and sends in bulk. See the
+[ingestion guide](/guides/ingest-timeseries) for the full story. Every client also has a
+[binary path](#binary-ingest) for sustained volume (Python on its synchronous client only).
+
+:::tip Survive outages with durable buffering
+Enable [durable buffering](/reference/client#durable-ingest-buffering) on the client and datapoint
+ingestion that can't reach the API spools to disk and flushes on the next call, bounded by a time
+and/or size window. Retries are idempotent (datapoints dedup on `(series, timestamp)`). The spool
+covers the JSON path only, not [binary ingest](#binary-ingest).
+:::
+
+
+
+
+`ingest` chunks, parallelises and retries, returning an [`IngestResult`](#ingestresult)
+tuned with [`IngestOptions`](#ingestoptions):
+
+```java
+IngestResult result = client.timeseries().ingest(data,
+ IngestOptions.builder()
+ .batchSize(10_000) // datapoints per request
+ .parallelism(16) // concurrent in-flight requests
+ .maxRetries(3)
+ .build());
+
+System.out.printf("ingested %,d, failed %,d%n", result.succeeded(), result.failed());
+```
+
+
+
+
+`insert_from_lists` takes whole arrays (NumPy / pandas) and splits them into requests for you:
+
+```python
+import numpy as np, pandas as pd
+
+client.timeseries.insert_from_lists(
+ timestamps=pd.date_range("2026-01-01", periods=1_000_000, freq="s", tz="UTC"),
+ values=np.random.rand(1_000_000),
+ ts=ts)
+```
+
+
+
+
+`insert_datapoints` splits a large input into requests for you:
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, DatapointsCollection, DatapointString};
+
+let mut dw = DataWrapper::new();
+dw.add_item(DatapointsCollection {
+ external_id: Some("engine_temperature".into()),
+ datapoints: points, // Vec
+ ..Default::default()
+});
+api.time_series.insert_datapoints(&mut dw).await?;
+```
+
+
+
+
+Python runs on the Rust implementation, so the two behave alike. A call is split into requests
+of at most 100 000 datapoints, and up to four are in flight at once. Change that with
+`DATAPOINT_INSERT_PARALLELISM` in the environment, which `create_api_service()` and Python's
+`from_env()` read, or with `set_datapoint_insert_parallelism` on a Rust `DataHubConfig`. Retries
+go through the [durable spool](./client#durable-ingest-buffering): with buffering on, the requests
+go one at a time, a transient failure (408, 429, 5xx, network, or a `401`/`403`) writes the input to
+disk, and the next insert sends the spool again, oldest first, before its own data. With buffering
+off, the first request to fail ends the call with its error and the requests not yet sent are
+dropped, so part of the input may have landed. Sending all of it again is safe either way, since
+datapoints dedup on `(series, timestamp)`. The split counts datapoints and ignores the value type,
+so keep a `text` or `mixed` series to 10 000 points per call.
+
+## Binary ingest {#binary-ingest}
+
+`ingestBinary` in Java, and `insert_datapoints_binary` in Python and Rust, send the same
+datapoints as zstd-compressed Arrow frames to [`POST /timeseries/data/binary`](./binary-datapoints)
+instead of JSON. Values are parsed, sorted and de-duplicated on the client, and one request
+carries a million points or more, so it is the path for sustained volume. The JSON path stays
+right for modest volume and for the [durable spool](./client#durable-ingest-buffering), which does
+not apply here. Python has it on the synchronous `DataHubClient` only; `AsyncDataHubClient` does
+not.
+
+| | JSON (`ingest`, `insert_datapoints`) | Binary (`ingestBinary`, `insert_datapoints_binary`) |
+| --- | --- | --- |
+| Points per request | 10 000 in Java, 100 000 in Python and Rust | up to 1 000 000 in Java, up to 32 frames in Python and Rust; frames of at most 100 000 |
+| Series named by | external id | internal id, resolved once through `/timeseries/byids` and cached for the life of the client |
+| Access needed on the data set | write | write, and **read** for the lookup ([access control](./datasets#access-control)) |
+| Durable spool | yes | no |
+| A request the server refuses | that batch fails | nothing was inserted; the whole request is retried |
+
+
+
+
+```java
+import ai.intellistream.datahub.sdk.ingest.BinaryIngestOptions;
+
+Map> byExternalId = Map.of(
+ "engine_temperature", List.of(Datapoint.of(Instant.now(), 92.4)),
+ "engine_rpm", List.of(Datapoint.of(Instant.now(), 1500L)));
+
+IngestResult result = client.timeseries().ingestBinary(byExternalId);
+
+// or tuned: zstd level 1, 3 or 9 (default 9; the client pays for it)
+client.timeseries().ingestBinary(byExternalId,
+ BinaryIngestOptions.builder().zstdLevel(3).build());
+```
+
+`ingestBinary` also takes a `List`, with or without options. Two things
+are caught before any request is sent and land in the [`IngestResult`](#ingestresult) as a
+`BatchError`: a series that does not exist or is not readable (`statusCode` 404) and a value
+that does not fit the series' type (422). A request the server refuses as `unknown-timeseries`
+or `external-id-mismatch`, a series removed or renamed since it was cached, is rebuilt once
+after re-resolving; `429`, `5xx` and network failures are retried as for `ingest`.
+
+
+
+
+`insert_datapoints_binary` takes the same collections as `insert_datapoints`, and
+`insert_from_lists_binary` the same arrays as `insert_from_lists`. Both are on the synchronous
+client only:
+
+```python
+import pandas as pd
+
+now = pd.Timestamp.now(tz="UTC")
+client.timeseries.insert_datapoints_binary([
+ intellistream_datahub_sdk.DatapointsCollectionString(
+ [intellistream_datahub_sdk.DatapointString(now, "92.4")], ts="engine_temperature"),
+ intellistream_datahub_sdk.DatapointsCollectionString(
+ [intellistream_datahub_sdk.DatapointString(now, "1500")], ts="engine_rpm"),
+])
+
+# or from arrays, tuned: zstd level 1, 3 or 9 (default 9; the client pays for it)
+client.timeseries.insert_from_lists_binary(
+ timestamps=[now], values=[92.4], ts="engine_temperature", zstd_level=3)
+```
+
+Both run on the Rust implementation, so they check and retry as the Rust tab describes and raise
+`DataHubException` where Rust returns an error. `zstd_level` is the only option Python passes.
+
+
+
+
+`insert_datapoints_binary` takes the same `DataWrapper` as `insert_datapoints`, with timestamps
+in epoch milliseconds, which `DatapointString::from_datetime` writes. An ISO string is refused
+before any request.
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::generic::{DataWrapper, DatapointString, DatapointsCollection};
+use intellistream_datahub_sdk::timeseries::BinaryIngestOptions;
+
+let now = Utc::now();
+let mut by_external_id = DataWrapper::new();
+for (external_id, value) in [("engine_temperature", "92.4"), ("engine_rpm", "1500")] {
+ let mut collection = DatapointsCollection::from_external_id(external_id);
+ collection.datapoints.push(DatapointString::from_datetime(now, value)); // epoch millis
+ by_external_id.add_item(collection);
+}
+
+api.time_series
+ .insert_datapoints_binary(&by_external_id, &BinaryIngestOptions::default())
+ .await?;
+
+// or tuned: zstd level 1, 3 or 9 (default 9; the client pays for it)
+api.time_series
+ .insert_datapoints_binary(&by_external_id, &BinaryIngestOptions::new().zstd_level(3))
+ .await?;
+```
+
+Two things are caught before any request is sent and returned as the error: a series that does
+not exist or is not readable (404, naming it) and a value that does not fit the series' type
+(422). A request the server refuses as `unknown-timeseries` or `external-id-mismatch` is rebuilt
+once after re-resolving every series. `429`, `5xx` and network failures are retried up to
+`max_retries` times, waiting a second longer before each attempt.
+
+
+
+
+For many small inserts, Java has a buffer that batches them into frames and sends once
+**10 000 points** have accumulated or the oldest is **200 ms** old, whichever comes first. `add`
+never blocks on the network; flushes run on a thread of their own. Python and Rust have no
+binary buffer: collect the readings and call `insert_datapoints_binary` yourself.
+
+```java
+import ai.intellistream.datahub.sdk.client.BinaryIngestBuffer;
+
+try (BinaryIngestBuffer buffer = client.timeseries().binaryBuffer()) { // 10 000 points or 200 ms
+ buffer.add("engine_temperature", Instant.now(), 92.4);
+ buffer.add("engine_rpm", Instant.now(), 1500L);
+} // close flushes what is left
+```
+
+`binaryBuffer(options, maxPoints, maxAge, onFlush)` sets the thresholds and a
+`Consumer` that receives each flush's result. `flush()` sends what is pending on the
+calling thread, `pending()` counts what is waiting, and `lastResult()` is the most recent
+outcome. `add` takes a `double`, `long` or `String` value, or a `Datapoint`.
+
+### BinaryIngestOptions
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `zstdLevel` | `9` | `1`, `3` or `9`. Compression is never off; the level trades client CPU against bytes on the wire. |
+| `compressInParallel` | `true` | Compress frames on several threads. `false` compresses one at a time, for a small device: a level-9 context needs about 30 MB. |
+| `parallelism` | `8` | Concurrent in-flight requests. |
+| `maxRetries` | `3` | Retries for transient failures (HTTP 429/5xx, network). |
+| `failFast` | `false` | If `true`, abort on the first failed request instead of collecting errors. |
+
+`BinaryIngestOptions.defaults()` returns the defaults. The wire format, caps and every
+`reason` the endpoint answers with are on [Binary datapoint frames](./binary-datapoints).
+
+Rust's `BinaryIngestOptions` has three knobs, each a public field and a builder method of the
+same name: `zstd_level` (default `9`, as above), `max_retries` (`3`, for 429, 5xx and network
+failures) and `request_concurrency` (`4`, requests in flight when a call needs more than one).
+Frames are always compressed in parallel, and the first failed request ends the call, as
+`failFast` would. Python passes `zstd_level` and leaves the other two at their defaults.
+
+## Retrieve datapoints
+
+`POST /timeseries/data/list` takes a list of `RetrieveFilter` items, one per series and
+window:
+
+| Field | Meaning |
+| --- | --- |
+| `id` / `externalId` | The series. |
+| `start`, `end` | [ISO-8601 or epoch millis](./client#timestamps). At least one is required. |
+| `limit` | Datapoints per page, default 100, at most 100 000. |
+| `aggregates` | Any of `avg`, `sum`, `min`, `max`, lower-case. A name outside that set is dropped, not rejected. `avg` comes back as `average`. |
+| `granularity` | A number and a unit: `s`, `m`, `h`, `d`, `w`, `mo`, `y`, or the words `sec`, `min`, `hour`, `day`, `week`, `month`, `year` and their plurals (`15m`, `1h`, `30 min`). Bare `m` is a minute; a month is `mo`. Required when `aggregates` is set. |
+| `includeOutsidePoints`, `mergeDuplicates` | Booleans, default `false`. The Python and Rust `RetrieveFilter` do not carry them, so those clients always get the default. |
+| `cursor` | The previous page's `nextCursor`, carried on each returned collection. |
+
+A datapoint comes back with an ISO-8601 UTC `timestamp` and a `value`. Python reads the
+timestamp as a timezone-aware `datetime.datetime` and the value as `float | None`; Rust as
+`chrono::DateTime` and `Option`; Java keeps both as strings on `DatapointString`.
+
+
+
+
+```java
+import java.time.ZonedDateTime;
+
+RetrieveFilter series = new RetrieveFilter();
+series.setExternalId("engine_temperature");
+series.setStart(ZonedDateTime.now().minusHours(1));
+series.setEnd(ZonedDateTime.now());
+series.setLimit(1000);
+
+DataRetriever request = new DataRetriever<>();
+request.setItems(List.of(series));
+
+DataWrapper points = client.timeseries().retrieve(request);
+points.getItems().forEach(c ->
+ System.out.println(c.getExternalId() + ": " + c.getDatapoints().size() + " points"));
+```
+
+
+
+
+```python
+import pandas as pd
+
+rf = intellistream_datahub_sdk.RetrieveFilter(
+ ts="engine_temperature",
+ start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=1),
+ end=pd.Timestamp.now(tz="UTC"),
+ limit=1000)
+
+collection = client.timeseries.retrieve_datapoints(rf)[0]
+for dp in collection.get_datapoints():
+ print(dp.timestamp, dp.value)
+```
+
+
+
+
+```rust
+use chrono::Utc;
+use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
+
+let filter = RetrieveFilter {
+ external_id: Some("engine_temperature".into()),
+ start: Some(Utc::now() - chrono::Duration::hours(1)),
+ end: Some(Utc::now()),
+ limit: Some(1000),
+ ..Default::default()
+};
+let points = api.time_series
+ .retrieve_datapoints(&DataWrapper::from(vec![filter]))
+ .await?;
+for c in points.get_items() {
+ println!("{} points", c.datapoints.len());
+}
+```
+
+
+
+
+### Latest datapoint {#latest-datapoints}
+
+`POST /timeseries/data/latest` returns the most recent point of each named series, by id or
+external id. A series with no data at all comes back as an empty collection rather than being
+omitted, so the response lines up with the request.
+
+It is much cheaper than [retrieve](#retrieve-datapoints) with a limit of one: the latest point
+is served from the cache the ingest path writes, not from a range scan.
+
+```java
+DataWrapper> now = client.timeseries().latest(List.of(
+ IdCollection.createFromExternalId("engine_temperature"),
+ IdCollection.createFromExternalId("engine_pressure")));
+```
+
+## Delete datapoints
+
+Clears part of a series and leaves the definition alone. To remove the series itself, see
+[delete a series](#delete-a-series).
+
+Each item names one series by `externalId` or `id`, and both window bounds are optional:
+
+| Bounds given | What is deleted |
+| --- | --- |
+| `inclusiveBegin` and `exclusiveEnd` | The half-open window between them |
+| `inclusiveBegin` only | Everything from that instant onward |
+| `exclusiveEnd` only | Everything before that instant |
+| Neither | Every datapoint of the series, leaving its definition, edges and subscriptions |
+
+A bound is either ISO-8601 or epoch milliseconds, on the [timestamp rules](#write-datapoints).
+
+Like a series delete, this is handed off and completes shortly after the call returns, and it
+cannot be undone.
+
+
+
+
+```java
+import java.time.Instant;
+
+client.timeseries().deleteDatapoints(
+ "engine_temperature",
+ Instant.parse("2026-01-01T00:00:00Z"),
+ Instant.parse("2026-02-01T00:00:00Z"));
+
+// A null bound leaves that side open, so two nulls empty the series:
+client.timeseries().deleteDatapoints("engine_temperature", null, null);
+```
+
+For several series at once, or to name one by id, pass `DeleteDatapoint` items instead:
+
+```java
+DeleteDatapoint window = new DeleteDatapoint();
+window.setId(7L);
+window.setInclusiveBegin("1767225600000"); // epoch millis is the other form, never seconds
+
+client.timeseries().deleteDatapoints(List.of(window));
+```
+
+
+
+
+```python
+import pandas as pd
+
+client.timeseries.delete_datapoints([
+ intellistream_datahub_sdk.DeleteFilter(
+ ts="engine_temperature",
+ inclusive_begin=pd.Timestamp("2026-01-01", tz="UTC"),
+ exclusive_end=pd.Timestamp("2026-02-01", tz="UTC"))])
+```
+
+
+
+
+```rust
+use chrono::{TimeZone, Utc};
+use intellistream_datahub_sdk::generic::{DataWrapper, DeleteFilter};
+
+let filter = DeleteFilter::from_external_id(
+ "engine_temperature".to_string(),
+ Some(Utc.with_ymd_and_hms(2026, 1, 1, 0, 0, 0).unwrap()),
+ Some(Utc.with_ymd_and_hms(2026, 2, 1, 0, 0, 0).unwrap()));
+
+api.time_series.delete_datapoints(&DataWrapper::from_vec(vec![filter])).await?;
+```
+
+
+
+
+## IngestOptions
+
+The Java `ingest` tuning knobs. Python's `insert_from_lists` and Rust's `insert_datapoints` take
+no options; their one knob is the request concurrency, see
+[high-throughput ingestion](#high-throughput-ingestion).
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `batchSize` | `10_000` | Maximum items per request. Also the server's [cap](./limits#batch-caps), so do not raise it. |
+| `parallelism` | `8` | Concurrent in-flight requests. |
+| `maxRetries` | `3` | Retries for transient failures (HTTP 429/5xx, network). |
+| `failFast` | `false` | If `true`, abort on the first failed batch instead of collecting errors. |
+
+`IngestOptions.defaults()` returns the defaults; `ingest(data)` (no options) uses them.
+
+## IngestResult
+
+```java
+long succeeded() // items ingested
+long failed() // items that could not be ingested
+long buffered() // items spooled to the durable buffer (0 unless buffering is on)
+boolean isComplete() // true when nothing failed and nothing was buffered
+List errors() // one entry per failed batch
+```
+
+`BatchError` is a record `(int datapointCount, int statusCode, String message, String body)`.
+`statusCode` is `0` when the failure was a network error rather than an HTTP status, and
+`body` carries the raw response when the server sent one, so a
+[problem `type`](./limits) is readable without a second request. A three-argument constructor
+leaves `body` null, so existing call sites are unaffected.
+
+```java
+if (!result.isComplete()) {
+ result.errors().forEach(e ->
+ System.err.println(e.statusCode() + " on " + e.datapointCount() + " items: " + e.message()));
+}
+```
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| Create | `timeseries().create` | `timeseries.create` | `time_series.create` / `create_one` |
+| Get by numeric id | `timeseries().getById` | `timeseries.get_by_id` | `time_series.get_by_id` |
+| Look up by id / external id | `timeseries().byIds` | `timeseries.by_ids` | `time_series.by_ids` |
+| Filter | `timeseries().filter` | `timeseries.filter` | `time_series.filter` |
+| Search | `timeseries().search` | `timeseries.search` | `time_series.search` |
+| List | `timeseries().list` | `timeseries.list` | `time_series.list` |
+| [Value-type hint](#value-type-hint) | `timeseries().recommendValueType` | `timeseries.recommend_value_type` | `time_series.recommend_value_type` |
+| Update | `timeseries().update` | `timeseries.update` | `time_series.update` |
+| Delete | `timeseries().delete` | `timeseries.delete` | `time_series.delete` |
+| Write datapoints | `insertDatapoints` / `ingest` | `insert_datapoints` / `insert_from_lists` | `insert_datapoint` / `insert_datapoints` |
+| [Binary ingest](#binary-ingest) | `ingestBinary` / `binaryBuffer` | `insert_datapoints_binary` / `insert_from_lists_binary`, synchronous client only | `insert_datapoints_binary` |
+| Create a `float32`, `numeric`, `decimal32` or `mixed` series | `setValueType` | no, see [value types](#value-types) | `set_value_type` |
+| Read datapoints (raw and [aggregated](#retrieve-datapoints)) | `retrieve` / `retrieveAggregated` | `retrieve_datapoints` | `retrieve_datapoints` |
+| [Latest datapoint](#latest-datapoints) | `timeseries().latest` | `retrieve_latest_datapoints` | `retrieve_latest_datapoint` |
+| Delete datapoints | `deleteDatapoints` | `timeseries.delete_datapoints` | `time_series.delete_datapoints` |
+| [Live datapoint tail](./subscriptions#live-tail) | — | `timeseries.listen_datapoints` | `time_series.listen_datapoints` |
+
+Java is the one with `ingest`, the chunking, parallelising, retrying path described above, and
+with `binaryBuffer`. Python and Rust chunk and send concurrently too, and retry through the
+durable spool when buffering is on; with it off, the first failed request is the call's error.
+All three have the [binary path](#binary-ingest), Python on its synchronous client only. Python
+can create a series of only three [value types](#value-types). Java has no
+[live datapoint tail](./subscriptions#live-tail); use `subscriptions().listen` there.
diff --git a/versioned_docs/version-1.0/reference/units.md b/versioned_docs/version-1.0/reference/units.md
new file mode 100644
index 0000000..b143b53
--- /dev/null
+++ b/versioned_docs/version-1.0/reference/units.md
@@ -0,0 +1,114 @@
+---
+sidebar_position: 7
+title: Units
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Units
+
+Units of measure (read-only reference data). The endpoints are `GET /units`,
+`GET /units/{externalId}` and `POST /units/byids`.
+
+## The unit object {#body}
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `id` | number | Crosses the wire as a JSON string, like every other id. |
+| `externalId` | string, 3–256 | The catalogue key, `_` in snake_case: `temperature_deg_c`, `pressure_bar`, `mass_flow_rate_kghr`. This is what a series' `unitExternalId` names. |
+| `name` | string, 1–64 | Short code (`DEG_C`). |
+| `longName` | string | `degree Celsius`. |
+| `symbol` | string | `°C`. |
+| `description` | string | Prose. |
+| `aliasNames` | string[] | Other spellings (`C`, `degC`). |
+| `quantity` | string | What it measures (`Temperature`). |
+| `conversion` | `{ multiplier, offset }` | To the quantity's base unit. |
+| `source`, `sourceReference` | string | Where the definition comes from (`qudt.org` and its URL). |
+
+## List all units
+
+
+
+
+```java
+DataWrapper units = client.units().list();
+for (UnitModel unit : units.getItems()) {
+ System.out.println(unit.getExternalId() + " — " + unit.getName() + " (" + unit.getSymbol() + ")");
+}
+```
+
+
+
+
+```python
+for unit in client.units.list():
+ print(unit.external_id, unit.name, unit.symbol)
+```
+
+
+
+
+```rust
+for unit in api.units.list().await?.get_items() {
+ println!("{} — {} ({})", unit.external_id, unit.name, unit.symbol);
+}
+```
+
+
+
+
+## Look up
+
+
+
+
+`byIds` takes `IdCollection`, by numeric id or by external id:
+
+```java
+DataWrapper byExt = client.units().byIds(List.of(
+ IdCollection.createFromExternalId("temperature_deg_c")));
+DataWrapper byId = client.units().byIds(List.of(IdCollection.createFromId(7)));
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+
+by_ext = client.units.by_external_id("temperature_deg_c")
+by_id = client.units.by_ids([intellistream_datahub_sdk.IdCollection(id=7)])
+```
+
+`by_external_id` takes one external id and returns a list, empty when no unit has it. The
+`AsyncDataHubClient` awaits the same call:
+
+```python
+by_ext = await client.units.by_external_id("temperature_deg_c")
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+
+let by_ext = api.units.by_external_id("temperature_deg_c").await?;
+let by_id = api.units.by_ids(&DataWrapper::from(vec![IdAndExtId::from_id(7)])).await?;
+```
+
+An external id no unit has is an error here, not an empty result: `by_external_id` returns
+`Err` with status `404`.
+
+
+
+
+## What each client covers {#client-coverage}
+
+| Operation | Java | Python | Rust |
+| --- | --- | --- | --- |
+| List all | `units().list` | `units.list` | `units.list` |
+| Look up by id | `units().byIds` | `units.by_ids` | `units.by_ids` |
+| Look up by external id | `units().byIds` | `units.by_external_id` | `units.by_external_id` |
+
+The unit catalogue is read-only in every client: units are platform-managed, not tenant data.
diff --git a/versioned_docs/version-1.0/tutorial.mdx b/versioned_docs/version-1.0/tutorial.mdx
new file mode 100644
index 0000000..8e83af1
--- /dev/null
+++ b/versioned_docs/version-1.0/tutorial.mdx
@@ -0,0 +1,1114 @@
+---
+sidebar_position: 3
+title: "Tutorial: stream system metrics"
+sidebar_label: Tutorial
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Tutorial: stream system metrics
+
+Build a small **metrics agent**, end to end: a program that samples this machine's
+memory every three seconds and streams it into DataHub as time-series data, and keeps
+working when the API doesn't. It runs on Linux, macOS, and Windows. The same program is
+shown in Java, Python and Rust; pick your language once and every code block on the
+page follows.
+
+The whole flow is five steps, and each is a section below:
+
+1. **Build the client**: authenticate with a bearer token and turn on a durable,
+ spool-to-disk ingest buffer.
+2. **Ensure the time series exist**: look up three series by external id and create
+ only the ones that are missing, so the program is safe to re-run.
+3. **Sample the data**: read memory usage via a small OS-info library (this is just
+ the tutorial's data source, swap in your own).
+4. **Ingest datapoints**: every three seconds, send one datapoint per series.
+5. **Survive outages**: when the API is unreachable, datapoints spool to disk and
+ flush automatically once it recovers.
+
+## Before you start
+
+You need a reachable DataHub API and credentials. This tutorial uses **OAuth2
+client-credentials**, the SDK fetches the bearer token and refreshes it when it
+expires, so a long-running agent never works with a stale token. The program reads the
+configuration from environment variables:
+
+```bash
+export BASE_URL="https://api.intellistream.ai" # or your own instance
+export CLIENT_ID="metrics-agent"
+export CLIENT_SECRET="..."
+export TOKEN_URI="https://auth.intellistream.ai/oauth2/token"
+```
+
+:::note Windows
+Setting environment variables uses different syntax: `set BASE_URL=...` (cmd.exe) or
+`$env:BASE_URL = "..."` (PowerShell), in place of `export`.
+:::
+
+If all you have is a static bearer token, that works too: set `TOKEN` instead and use
+the token variant shown in step 1. See
+[Authentication](/reference/client#authentication) for the full picture.
+
+## Set up a project
+
+
+
+
+One dependency and a main class. Memory and swap come from a JDK-builtin API, so
+nothing extra is needed for that. The SDK comes from Maven Central:
+
+```kotlin
+// build.gradle.kts
+plugins {
+ application
+}
+
+repositories {
+ mavenCentral()
+}
+
+dependencies {
+ implementation("ai.intellistream:datahub-sdk:1.0.0")
+}
+
+application {
+ mainClass = "example.MemoryIngestTutorial"
+}
+```
+
+The code below lives in `src/main/java/example/MemoryIngestTutorial.java`.
+
+
+
+
+`psutil` reads memory and swap portably across Linux, macOS and Windows:
+
+```bash
+pip install intellistream-datahub-sdk psutil
+```
+
+The code below lives in a single file, `memory_ingest.py`.
+
+
+
+
+`sysinfo` reads memory and swap portably; `hostname` reads the machine's hostname
+portably:
+
+```toml
+# Cargo.toml
+[dependencies]
+intellistream-datahub-sdk = "1.0"
+tokio = { version = "1", features = ["full"] }
+chrono = "0.4"
+sysinfo = "0.37"
+hostname = "0.4"
+```
+
+The code below lives in `src/main.rs`, with `#[tokio::main]` driving the async calls.
+
+
+
+
+```toml
+# Cargo.toml
+[dependencies]
+intellistream-datahub-sdk = { version = "1.0", features = ["blocking"] }
+chrono = "0.4"
+sysinfo = "0.37"
+hostname = "0.4"
+```
+
+The code below lives in `src/main.rs`. The `blocking` feature enables the SDK's
+synchronous client (`intellistream_datahub_sdk::blocking`): no async runtime and no
+`async`/`.await` in your code. It has every call this tutorial makes, but not yet all of the
+async API: `subscriptions` and a few calls such as `time_series.filter` are async only.
+
+
+
+
+## Step 1: Build the client
+
+Everything starts from one client object. Two things happen here: authentication
+(client-credentials from the environment, the SDK exchanges them for a bearer token
+and refreshes it as needed) and, the part that makes this program resilient, turning
+on the **durable ingest buffer**. With it enabled, an ingest that can't reach the API
+is written to compressed segments on disk and replayed automatically later. Without
+it, ingestion is best-effort and an outage loses data.
+
+
+
+
+```java
+DatahubClient client = DatahubClient.create(DatahubConfig.builder()
+ .baseUrl(System.getenv("BASE_URL"))
+ .clientCredentials(System.getenv("CLIENT_ID"),
+ System.getenv("CLIENT_SECRET"),
+ System.getenv("TOKEN_URI"))
+ .enableBuffering() // spool to disk when unreachable
+ .bufferRetention(Duration.ofMinutes(60)) // keep up to an hour of backlog
+ .bufferDirectory(Path.of("datahub-spool"))
+ .build());
+```
+
+
+
+
+```python
+import intellistream_datahub_sdk
+from intellistream_datahub_sdk import DataHubClient
+
+client = DataHubClient(
+ base_url=os.environ["BASE_URL"],
+ client_id=os.environ["CLIENT_ID"],
+ client_secret=os.environ["CLIENT_SECRET"],
+ token_url=os.environ["TOKEN_URI"],
+ enable_buffering=True, # spool to disk when unreachable
+ buffer_retention_secs=3600, # keep up to an hour of backlog
+ buffer_dir="datahub-spool",
+)
+```
+
+
+
+
+`from_env` picks up `CLIENT_ID` / `CLIENT_SECRET` / `TOKEN_URI` (or a static `TOKEN`)
+by itself:
+
+```rust
+use intellistream_datahub_sdk::{ApiService, datahub::DataHubConfig};
+
+let mut config = DataHubConfig::from_env().expect("BASE_URL and credentials must be set");
+config
+ .enable_buffering() // spool to disk when unreachable
+ .set_buffer_retention_secs(3600) // keep up to an hour of backlog
+ .set_buffer_dir("datahub-spool");
+let api = ApiService::new(config);
+```
+
+
+
+
+`from_env` picks up `CLIENT_ID` / `CLIENT_SECRET` / `TOKEN_URI` (or a static `TOKEN`)
+by itself:
+
+```rust
+use intellistream_datahub_sdk::blocking;
+use intellistream_datahub_sdk::datahub::DataHubConfig;
+
+let mut config = DataHubConfig::from_env().expect("BASE_URL and credentials must be set");
+config
+ .enable_buffering() // spool to disk when unreachable
+ .set_buffer_retention_secs(3600) // keep up to an hour of backlog
+ .set_buffer_dir("datahub-spool");
+let api = blocking::ApiService::new(config);
+```
+
+
+
+
+:::note Static token instead?
+Replace the credentials with `.token(System.getenv("TOKEN"))` in Java or
+`token=os.environ["TOKEN"]` in Python; in Rust just export `TOKEN` instead of the
+three OAuth variables. Everything else on this page stays the same.
+:::
+
+:::note
+Buffering is optional and off by default, bounded by a time window and a size cap
+(72 h / 5 GiB unless overridden). See
+[Durable ingest buffering](/reference/client#durable-ingest-buffering) for the full
+story.
+:::
+
+## Step 2: Ensure the time series exist
+
+Before you can ingest, the target series have to exist. The create endpoint rejects a
+duplicate external id with `409`, so rather than blindly creating them the program **looks up first, then
+creates only the gap**, which makes it safe to re-run and safe to start on a machine
+where the series already exist.
+
+Each external id is **namespaced by hostname** (`system_memory_used_web01`, …), so the
+same program running on several machines writes to distinct series instead of
+overwriting one another. The three series, all integer-valued in `bytes`:
+
+| External id | What it tracks |
+| --- | --- |
+| `system_memory_used_` | Used physical memory |
+| `system_memory_available_` | Available physical memory |
+| `system_swap_used_` | Used swap space |
+
+:::note Numbers aren't identical across languages
+`used`/`available` come from a different OS API per language, and those APIs don't all
+mean quite the same thing. Java's `available` is raw OS free memory; Python's and
+Rust's `available` is closer to Linux's cache-aware `MemAvailable` estimate, so on the
+same machine, Java tends to read a lower `available` (and higher `used`) than Python or
+Rust. `used + available == total` holds for Java but generally not for Python/Rust,
+that's expected, not a bug.
+:::
+
+
+
+
+```java
+record Metric(String baseId, String name) {}
+
+List METRICS = List.of(
+ new Metric("system_memory_used", "System memory used"),
+ new Metric("system_memory_available", "System memory available"),
+ new Metric("system_swap_used", "System swap used"));
+
+List lookups = METRICS.stream()
+ .map(m -> IdCollection.createFromExternalId(m.baseId() + "_" + host))
+ .toList();
+Set existing = client.timeseries().byIds(lookups).getItems().stream()
+ .map(Timeseries::getExternalId)
+ .collect(Collectors.toSet());
+
+List missing = METRICS.stream()
+ .filter(m -> !existing.contains(m.baseId() + "_" + host))
+ .map(m -> {
+ Timeseries ts = new Timeseries();
+ ts.setExternalId(m.baseId() + "_" + host);
+ ts.setName(m.name() + " (" + host + ")");
+ ts.setValueType("BIGINT");
+ ts.setUnit("bytes");
+ return ts;
+ })
+ .toList();
+if (!missing.isEmpty()) {
+ client.timeseries().create(missing);
+}
+```
+
+
+
+
+```python
+wanted = {f"{base}_{host}": name for base, name in METRICS.items()}
+existing = {ts.external_id for ts in client.timeseries.by_ids(list(wanted))}
+
+missing = [
+ intellistream_datahub_sdk.TimeSeries(
+ external_id=ext_id, name=f"{name} ({host})",
+ unit="bytes", value_type="BIGINT")
+ for ext_id, name in wanted.items()
+ if ext_id not in existing
+]
+if missing:
+ client.timeseries.create(missing)
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+let lookups: Vec = METRICS.iter()
+ .map(|(base, _)| IdAndExtId::from_external_id(&format!("{base}_{host}")))
+ .collect();
+let existing: Vec = api.time_series.by_ids(&DataWrapper::from(lookups)).await?
+ .get_items().iter()
+ .map(|ts| ts.external_id.clone())
+ .collect();
+
+for (base, name) in METRICS {
+ let ext_id = format!("{base}_{host}");
+ if existing.contains(&ext_id) {
+ continue;
+ }
+ let mut ts = TimeSeries::new(&ext_id, &format!("{name} ({host})"));
+ ts.unit = Some("bytes".into());
+ ts.value_type = Some("BIGINT".into());
+ api.time_series.create_one(&ts).await?;
+}
+```
+
+
+
+
+```rust
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+
+let lookups: Vec = METRICS.iter()
+ .map(|(base, _)| IdAndExtId::from_external_id(&format!("{base}_{host}")))
+ .collect();
+let existing: Vec = api.time_series.by_ids(&DataWrapper::from(lookups))?
+ .get_items().iter()
+ .map(|ts| ts.external_id.clone())
+ .collect();
+
+for (base, name) in METRICS {
+ let ext_id = format!("{base}_{host}");
+ if existing.contains(&ext_id) {
+ continue;
+ }
+ let mut ts = TimeSeries::new(&ext_id, &format!("{name} ({host})"));
+ ts.unit = Some("bytes".into());
+ ts.value_type = Some("BIGINT".into());
+ api.time_series.create_one(&ts)?;
+}
+```
+
+
+
+
+The lookup is **best effort**: if the API is down at startup, the program just retries
+on the next tick, and thanks to the buffer from step 1, datapoints ingested in the
+meantime aren't lost.
+
+## Step 3: Sample the data
+
+This step has nothing to do with the SDK. It is just where the numbers come from. Each
+language uses a small OS-info library to read memory and swap in one call, in bytes, so
+the same code works on Linux, macOS and Windows. Replace this function with any data
+source of your own and the DataHub steps (1, 2, 4, 5) don't change.
+
+
+
+
+`com.sun.management.OperatingSystemMXBean` is built into the JDK, no dependency
+needed. There's no direct "used" getter, so it's derived as total minus free:
+
+```java
+private static Map sample() {
+ OperatingSystemMXBean os =
+ ManagementFactory.getPlatformMXBean(OperatingSystemMXBean.class);
+
+ Map values = new HashMap<>();
+ values.put("system_memory_used", os.getTotalMemorySize() - os.getFreeMemorySize());
+ values.put("system_memory_available", os.getFreeMemorySize());
+ values.put("system_swap_used", os.getTotalSwapSpaceSize() - os.getFreeSwapSpaceSize());
+ return values;
+}
+```
+
+
+
+
+`psutil.virtual_memory()` and `psutil.swap_memory()` expose `used`/`available`
+directly, no arithmetic needed:
+
+```python
+def sample():
+ vm = psutil.virtual_memory()
+ swap = psutil.swap_memory()
+ return {
+ "system_memory_used": vm.used,
+ "system_memory_available": vm.available,
+ "system_swap_used": swap.used,
+ }
+```
+
+
+
+
+The sampler is plain synchronous code, identical in both Rust flavors. `sysinfo`
+exposes `used_memory()`/`available_memory()`/`used_swap()` directly:
+
+```rust
+use std::collections::HashMap;
+use sysinfo::System;
+
+fn sample() -> HashMap<&'static str, i64> {
+ let mut sys = System::new();
+ sys.refresh_memory();
+ HashMap::from([
+ ("system_memory_used", sys.used_memory() as i64),
+ ("system_memory_available", sys.available_memory() as i64),
+ ("system_swap_used", sys.used_swap() as i64),
+ ])
+}
+```
+
+
+
+
+The sampler is plain synchronous code, identical in both Rust flavors. `sysinfo`
+exposes `used_memory()`/`available_memory()`/`used_swap()` directly:
+
+```rust
+use std::collections::HashMap;
+use sysinfo::System;
+
+fn sample() -> HashMap<&'static str, i64> {
+ let mut sys = System::new();
+ sys.refresh_memory();
+ HashMap::from([
+ ("system_memory_used", sys.used_memory() as i64),
+ ("system_memory_available", sys.available_memory() as i64),
+ ("system_swap_used", sys.used_swap() as i64),
+ ])
+}
+```
+
+
+
+
+## Step 4: Ingest on a tick
+
+A timer fires every three seconds. Each tick samples memory and sends one datapoint per
+series, three datapoints, stamped with the same timestamp.
+
+
+
+
+The payload is a map of external id to datapoints. `ingest` chunks, parallelises and
+retries under the hood, and returns an `IngestResult` saying what happened:
+
+```java
+Instant now = Instant.now();
+Map> byExternalId = new LinkedHashMap<>();
+for (Metric m : METRICS) {
+ byExternalId.put(m.baseId() + "_" + host,
+ List.of(Datapoint.of(now, sample.get(m.baseId()))));
+}
+
+IngestResult result = client.timeseries().ingest(byExternalId);
+```
+
+
+
+
+One `insert_from_lists` call per series, here each carries a single datapoint, but the
+same call takes whole arrays when you have them. Any timezone-aware timestamps work
+(plain `datetime` here; pandas if you already have it):
+
+```python
+now = datetime.now(timezone.utc)
+values = sample()
+for base in METRICS:
+ client.timeseries.insert_from_lists(
+ timestamps=[now], values=[values[base]], ts=f"{base}_{host}")
+```
+
+
+
+
+One `insert_datapoint` call per series:
+
+```rust
+use chrono::Utc;
+
+let now = Utc::now();
+let values = sample();
+for (base, _) in METRICS {
+ api.time_series
+ .insert_datapoint(None, Some(format!("{base}_{host}")), now, values[base].to_string())
+ .await?;
+}
+```
+
+
+
+
+One `insert_datapoint` call per series:
+
+```rust
+use chrono::Utc;
+
+let now = Utc::now();
+let values = sample();
+for (base, _) in METRICS {
+ api.time_series
+ .insert_datapoint(None, Some(format!("{base}_{host}")), now, values[base].to_string())?;
+}
+```
+
+
+
+
+Whatever a single tick does, a sampling error, a rejected request, it must never kill
+the loop: catch, log, and let the next tick try again. The complete programs below wrap
+each tick that way.
+
+## Step 5: Survive an outage
+
+This is what the buffer from step 1 buys you. Stop the DataHub API while the program is
+running: instead of raising, each ingest spools its datapoints to compressed segments
+under `datahub-spool/`. Start the API again and the backlog flushes automatically on
+the next call, no data lost within the retention window. The same happens when the
+token is rejected (HTTP 401/403, e.g. expired): datapoints keep spooling until you
+restore a valid token.
+
+
+
+
+The Java client reports buffering explicitly, `IngestResult.buffered()` is the number
+of datapoints that were spooled instead of sent:
+
+```java
+if (result.buffered() > 0) {
+ System.out.printf("%s API unreachable: %d datapoints buffered%n",
+ now, result.buffered());
+ wasBuffering = true;
+} else {
+ if (wasBuffering) {
+ System.out.println(now + " recovered: buffered backlog flushed");
+ wasBuffering = false;
+ }
+ System.out.printf("%s ingested %d datapoints%n", now, result.succeeded());
+}
+```
+
+
+
+
+With buffering enabled the calls simply don't raise on an outage, the datapoints are
+spooled and the program keeps ticking. Watch the spool itself to see it happen:
+
+```bash
+ls datahub-spool/ # segments appear while the API is down, drain when it's back
+```
+
+Exceptions still surface for non-retriable errors (e.g. a malformed request), so keep
+the tick wrapped in `try`/`except`.
+
+
+
+
+With buffering enabled the calls return `Ok` on an outage, the datapoints are spooled
+and the program keeps ticking. Watch the spool itself to see it happen:
+
+```bash
+ls datahub-spool/ # segments appear while the API is down, drain when it's back
+```
+
+Errors still surface for non-retriable failures (e.g. a malformed request), so keep
+handling the `Result`.
+
+
+
+
+With buffering enabled the calls return `Ok` on an outage, the datapoints are spooled
+and the program keeps ticking. Watch the spool itself to see it happen:
+
+```bash
+ls datahub-spool/ # segments appear while the API is down, drain when it's back
+```
+
+Errors still surface for non-retriable failures (e.g. a malformed request), so keep
+handling the `Result`.
+
+
+
+
+:::note Retries are idempotent
+A flush re-sends buffered data, which is safe: datapoints are keyed by
+`(series, timestamp)`, so the backend collapses duplicates. See
+[Durable ingest buffering](/reference/client#durable-ingest-buffering).
+:::
+
+## Run it
+
+
+
+
+```bash
+./gradlew run
+```
+
+On Windows, use `gradlew.bat run` instead.
+
+
+
+
+```bash
+python memory_ingest.py
+```
+
+
+
+
+```bash
+cargo run
+```
+
+
+
+
+```bash
+cargo run
+```
+
+
+
+
+On the first run the program creates the three series; on later runs they already
+exist and it goes straight to ingesting:
+
+```
+2026-07-02T09:00:00Z ingested 3 datapoints
+2026-07-02T09:00:03Z ingested 3 datapoints
+...
+```
+
+Now simulate the outage: stop the API for a minute, watch the ticks report buffering in Java
+(and `datahub-spool/` fill up in every language; the Python and Rust programs keep printing
+`ingested`, since a spooled call succeeds), then start it again and watch the backlog flush. To see
+the ingested data, read it back with a
+[retrieve filter](/reference/timeseries#retrieve-datapoints) or aggregate it as in
+[Query & aggregate](/guides/query-and-aggregate).
+
+## The complete program
+
+
+
+
+```java title="src/main/java/example/MemoryIngestTutorial.java"
+package example;
+
+import ai.intellistream.datahub.models.IdCollection;
+import ai.intellistream.datahub.sdk.client.DatahubClient;
+import ai.intellistream.datahub.sdk.client.DatahubConfig;
+import ai.intellistream.datahub.sdk.ingest.IngestResult;
+import ai.intellistream.datahub.sdk.timeseries.Datapoint;
+import ai.intellistream.datahub.timeseries.Timeseries;
+
+import com.sun.management.OperatingSystemMXBean;
+
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.nio.file.Path;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+public final class MemoryIngestTutorial {
+
+ private record Metric(String baseId, String name) {}
+
+ /** The three series this program owns; external ids get "_" appended. */
+ private static final List METRICS = List.of(
+ new Metric("system_memory_used", "System memory used"),
+ new Metric("system_memory_available", "System memory available"),
+ new Metric("system_swap_used", "System swap used"));
+
+ private static boolean ensured;
+ private static boolean wasBuffering;
+
+ public static void main(String[] args) throws Exception {
+ // Step 1: build the client with a durable, spool-to-disk ingest buffer.
+ DatahubClient client = DatahubClient.create(DatahubConfig.builder()
+ .baseUrl(System.getenv("BASE_URL"))
+ .clientCredentials(System.getenv("CLIENT_ID"),
+ System.getenv("CLIENT_SECRET"),
+ System.getenv("TOKEN_URI"))
+ .enableBuffering()
+ .bufferRetention(Duration.ofMinutes(60))
+ .bufferDirectory(Path.of("datahub-spool"))
+ .build());
+ String host = System.getenv().getOrDefault("HOST_ID",
+ InetAddress.getLocalHost().getHostName().toLowerCase());
+
+ Executors.newSingleThreadScheduledExecutor()
+ .scheduleAtFixedRate(() -> tick(client, host), 0, 3, TimeUnit.SECONDS);
+ }
+
+ /** One tick: ensure the series exist, sample memory, ingest, report. */
+ private static void tick(DatahubClient client, String host) {
+ try {
+ try {
+ ensureSeries(client, host);
+ } catch (RuntimeException e) {
+ // API down: retry next tick — datapoints buffer meanwhile.
+ }
+
+ // Steps 3 & 4: sample and send one datapoint per series.
+ Map sample = sample();
+ Instant now = Instant.now();
+ Map> byExternalId = new LinkedHashMap<>();
+ for (Metric m : METRICS) {
+ byExternalId.put(m.baseId() + "_" + host,
+ List.of(Datapoint.of(now, sample.get(m.baseId()))));
+ }
+ IngestResult result = client.timeseries().ingest(byExternalId);
+
+ // Step 5: report — buffered() > 0 means spooled, not sent.
+ if (result.buffered() > 0) {
+ System.out.printf("%s API unreachable: %d datapoints buffered%n",
+ now, result.buffered());
+ wasBuffering = true;
+ } else {
+ if (wasBuffering) {
+ System.out.println(now + " recovered: buffered backlog flushed");
+ wasBuffering = false;
+ }
+ System.out.printf("%s ingested %d datapoints%n", now, result.succeeded());
+ }
+ } catch (Exception e) {
+ System.err.println("tick failed: " + e.getMessage()); // never kill the loop
+ }
+ }
+
+ /** Step 2: look up the three series by external id, create only the missing ones. */
+ private static void ensureSeries(DatahubClient client, String host) {
+ if (ensured) {
+ return;
+ }
+ List lookups = METRICS.stream()
+ .map(m -> IdCollection.createFromExternalId(m.baseId() + "_" + host))
+ .toList();
+ Set existing = client.timeseries().byIds(lookups).getItems().stream()
+ .map(Timeseries::getExternalId)
+ .collect(Collectors.toSet());
+
+ List missing = METRICS.stream()
+ .filter(m -> !existing.contains(m.baseId() + "_" + host))
+ .map(m -> {
+ Timeseries ts = new Timeseries();
+ ts.setExternalId(m.baseId() + "_" + host);
+ ts.setName(m.name() + " (" + host + ")");
+ ts.setValueType("BIGINT");
+ ts.setUnit("bytes");
+ return ts;
+ })
+ .toList();
+ if (!missing.isEmpty()) {
+ client.timeseries().create(missing);
+ }
+ ensured = true;
+ }
+
+ /** Step 3: sample memory and swap via the JDK's platform MXBean, in bytes. */
+ private static Map sample() {
+ OperatingSystemMXBean os =
+ ManagementFactory.getPlatformMXBean(OperatingSystemMXBean.class);
+
+ Map values = new HashMap<>();
+ values.put("system_memory_used", os.getTotalMemorySize() - os.getFreeMemorySize());
+ values.put("system_memory_available", os.getFreeMemorySize());
+ values.put("system_swap_used", os.getTotalSwapSpaceSize() - os.getFreeSwapSpaceSize());
+ return values;
+ }
+}
+```
+
+
+
+
+```python title="memory_ingest.py"
+import os
+import socket
+import time
+from datetime import datetime, timezone
+
+import psutil
+
+import intellistream_datahub_sdk
+from intellistream_datahub_sdk import DataHubClient
+
+INTERVAL_SECS = 3
+
+# The three series this program owns; external ids get "_" appended.
+METRICS = {
+ "system_memory_used": "System memory used",
+ "system_memory_available": "System memory available",
+ "system_swap_used": "System swap used",
+}
+
+
+def sample():
+ """Step 3: sample memory and swap via psutil, in bytes."""
+ vm = psutil.virtual_memory()
+ swap = psutil.swap_memory()
+ return {
+ "system_memory_used": vm.used,
+ "system_memory_available": vm.available,
+ "system_swap_used": swap.used,
+ }
+
+
+def ensure_series(client, host):
+ """Step 2: look up the three series by external id, create only the missing ones."""
+ wanted = {f"{base}_{host}": name for base, name in METRICS.items()}
+ existing = {ts.external_id for ts in client.timeseries.by_ids(list(wanted))}
+
+ missing = [
+ intellistream_datahub_sdk.TimeSeries(
+ external_id=ext_id, name=f"{name} ({host})",
+ unit="bytes", value_type="BIGINT")
+ for ext_id, name in wanted.items()
+ if ext_id not in existing
+ ]
+ if missing:
+ client.timeseries.create(missing)
+
+
+def main():
+ # Step 1: build the client with a durable, spool-to-disk ingest buffer.
+ client = DataHubClient(
+ base_url=os.environ["BASE_URL"],
+ client_id=os.environ["CLIENT_ID"],
+ client_secret=os.environ["CLIENT_SECRET"],
+ token_url=os.environ["TOKEN_URI"],
+ enable_buffering=True,
+ buffer_retention_secs=3600,
+ buffer_dir="datahub-spool",
+ )
+ host = os.environ.get("HOST_ID", socket.gethostname().lower())
+
+ ensured = False
+ while True:
+ try:
+ if not ensured:
+ try:
+ ensure_series(client, host)
+ ensured = True
+ except Exception:
+ pass # API down: retry next tick — datapoints buffer meanwhile
+
+ # Steps 4 & 5: send one datapoint per series. With buffering on, an
+ # outage spools to datahub-spool/ instead of raising.
+ now = datetime.now(timezone.utc)
+ values = sample()
+ for base in METRICS:
+ client.timeseries.insert_from_lists(
+ timestamps=[now], values=[values[base]], ts=f"{base}_{host}")
+ print(f"{now} ingested {len(METRICS)} datapoints")
+ except Exception as e:
+ print(f"tick failed: {e}") # never kill the loop
+ time.sleep(INTERVAL_SECS)
+
+
+if __name__ == "__main__":
+ main()
+```
+
+
+
+
+```rust title="src/main.rs"
+use std::collections::HashMap;
+use std::time::Duration;
+
+use chrono::Utc;
+use intellistream_datahub_sdk::datahub::DataHubConfig;
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use intellistream_datahub_sdk::ApiService;
+use sysinfo::System;
+
+const INTERVAL_SECS: u64 = 3;
+
+// The three series this program owns; external ids get "_" appended.
+const METRICS: [(&str, &str); 3] = [
+ ("system_memory_used", "System memory used"),
+ ("system_memory_available", "System memory available"),
+ ("system_swap_used", "System swap used"),
+];
+
+#[tokio::main]
+async fn main() {
+ // Step 1: build the client with a durable, spool-to-disk ingest buffer.
+ let mut config = DataHubConfig::from_env().expect("BASE_URL and credentials must be set");
+ config
+ .enable_buffering()
+ .set_buffer_retention_secs(3600)
+ .set_buffer_dir("datahub-spool");
+ let api = ApiService::new(config);
+
+ let host = std::env::var("HOST_ID").unwrap_or_else(|_| hostname());
+
+ let mut ensured = false;
+ let mut ticker = tokio::time::interval(Duration::from_secs(INTERVAL_SECS));
+ loop {
+ ticker.tick().await;
+
+ if !ensured {
+ // API down: retry next tick — datapoints buffer meanwhile.
+ ensured = ensure_series(&api, &host).await.is_ok();
+ }
+
+ // Steps 4 & 5: send one datapoint per series. With buffering on, an
+ // outage spools to datahub-spool/ instead of failing the call.
+ let values = sample();
+ let now = Utc::now();
+ let mut sent = 0;
+ for (base, _) in METRICS {
+ let ext_id = format!("{base}_{host}");
+ match api.time_series
+ .insert_datapoint(None, Some(ext_id), now, values[base].to_string())
+ .await
+ {
+ Ok(_) => sent += 1,
+ Err(e) => eprintln!("{now} ingest failed: {e}"),
+ }
+ }
+ println!("{now} ingested {sent} datapoints");
+ }
+}
+
+/// Step 2: look up the three series by external id, create only the missing ones.
+async fn ensure_series(api: &ApiService, host: &str) -> Result<(), Box> {
+ let lookups: Vec = METRICS.iter()
+ .map(|(base, _)| IdAndExtId::from_external_id(&format!("{base}_{host}")))
+ .collect();
+ let existing: Vec = api.time_series.by_ids(&DataWrapper::from(lookups)).await?
+ .get_items().iter()
+ .map(|ts| ts.external_id.clone())
+ .collect();
+
+ for (base, name) in METRICS {
+ let ext_id = format!("{base}_{host}");
+ if existing.contains(&ext_id) {
+ continue;
+ }
+ let mut ts = TimeSeries::new(&ext_id, &format!("{name} ({host})"));
+ ts.unit = Some("bytes".into());
+ ts.value_type = Some("BIGINT".into());
+ api.time_series.create_one(&ts).await?;
+ }
+ Ok(())
+}
+
+/// Step 3: sample memory and swap via sysinfo, in bytes.
+fn sample() -> HashMap<&'static str, i64> {
+ let mut sys = System::new();
+ sys.refresh_memory();
+ HashMap::from([
+ ("system_memory_used", sys.used_memory() as i64),
+ ("system_memory_available", sys.available_memory() as i64),
+ ("system_swap_used", sys.used_swap() as i64),
+ ])
+}
+
+/// Cross-platform hostname lookup.
+fn hostname() -> String {
+ hostname::get()
+ .ok()
+ .and_then(|s| s.into_string().ok())
+ .map(|s| s.to_lowercase())
+ .unwrap_or_else(|| "localhost".into())
+}
+```
+
+
+
+
+```rust title="src/main.rs"
+use std::collections::HashMap;
+use std::time::Duration;
+
+use chrono::Utc;
+use intellistream_datahub_sdk::blocking;
+use intellistream_datahub_sdk::datahub::DataHubConfig;
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+use intellistream_datahub_sdk::timeseries::TimeSeries;
+use sysinfo::System;
+
+const INTERVAL_SECS: u64 = 3;
+
+// The three series this program owns; external ids get "_" appended.
+const METRICS: [(&str, &str); 3] = [
+ ("system_memory_used", "System memory used"),
+ ("system_memory_available", "System memory available"),
+ ("system_swap_used", "System swap used"),
+];
+
+fn main() {
+ // Step 1: build the client with a durable, spool-to-disk ingest buffer.
+ let mut config = DataHubConfig::from_env().expect("BASE_URL and credentials must be set");
+ config
+ .enable_buffering()
+ .set_buffer_retention_secs(3600)
+ .set_buffer_dir("datahub-spool");
+ let api = blocking::ApiService::new(config);
+
+ let host = std::env::var("HOST_ID").unwrap_or_else(|_| hostname());
+
+ let mut ensured = false;
+ loop {
+ if !ensured {
+ // API down: retry next tick — datapoints buffer meanwhile.
+ ensured = ensure_series(&api, &host).is_ok();
+ }
+
+ // Steps 4 & 5: send one datapoint per series. With buffering on, an
+ // outage spools to datahub-spool/ instead of failing the call.
+ let values = sample();
+ let now = Utc::now();
+ let mut sent = 0;
+ for (base, _) in METRICS {
+ let ext_id = format!("{base}_{host}");
+ match api.time_series
+ .insert_datapoint(None, Some(ext_id), now, values[base].to_string())
+ {
+ Ok(_) => sent += 1,
+ Err(e) => eprintln!("{now} ingest failed: {e}"),
+ }
+ }
+ println!("{now} ingested {sent} datapoints");
+ std::thread::sleep(Duration::from_secs(INTERVAL_SECS));
+ }
+}
+
+/// Step 2: look up the three series by external id, create only the missing ones.
+fn ensure_series(api: &blocking::ApiService, host: &str) -> Result<(), Box> {
+ let lookups: Vec = METRICS.iter()
+ .map(|(base, _)| IdAndExtId::from_external_id(&format!("{base}_{host}")))
+ .collect();
+ let existing: Vec = api.time_series.by_ids(&DataWrapper::from(lookups))?
+ .get_items().iter()
+ .map(|ts| ts.external_id.clone())
+ .collect();
+
+ for (base, name) in METRICS {
+ let ext_id = format!("{base}_{host}");
+ if existing.contains(&ext_id) {
+ continue;
+ }
+ let mut ts = TimeSeries::new(&ext_id, &format!("{name} ({host})"));
+ ts.unit = Some("bytes".into());
+ ts.value_type = Some("BIGINT".into());
+ api.time_series.create_one(&ts)?;
+ }
+ Ok(())
+}
+
+/// Step 3: sample memory and swap via sysinfo, in bytes.
+fn sample() -> HashMap<&'static str, i64> {
+ let mut sys = System::new();
+ sys.refresh_memory();
+ HashMap::from([
+ ("system_memory_used", sys.used_memory() as i64),
+ ("system_memory_available", sys.available_memory() as i64),
+ ("system_swap_used", sys.used_swap() as i64),
+ ])
+}
+
+/// Cross-platform hostname lookup.
+fn hostname() -> String {
+ hostname::get()
+ .ok()
+ .and_then(|s| s.into_string().ok())
+ .map(|s| s.to_lowercase())
+ .unwrap_or_else(|| "localhost".into())
+}
+```
+
+
+
+
+## Try this next
+
+- **Add a metric.** Add one entry to the metric table (e.g. total memory) and extend
+ the sampler, series creation and ingestion both pick it up with no other changes.
+- **Namespace by host.** Run the program on two machines (or set `HOST_ID` to two
+ different values) and watch each get its own set of series.
+- **Swap the data source.** Replace the sampler with your own readings, the DataHub
+ steps don't change.
+- **Read it back.** Chart the trend with
+ [Query & aggregate time-series](/guides/query-and-aggregate), or tail it live with
+ [Real-time subscriptions](/guides/realtime-subscriptions).
+- **Scale it up.** When one datapoint per tick becomes millions, see
+ [High-throughput ingestion](/guides/ingest-timeseries).
diff --git a/versioned_sidebars/version-1.0-sidebars.json b/versioned_sidebars/version-1.0-sidebars.json
new file mode 100644
index 0000000..caea0c0
--- /dev/null
+++ b/versioned_sidebars/version-1.0-sidebars.json
@@ -0,0 +1,8 @@
+{
+ "tutorialSidebar": [
+ {
+ "type": "autogenerated",
+ "dirName": "."
+ }
+ ]
+}
diff --git a/versions.json b/versions.json
new file mode 100644
index 0000000..9a369ed
--- /dev/null
+++ b/versions.json
@@ -0,0 +1,3 @@
+[
+ "1.0"
+]