diff --git a/README.md b/README.md index a455316c..2221decd 100644 --- a/README.md +++ b/README.md @@ -4,13 +4,16 @@ [![Python Versions](https://img.shields.io/pypi/pyversions/neural-sdk.svg)](https://pypi.org/project/neural-sdk/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) -Professional-grade SDK for algorithmic trading on prediction markets, built as the public surface of the Neural stack. +Deterministic prediction-market kernel for portable contracts, normalization, +replay, and paper-first integration beneath Vaticor. [Documentation](https://github.com/IntelIP/Neural/tree/main/docs) | [Examples](./examples) | [Contributing](./CONTRIBUTING.md) -## Overview +## Stable Kernel -Neural SDK is the public Python control surface for market access, paper trading, provider discovery, and the CLI bridge consumed by the Neural TUI. +The dependency-free stable surface lives at `neural.kernel`. Provider access, +strategy analysis, deployment, sentiment, and FIX helpers remain experimental +or deprecated compatibility modules. ## Install @@ -28,6 +31,16 @@ pip install neural-sdk pip install "neural-sdk[trading]" ``` +Verify the stable installed surface without credentials or network access: + +```bash +neural --json capabilities +neural --json replay demo +``` + +See [Stable Kernel and Compatibility](./docs/architecture/stability.mdx) for +the capability matrix, extras, and deprecation policy. + ## CLI Bridge The base install ships a `neural` CLI intended to be the stable machine-readable bridge for the TypeScript Neural TUI. @@ -41,6 +54,7 @@ neural --json providers list Current bridge commands: - `doctor` - `capabilities` +- `replay demo` - `providers list` - `markets list` - `quote` diff --git a/docs/architecture/meta.json b/docs/architecture/meta.json index 1fd5ab32..d49b7645 100644 --- a/docs/architecture/meta.json +++ b/docs/architecture/meta.json @@ -1,4 +1,4 @@ { "title": "Architecture", - "pages": ["start-here", "overview"] + "pages": ["start-here", "overview", "stability"] } diff --git a/docs/architecture/stability.mdx b/docs/architecture/stability.mdx new file mode 100644 index 00000000..f0857a9c --- /dev/null +++ b/docs/architecture/stability.mdx @@ -0,0 +1,72 @@ +--- +title: Stable Kernel and Compatibility +description: Neural's supported kernel surface, experimental modules, and deprecation policy +--- + +Neural is the deterministic Python kernel beneath Vaticor. The stable base +does not authorize provider calls, live orders, deployment, or model-driven +side effects. + +## Stable surface + +Import stable capabilities from `neural.kernel`: + +```python +from neural.kernel import NormalizedMarket, ReplayEvent, replay +``` + +The stable surface contains: + +- normalized market, quote, order, position, capability, and policy types; +- dependency-free deterministic replay with a canonical SHA-256 digest; +- `neural doctor`, `neural capabilities`, and `neural replay demo`. + +Run the installed-wheel demo without credentials or network access: + +```bash +neural --json replay demo +``` + +The result contains a digest, event count, market count, and final snapshot. +The same fixture must produce the same result regardless of input order. + +## Capability matrix + +`neural --json capabilities` is the machine-readable source for the current +matrix. + +| Capability | Status | Install path | Compatibility | +| --- | --- | --- | --- | +| Kernel normalization | Stable | base wheel | Public `neural.kernel` contract | +| Kernel replay | Stable | base wheel | Deterministic output and digest | +| CLI diagnostics | Stable | base wheel | JSON envelope retained | +| Kalshi auth and collection | Experimental | `neural-sdk[trading]` | May change before 1.0 | +| Paper and venue adapters | Experimental | `neural-sdk[trading]` | Paper-first; no live authority | +| Strategy and backtesting | Experimental | `neural-sdk[analysis]` | May change before 1.0 | +| Sentiment | Deprecated | `neural-sdk[sentiment]` | Compatibility only | +| Generic deployment | Deprecated | `neural-sdk[deployment]` | Move runtime ownership to Vaticor | +| FIX experiments | Deprecated | `neural-sdk[fix]` | Compatibility only | + +Existing imports remain available during the `0.4.x` compatibility window. +New code should use `neural.kernel` for stable contracts. + +## Compatibility policy + +- Stable symbols retain compatible behavior within a minor release line. +- Experimental symbols can change before `1.0`; changes require release notes. +- Deprecated symbols remain through at least one later minor release and name + their replacement before removal. +- Removal never silently enables a provider, credential, deployment, or live + execution path. +- Model output remains untrusted. Schema or replay success cannot authorize an + external side effect. +- Exact wheel compatibility requires the Neural commit, wheel digest, Python + version, and validator result. Local source-path imports are insufficient. + +## Legacy import path + +Existing `neural.exchanges`, `neural.trading`, `neural.analysis`, +`neural.data_collection`, and `neural.deployment` imports are preserved for +compatibility. Install the matching extra before using those surfaces. Moving +to `neural.kernel` removes provider and optional-stack coupling from portable +contract and replay code. diff --git a/docs/getting-started.mdx b/docs/getting-started.mdx index e4b11da4..1c18eba1 100644 --- a/docs/getting-started.mdx +++ b/docs/getting-started.mdx @@ -1,6 +1,6 @@ --- title: Getting Started -description: Install, configure credentials, and ship your first Neural workflow +description: Install the stable kernel and run a deterministic replay --- Follow these steps in order. Every command has a matching verification check so you know the environment is ready before moving on. @@ -9,7 +9,7 @@ Follow these steps in order. Every command has a matching verification check so - Python 3.10 or newer - `pip` or `uv` -- Kalshi API key ID and RSA private key (downloadable from the Kalshi console) +- No credentials for the stable kernel path ## Step 1 – Install Neural @@ -29,13 +29,22 @@ uv add "neural-sdk[trading]" **Verify:** -```bash -python - <<'PY' -import neural -print("Neural version:", neural.__version__) -PY +```console +neural --json capabilities +neural --json replay demo ``` +Both commands run without provider credentials or network access. The replay +returns a deterministic digest and final market snapshot. + +See `architecture/stability` before importing experimental modules. + +## Experimental provider path + +Everything below this point uses experimental provider or analysis modules. +It is not part of the stable base contract and does not grant live-execution +authority. + ## Step 2 – Configure credentials 1. Place your keys in `secrets/` (keep the directory out of version control): @@ -124,11 +133,11 @@ python examples/build_first_bot.py This script fetches live markets, applies a toy signal, and routes simulated orders through `PaperTradingClient`. Use it as a template before wiring in your own strategy. -## Step 7 – Move into paper or live trading +## Step 7 – Continue with paper execution -1. Swap `TradingClient` in for live trading (credentials required). -2. Keep `PaperTradingClient` for rehearsals and backtesting comparisons. -3. Use `OrderManager` to keep strategy logic independent from the execution target. +1. Keep `PaperTradingClient` for rehearsals and backtesting comparisons. +2. Use `OrderManager` to keep strategy logic independent from the execution target. +3. Treat any provider write or live order as a separate human-approved action. ## Step 8 – Run tests diff --git a/docs/index.mdx b/docs/index.mdx index 13075265..2d9b4ff2 100644 --- a/docs/index.mdx +++ b/docs/index.mdx @@ -1,10 +1,10 @@ --- title: 'Neural SDK' -description: 'Build, test, and operate prediction-market systems with one Python SDK.' +description: 'Use a deterministic prediction-market kernel beneath Vaticor.' --- -Neural SDK is the public Python control surface for market data, strategy analysis, -paper trading, provider discovery, and automation. +Neural SDK supplies portable normalization, deterministic replay, and +paper-first compatibility primitives beneath the Vaticor product. ## Choose your path @@ -12,6 +12,9 @@ paper trading, provider discovery, and automation. Map data collection, analysis, and execution before writing code. + + Review supported imports, experimental extras, and deprecation policy. + Configure a local environment and run the first authenticated checks. @@ -21,6 +24,6 @@ paper trading, provider discovery, and automation. - Neural is beta software. Prediction-market trading can lose money. Validate - strategies with deterministic tests and paper execution before using live funds. + Neural is beta software. Stable kernel output cannot authorize provider + calls, deployment, or live execution. diff --git a/neural/__init__.py b/neural/__init__.py index c75d98f3..08269393 100644 --- a/neural/__init__.py +++ b/neural/__init__.py @@ -16,6 +16,7 @@ "data_collection", "deployment", "exchanges", + "kernel", "trading", } @@ -72,7 +73,6 @@ def __dir__() -> list[str]: return sorted(set(globals()) | _OPTIONAL_SUBMODULES) - __all__ = [ "__version__", "analysis", @@ -80,7 +80,7 @@ def __dir__() -> list[str]: "data_collection", "deployment", "exchanges", + "kernel", "trading", "_warn_experimental", ] - diff --git a/neural/cli.py b/neural/cli.py index e263dfae..c64db7d6 100644 --- a/neural/cli.py +++ b/neural/cli.py @@ -19,6 +19,7 @@ CLI_COMMANDS = [ "doctor", "capabilities", + "replay demo", "providers list", "markets list", "quote", @@ -68,9 +69,18 @@ def _build_parser() -> argparse.ArgumentParser: ) capabilities_parser.set_defaults(handler=_handle_capabilities, formatter=_format_capabilities) + replay_parser = subparsers.add_parser("replay", help="Run deterministic Neural kernel replays") + replay_subparsers = replay_parser.add_subparsers(dest="replay_command") + replay_demo_parser = replay_subparsers.add_parser( + "demo", help="Run the dependency-free deterministic fixture" + ) + replay_demo_parser.set_defaults(handler=_handle_replay_demo) + providers_parser = subparsers.add_parser("providers", help="Inspect deployment providers") providers_subparsers = providers_parser.add_subparsers(dest="providers_command") - providers_list_parser = providers_subparsers.add_parser("list", help="List discovered providers") + providers_list_parser = providers_subparsers.add_parser( + "list", help="List discovered providers" + ) providers_list_parser.set_defaults(handler=_handle_providers_list, formatter=_format_providers) markets_parser = subparsers.add_parser("markets", help="Query Neural market data") @@ -102,7 +112,9 @@ def _build_parser() -> argparse.ArgumentParser: deployments_parser = subparsers.add_parser("deployments", help="Inspect runtime deployments") deployments_parser.add_argument("--provider", default=os.getenv("NEURAL_DEPLOYMENT_PROVIDER")) - deployments_parser.add_argument("--workspace-name", default=os.getenv("NEURAL_DAYTONA_WORKSPACE")) + deployments_parser.add_argument( + "--workspace-name", default=os.getenv("NEURAL_DAYTONA_WORKSPACE") + ) deployments_parser.add_argument("--runner-image", default=os.getenv("NEURAL_DAYTONA_IMAGE")) deployments_parser.add_argument("--project-name", default="neural") deployments_parser.add_argument("--environment", default="paper") @@ -110,15 +122,21 @@ def _build_parser() -> argparse.ArgumentParser: deployments_subparsers = deployments_parser.add_subparsers(dest="deployments_command") deployments_list_parser = deployments_subparsers.add_parser("list", help="List deployments") - deployments_list_parser.set_defaults(handler=_handle_deployments_list, formatter=_format_deployments) + deployments_list_parser.set_defaults( + handler=_handle_deployments_list, formatter=_format_deployments + ) - deployments_status_parser = deployments_subparsers.add_parser("status", help="Get deployment status") + deployments_status_parser = deployments_subparsers.add_parser( + "status", help="Get deployment status" + ) deployments_status_parser.add_argument("deployment_id") deployments_status_parser.set_defaults( handler=_handle_deployments_status, formatter=_format_deployment_status ) - deployments_logs_parser = deployments_subparsers.add_parser("logs", help="Fetch deployment logs") + deployments_logs_parser = deployments_subparsers.add_parser( + "logs", help="Fetch deployment logs" + ) deployments_logs_parser.add_argument("deployment_id") deployments_logs_parser.add_argument("--tail", type=int, default=50) deployments_logs_parser.set_defaults( @@ -166,8 +184,15 @@ def _handle_doctor(_: argparse.Namespace) -> dict[str, Any]: def _handle_capabilities(_: argparse.Namespace) -> dict[str, Any]: + from neural.kernel import capability_matrix + providers = _safe_list_providers() return { + "kernel": { + "stable": True, + "dependency_free": True, + "capabilities": capability_matrix(), + }, "cli": { "commands": CLI_COMMANDS, "json_envelope": { @@ -193,6 +218,12 @@ def _handle_capabilities(_: argparse.Namespace) -> dict[str, Any]: } +def _handle_replay_demo(_: argparse.Namespace) -> dict[str, Any]: + from neural.kernel import run_demo_replay + + return {"replay": run_demo_replay().as_dict()} + + def _handle_providers_list(_: argparse.Namespace) -> dict[str, Any]: providers = _safe_list_providers() return {"providers": providers, "count": len(providers)} @@ -315,7 +346,11 @@ def _handle_deployments_logs(args: argparse.Namespace) -> dict[str, Any]: def _handle_deployments_stop(args: argparse.Namespace) -> dict[str, Any]: provider_name, provider = _build_provider(args) stopped = asyncio.run(provider.stop(args.deployment_id)) - return {"provider": provider_name, "deployment_id": args.deployment_id, "stopped": bool(stopped)} + return { + "provider": provider_name, + "deployment_id": args.deployment_id, + "stopped": bool(stopped), + } def _build_provider(args: argparse.Namespace) -> tuple[str, Any]: diff --git a/neural/kernel/__init__.py b/neural/kernel/__init__.py new file mode 100644 index 00000000..190dd21c --- /dev/null +++ b/neural/kernel/__init__.py @@ -0,0 +1,54 @@ +"""Stable, dependency-free Neural kernel surface.""" + +from neural.exchanges.types import ( + ExchangeCapabilities, + ExchangeName, + NormalizedMarket, + NormalizedOrderRequest, + NormalizedOrderResult, + NormalizedPosition, + NormalizedQuote, + OrderSide, + OrderType, + TradingPolicy, +) + +from .capabilities import ( + CAPABILITIES, + Capability, + CapabilityStatus, + capability_matrix, + get_capability, +) +from .replay import ( + DEMO_REPLAY_DIGEST, + ReplayEvent, + ReplayResult, + demo_events, + replay, + run_demo_replay, +) + +__all__ = [ + "CAPABILITIES", + "Capability", + "CapabilityStatus", + "DEMO_REPLAY_DIGEST", + "ExchangeCapabilities", + "ExchangeName", + "NormalizedMarket", + "NormalizedOrderRequest", + "NormalizedOrderResult", + "NormalizedPosition", + "NormalizedQuote", + "OrderSide", + "OrderType", + "ReplayEvent", + "ReplayResult", + "TradingPolicy", + "capability_matrix", + "demo_events", + "get_capability", + "replay", + "run_demo_replay", +] diff --git a/neural/kernel/capabilities.py b/neural/kernel/capabilities.py new file mode 100644 index 00000000..a71ce1f2 --- /dev/null +++ b/neural/kernel/capabilities.py @@ -0,0 +1,124 @@ +"""Machine-readable stability contract for Neural package capabilities.""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +from enum import Enum + + +class CapabilityStatus(str, Enum): + """Compatibility level for one public capability.""" + + STABLE = "stable" + EXPERIMENTAL = "experimental" + DEPRECATED = "deprecated" + + +@dataclass(frozen=True, slots=True) +class Capability: + """One capability and its supported installation path.""" + + name: str + module: str + status: CapabilityStatus + extra: str | None + summary: str + replacement: str | None = None + + def as_dict(self) -> dict[str, str | None]: + """Return a JSON-compatible representation.""" + payload = asdict(self) + payload["status"] = self.status.value + return payload + + +CAPABILITIES: tuple[Capability, ...] = ( + Capability( + name="kernel.normalization", + module="neural.kernel", + status=CapabilityStatus.STABLE, + extra=None, + summary="JSON-compatible market, quote, order, position, and policy types.", + ), + Capability( + name="kernel.replay", + module="neural.kernel", + status=CapabilityStatus.STABLE, + extra=None, + summary="Dependency-free deterministic fixture replay and digest.", + ), + Capability( + name="cli.diagnostics", + module="neural.cli", + status=CapabilityStatus.STABLE, + extra=None, + summary="Machine-readable doctor, capabilities, and replay demo commands.", + ), + Capability( + name="auth.kalshi", + module="neural.auth", + status=CapabilityStatus.EXPERIMENTAL, + extra="trading", + summary="Kalshi authentication and signed HTTP clients.", + ), + Capability( + name="data_collection", + module="neural.data_collection", + status=CapabilityStatus.EXPERIMENTAL, + extra="analysis", + summary="Provider-specific market collection and tabular normalization.", + ), + Capability( + name="trading.paper", + module="neural.trading", + status=CapabilityStatus.EXPERIMENTAL, + extra="trading", + summary="Paper portfolio and venue adapter utilities.", + ), + Capability( + name="analysis", + module="neural.analysis", + status=CapabilityStatus.EXPERIMENTAL, + extra="analysis", + summary="Strategy, risk-sizing, and backtesting utilities.", + ), + Capability( + name="analysis.sentiment", + module="neural.analysis.sentiment", + status=CapabilityStatus.DEPRECATED, + extra="sentiment", + summary="Legacy sentiment adapters retained for compatibility.", + replacement="Evidence-backed analysis in the Vaticor product workflow.", + ), + Capability( + name="deployment", + module="neural.deployment", + status=CapabilityStatus.DEPRECATED, + extra="deployment", + summary="Legacy generic deployment helpers retained for compatibility.", + replacement="A separately owned Vaticor runtime provider.", + ), + Capability( + name="trading.fix", + module="neural.trading.fix", + status=CapabilityStatus.DEPRECATED, + extra="fix", + summary="Experimental FIX helpers retained for compatibility.", + replacement="Kalshi conformance through the supported adapter boundary.", + ), +) + +_CAPABILITIES_BY_NAME = {capability.name: capability for capability in CAPABILITIES} + + +def capability_matrix() -> list[dict[str, str | None]]: + """Return the ordered public capability matrix.""" + return [capability.as_dict() for capability in CAPABILITIES] + + +def get_capability(name: str) -> Capability: + """Return one named capability or raise a clear lookup error.""" + try: + return _CAPABILITIES_BY_NAME[name] + except KeyError as exc: + raise KeyError(f"Unknown Neural capability: {name}") from exc diff --git a/neural/kernel/replay.py b/neural/kernel/replay.py new file mode 100644 index 00000000..adf5bcb8 --- /dev/null +++ b/neural/kernel/replay.py @@ -0,0 +1,204 @@ +"""Dependency-free deterministic replay primitives.""" + +from __future__ import annotations + +import hashlib +import json +from collections.abc import Iterable, Mapping +from dataclasses import dataclass +from datetime import datetime, timezone +from decimal import Decimal, InvalidOperation +from typing import Any + +DEMO_REPLAY_DIGEST = "569f87947428d4d093425134181b754ca974675aa7c74fc2cb73a8e193f5b4e6" + + +def _decimal(value: Decimal | float | int | str, *, field: str) -> Decimal: + try: + parsed = Decimal(str(value)) + except (InvalidOperation, ValueError) as exc: + raise ValueError(f"{field} must be a finite decimal") from exc + if not parsed.is_finite(): + raise ValueError(f"{field} must be a finite decimal") + return parsed + + +def _price(value: Decimal | float | int | str, *, field: str) -> Decimal: + parsed = _decimal(value, field=field) + if parsed < 0 or parsed > 1: + raise ValueError(f"{field} must be between 0 and 1") + return parsed + + +def _timestamp(value: str) -> str: + normalized = value.replace("Z", "+00:00") + try: + parsed = datetime.fromisoformat(normalized) + except ValueError as exc: + raise ValueError("observed_at must be an ISO 8601 timestamp") from exc + if parsed.tzinfo is None or parsed.utcoffset() is None: + raise ValueError("observed_at must include a timezone") + return parsed.astimezone(timezone.utc).isoformat().replace("+00:00", "Z") + + +def _decimal_text(value: Decimal) -> str: + if value.is_zero(): + return "0" + text = format(value, "f") + return text.rstrip("0").rstrip(".") if "." in text else text + + +@dataclass(frozen=True, slots=True) +class ReplayEvent: + """One normalized, immutable market observation.""" + + observed_at: str + market_id: str + yes_price: Decimal + no_price: Decimal + source: str = "fixture" + volume: Decimal = Decimal("0") + + def __post_init__(self) -> None: + object.__setattr__(self, "observed_at", _timestamp(self.observed_at)) + object.__setattr__(self, "market_id", self.market_id.strip()) + object.__setattr__(self, "source", self.source.strip()) + object.__setattr__(self, "yes_price", _price(self.yes_price, field="yes_price")) + object.__setattr__(self, "no_price", _price(self.no_price, field="no_price")) + object.__setattr__(self, "volume", _decimal(self.volume, field="volume")) + if not self.market_id: + raise ValueError("market_id must not be empty") + if not self.source: + raise ValueError("source must not be empty") + if self.volume < 0: + raise ValueError("volume must be non-negative") + + @classmethod + def from_mapping(cls, value: Mapping[str, Any]) -> ReplayEvent: + """Build an event from a JSON-compatible mapping.""" + return cls( + observed_at=str(value["observed_at"]), + market_id=str(value["market_id"]), + yes_price=_price(value["yes_price"], field="yes_price"), + no_price=_price(value["no_price"], field="no_price"), + source=str(value.get("source", "fixture")), + volume=_decimal(value.get("volume", 0), field="volume"), + ) + + def as_dict(self) -> dict[str, str]: + """Return the canonical JSON-compatible event.""" + return { + "market_id": self.market_id, + "no_price": _decimal_text(self.no_price), + "observed_at": self.observed_at, + "source": self.source, + "volume": _decimal_text(self.volume), + "yes_price": _decimal_text(self.yes_price), + } + + +@dataclass(frozen=True, slots=True) +class ReplayResult: + """Canonical replay output bound to a SHA-256 digest.""" + + events: tuple[ReplayEvent, ...] + snapshot: tuple[ReplayEvent, ...] + digest: str + + def as_dict(self) -> dict[str, Any]: + """Return a JSON-compatible replay result.""" + return { + "digest": self.digest, + "event_count": len(self.events), + "market_count": len(self.snapshot), + "snapshot": [event.as_dict() for event in self.snapshot], + } + + +def _event_sort_key(event: ReplayEvent) -> tuple[datetime, str, str, str, str, str]: + payload = event.as_dict() + return ( + datetime.fromisoformat(payload["observed_at"].replace("Z", "+00:00")), + payload["source"], + payload["market_id"], + payload["yes_price"], + payload["no_price"], + payload["volume"], + ) + + +def replay(events: Iterable[ReplayEvent | Mapping[str, Any]]) -> ReplayResult: + """Replay observations into a deterministic final market snapshot.""" + normalized = tuple( + sorted( + ( + event if isinstance(event, ReplayEvent) else ReplayEvent.from_mapping(event) + for event in events + ), + key=_event_sort_key, + ) + ) + if not normalized: + raise ValueError("replay requires at least one event") + + latest: dict[tuple[str, str], ReplayEvent] = {} + for event in normalized: + latest[(event.source, event.market_id)] = event + snapshot = tuple(sorted(latest.values(), key=_event_sort_key)) + + payload = { + "events": [event.as_dict() for event in normalized], + "snapshot": [event.as_dict() for event in snapshot], + } + canonical = json.dumps(payload, separators=(",", ":"), sort_keys=True).encode("utf-8") + return ReplayResult( + events=normalized, + snapshot=snapshot, + digest=hashlib.sha256(canonical).hexdigest(), + ) + + +def demo_events() -> tuple[ReplayEvent, ...]: + """Return a small deterministic fixture with two markets and two rounds.""" + return ( + ReplayEvent( + observed_at="2026-07-29T12:00:00Z", + source="kalshi-fixture", + market_id="KX-DEMO-YES", + yes_price=Decimal("0.45"), + no_price=Decimal("0.55"), + volume=Decimal("10"), + ), + ReplayEvent( + observed_at="2026-07-29T12:01:00Z", + source="kalshi-fixture", + market_id="KX-DEMO-YES", + yes_price=Decimal("0.52"), + no_price=Decimal("0.48"), + volume=Decimal("14"), + ), + ReplayEvent( + observed_at="2026-07-29T12:00:00Z", + source="kalshi-fixture", + market_id="KX-DEMO-NO", + yes_price=Decimal("0.61"), + no_price=Decimal("0.39"), + volume=Decimal("7"), + ), + ReplayEvent( + observed_at="2026-07-29T12:01:00Z", + source="kalshi-fixture", + market_id="KX-DEMO-NO", + yes_price=Decimal("0.58"), + no_price=Decimal("0.42"), + volume=Decimal("9"), + ), + ) + + +def run_demo_replay() -> ReplayResult: + """Run the built-in deterministic replay fixture.""" + result = replay(demo_events()) + if result.digest != DEMO_REPLAY_DIGEST: + raise RuntimeError("built-in replay fixture digest changed") + return result diff --git a/pyproject.toml b/pyproject.toml index 5f7ba9e6..169e73e8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -76,6 +76,10 @@ analysis = [ trading = [ "kalshi-python>=2.1.4,<3", ] +fix = [ + "simplefix>=1.0.17", + "cryptography>=41.0.0", +] sentiment = [ "textblob>=0.17.1", "vaderSentiment>=3.3.2", diff --git a/scripts/release_candidate_smoke.py b/scripts/release_candidate_smoke.py index 4ab6e864..ccc82ab5 100644 --- a/scripts/release_candidate_smoke.py +++ b/scripts/release_candidate_smoke.py @@ -7,6 +7,26 @@ from importlib.metadata import PackageNotFoundError, version from pathlib import Path +EXPECTED_DEMO_REPLAY_DIGEST = "569f87947428d4d093425134181b754ca974675aa7c74fc2cb73a8e193f5b4e6" +EXPECTED_DEMO_SNAPSHOT = [ + { + "market_id": "KX-DEMO-NO", + "no_price": "0.42", + "observed_at": "2026-07-29T12:01:00Z", + "source": "kalshi-fixture", + "volume": "9", + "yes_price": "0.58", + }, + { + "market_id": "KX-DEMO-YES", + "no_price": "0.48", + "observed_at": "2026-07-29T12:01:00Z", + "source": "kalshi-fixture", + "volume": "14", + "yes_price": "0.52", + }, +] + class ReleaseSmokeError(AssertionError): """An installed release artifact failed its smoke contract.""" @@ -31,6 +51,21 @@ def validate_install( raise ReleaseSmokeError(f"neural imported outside an installed package: {module_file}") +def validate_kernel_replay(result: dict[str, object]) -> None: + """Validate the installed dependency-free replay contract.""" + digest = result.get("digest") + if not isinstance(digest, str) or len(digest) != 64: + raise ReleaseSmokeError("kernel replay did not produce a SHA-256 digest") + if digest != EXPECTED_DEMO_REPLAY_DIGEST: + raise ReleaseSmokeError("kernel replay digest changed") + if result.get("event_count") != 4: + raise ReleaseSmokeError("kernel replay event count changed") + if result.get("market_count") != 2: + raise ReleaseSmokeError("kernel replay market count changed") + if result.get("snapshot") != EXPECTED_DEMO_SNAPSHOT: + raise ReleaseSmokeError("kernel replay snapshot changed") + + def main() -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--expected-version", required=True) @@ -42,6 +77,7 @@ def main() -> int: raise ReleaseSmokeError("neural-sdk distribution is not installed") from exc import neural + from neural.kernel import run_demo_replay validate_install( args.expected_version, @@ -49,6 +85,7 @@ def main() -> int: getattr(neural, "__version__", None), Path(neural.__file__).resolve(), ) + validate_kernel_replay(run_demo_replay().as_dict()) return 0 diff --git a/tests/kernel/test_capabilities.py b/tests/kernel/test_capabilities.py new file mode 100644 index 00000000..c5054773 --- /dev/null +++ b/tests/kernel/test_capabilities.py @@ -0,0 +1,50 @@ +from __future__ import annotations + +import json +import subprocess +import sys + +import pytest + +from neural.kernel import CapabilityStatus, capability_matrix, get_capability + + +def test_capability_matrix_is_unique_and_has_stable_kernel() -> None: + matrix = capability_matrix() + names = [item["name"] for item in matrix] + + assert len(names) == len(set(names)) + assert get_capability("kernel.replay").status is CapabilityStatus.STABLE + assert get_capability("deployment").status is CapabilityStatus.DEPRECATED + + +def test_unknown_capability_fails_clearly() -> None: + with pytest.raises(KeyError, match="Unknown Neural capability"): + get_capability("missing") + + +def test_stable_kernel_import_does_not_load_optional_stacks() -> None: + code = """ +import json +import sys +import neural +from neural.kernel import run_demo_replay +run_demo_replay() +blocked = [ + name for name in ( + "aiohttp", "docker", "jinja2", "numpy", "pandas", "plotly", + "pydantic", "requests", "simplefix", "sqlalchemy", "torch", + "transformers", "websocket", "websockets" + ) + if name in sys.modules +] +print(json.dumps(blocked)) +""" + completed = subprocess.run( + [sys.executable, "-c", code], + check=True, + capture_output=True, + text=True, + ) + + assert json.loads(completed.stdout) == [] diff --git a/tests/kernel/test_replay.py b/tests/kernel/test_replay.py new file mode 100644 index 00000000..b12fc66f --- /dev/null +++ b/tests/kernel/test_replay.py @@ -0,0 +1,106 @@ +from __future__ import annotations + +from decimal import Decimal, localcontext + +import pytest + +from neural.kernel import ( + DEMO_REPLAY_DIGEST, + ReplayEvent, + demo_events, + replay, + run_demo_replay, +) + + +def test_demo_replay_is_deterministic_across_input_order() -> None: + expected = run_demo_replay() + reversed_result = replay(reversed(demo_events())) + + assert expected.digest == reversed_result.digest + assert expected.digest == DEMO_REPLAY_DIGEST + assert expected.as_dict() == reversed_result.as_dict() + assert len(expected.digest) == 64 + assert expected.as_dict()["event_count"] == 4 + assert expected.as_dict()["market_count"] == 2 + + +def test_replay_snapshot_keeps_latest_market_observation() -> None: + result = run_demo_replay() + snapshot = {event.market_id: event for event in result.snapshot} + + assert snapshot["KX-DEMO-YES"].yes_price == Decimal("0.52") + assert snapshot["KX-DEMO-NO"].no_price == Decimal("0.42") + + +def test_replay_orders_fractional_timestamps_chronologically() -> None: + result = replay( + [ + { + "observed_at": "2026-07-29T12:00:00.100000Z", + "market_id": "KX-DEMO", + "yes_price": "0.6", + "no_price": "0.4", + }, + { + "observed_at": "2026-07-29T12:00:00Z", + "market_id": "KX-DEMO", + "yes_price": "0.5", + "no_price": "0.5", + }, + ] + ) + + assert result.snapshot[0].yes_price == Decimal("0.6") + + +def test_replay_digest_is_independent_of_decimal_context() -> None: + expected = run_demo_replay() + + with localcontext() as context: + context.prec = 1 + actual = run_demo_replay() + + assert actual.digest == expected.digest + assert actual.as_dict() == expected.as_dict() + + +def test_replay_normalizes_timestamps_to_utc() -> None: + event = ReplayEvent.from_mapping( + { + "observed_at": "2026-07-29T13:00:00+01:00", + "market_id": "KX-DEMO", + "yes_price": "0.5", + "no_price": "0.5", + } + ) + + assert event.observed_at == "2026-07-29T12:00:00Z" + + +@pytest.mark.parametrize( + ("field", "value", "message"), + [ + ("yes_price", "1.01", "between 0 and 1"), + ("no_price", "-0.01", "between 0 and 1"), + ("volume", "-1", "non-negative"), + ("observed_at", "2026-07-29T12:00:00", "include a timezone"), + ], +) +def test_replay_event_rejects_invalid_input(field: str, value: str, message: str) -> None: + payload = { + "observed_at": "2026-07-29T12:00:00Z", + "market_id": "KX-DEMO", + "yes_price": "0.5", + "no_price": "0.5", + "volume": "1", + } + payload[field] = value + + with pytest.raises(ValueError, match=message): + ReplayEvent.from_mapping(payload) + + +def test_empty_replay_is_rejected() -> None: + with pytest.raises(ValueError, match="at least one event"): + replay([]) diff --git a/tests/test_cli.py b/tests/test_cli.py index 1a0e3a91..97f74b97 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -45,10 +45,29 @@ def test_capabilities_json_output(monkeypatch: pytest.MonkeyPatch) -> None: assert exit_code == 0 payload = json.loads(stdout.getvalue()) assert payload["ok"] is True + assert payload["data"]["kernel"]["stable"] is True + assert payload["data"]["kernel"]["dependency_free"] is True + assert any( + item["name"] == "kernel.replay" for item in payload["data"]["kernel"]["capabilities"] + ) assert payload["data"]["platform"]["private_provider_installed"] is True assert "stop" in payload["data"]["platform"]["deployment_controls"] +def test_replay_demo_json_output(monkeypatch: pytest.MonkeyPatch) -> None: + stdout, stderr = _capture(monkeypatch) + + exit_code = cli.main(["--json", "replay", "demo"]) + + assert exit_code == 0 + assert stderr.getvalue() == "" + payload = json.loads(stdout.getvalue()) + assert payload["ok"] is True + assert payload["data"]["replay"]["event_count"] == 4 + assert payload["data"]["replay"]["market_count"] == 2 + assert len(payload["data"]["replay"]["digest"]) == 64 + + def test_paper_order_json_output(monkeypatch: pytest.MonkeyPatch) -> None: stdout, _ = _capture(monkeypatch) @@ -204,6 +223,7 @@ def test_python_module_cli_json_is_clean_on_stderr() -> None: payload = json.loads(result.stdout) assert payload["ok"] is True + def test_missing_subcommand_json_output(monkeypatch: pytest.MonkeyPatch) -> None: stdout, stderr = _capture(monkeypatch) diff --git a/tests/test_release_candidate_validation.py b/tests/test_release_candidate_validation.py index 9cfe9ec3..6db85308 100644 --- a/tests/test_release_candidate_validation.py +++ b/tests/test_release_candidate_validation.py @@ -5,7 +5,12 @@ import pytest -from scripts.release_candidate_smoke import ReleaseSmokeError, validate_install +from scripts.release_candidate_smoke import ( + EXPECTED_DEMO_REPLAY_DIGEST, + ReleaseSmokeError, + validate_install, + validate_kernel_replay, +) from scripts.validate_release_candidate import ( ReleaseCandidateError, project_version, @@ -189,6 +194,56 @@ def test_installed_smoke_requires_version_agreement_and_installed_path() -> None ) +def test_installed_smoke_requires_deterministic_kernel_replay() -> None: + validate_kernel_replay( + { + "digest": EXPECTED_DEMO_REPLAY_DIGEST, + "event_count": 4, + "market_count": 2, + "snapshot": [ + { + "market_id": "KX-DEMO-NO", + "no_price": "0.42", + "observed_at": "2026-07-29T12:01:00Z", + "source": "kalshi-fixture", + "volume": "9", + "yes_price": "0.58", + }, + { + "market_id": "KX-DEMO-YES", + "no_price": "0.48", + "observed_at": "2026-07-29T12:01:00Z", + "source": "kalshi-fixture", + "volume": "14", + "yes_price": "0.52", + }, + ], + } + ) + + with pytest.raises(ReleaseSmokeError, match="digest"): + validate_kernel_replay({"digest": "", "event_count": 4, "market_count": 2}) + with pytest.raises(ReleaseSmokeError, match="digest changed"): + validate_kernel_replay({"digest": "a" * 64, "event_count": 4, "market_count": 2}) + with pytest.raises(ReleaseSmokeError, match="event count"): + validate_kernel_replay( + { + "digest": EXPECTED_DEMO_REPLAY_DIGEST, + "event_count": 3, + "market_count": 2, + } + ) + with pytest.raises(ReleaseSmokeError, match="snapshot"): + validate_kernel_replay( + { + "digest": EXPECTED_DEMO_REPLAY_DIGEST, + "event_count": 4, + "market_count": 2, + "snapshot": [], + } + ) + + def test_workflow_is_read_only_non_publishing_and_sha_bound() -> None: workflow = Path(".github/workflows/release-candidate-validation.yml").read_text( encoding="utf-8" diff --git a/uv.lock b/uv.lock index 5d9725bf..0da4ba53 100644 --- a/uv.lock +++ b/uv.lock @@ -1668,6 +1668,10 @@ dev = [ { name = "twine" }, { name = "types-requests" }, ] +fix = [ + { name = "cryptography" }, + { name = "simplefix" }, +] sentiment = [ { name = "scikit-learn", version = "1.7.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "scikit-learn", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, @@ -1688,6 +1692,7 @@ requires-dist = [ { name = "bump2version", marker = "extra == 'dev'", specifier = ">=1.0.0" }, { name = "certifi", specifier = ">=2023.0.0" }, { name = "cryptography", specifier = ">=41.0.0" }, + { name = "cryptography", marker = "extra == 'fix'", specifier = ">=41.0.0" }, { name = "docker", marker = "extra == 'deployment'", specifier = ">=6.1.0" }, { name = "fastapi", marker = "extra == 'deployment'", specifier = ">=0.100.0" }, { name = "jinja2", specifier = ">=3.1.0" }, @@ -1706,6 +1711,7 @@ requires-dist = [ { name = "ruff", marker = "extra == 'dev'", specifier = ">=0.0.282" }, { name = "scikit-learn", marker = "extra == 'sentiment'", specifier = ">=1.1.0" }, { name = "simplefix", specifier = ">=1.0.17" }, + { name = "simplefix", marker = "extra == 'fix'", specifier = ">=1.0.17" }, { name = "sqlalchemy", marker = "extra == 'deployment'", specifier = ">=2.0.0" }, { name = "textblob", marker = "extra == 'sentiment'", specifier = ">=0.17.1" }, { name = "tomli", marker = "python_full_version < '3.11' and extra == 'dev'", specifier = ">=2.0.1" }, @@ -1718,7 +1724,7 @@ requires-dist = [ { name = "websocket-client", specifier = ">=1.7.0" }, { name = "websockets", specifier = ">=11.0.0" }, ] -provides-extras = ["dev", "analysis", "trading", "sentiment", "deployment"] +provides-extras = ["dev", "analysis", "trading", "fix", "sentiment", "deployment"] [[package]] name = "nh3"