From a40ffd2df8c83ce42aaa234c4f212b0286a74030 Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 15:42:56 +0800 Subject: [PATCH 01/11] test(bounty): verify public lifecycle and CLI retry after restart --- PROGRESS.md | 32 ++++--- docs/BOUNTY_ACCEPTANCE.md | 53 +++++++++++ tests/test_bounty_public_lifecycle.py | 126 ++++++++++++++++++++++++++ 3 files changed, 199 insertions(+), 12 deletions(-) create mode 100644 docs/BOUNTY_ACCEPTANCE.md create mode 100644 tests/test_bounty_public_lifecycle.py diff --git a/PROGRESS.md b/PROGRESS.md index 10aa1508..4ab2643d 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -5,29 +5,37 @@ ## 本轮基线与完成事项 -- 已核对 16 个开放 issue(#64、#65、#68、#69、#70、#72、#76–#85)与唯一开放 PR #186。 +- 已核对 16 个开放 issue(#64、#65、#68、#69、#70、#72、#76–#85); + #186、#187、#188 已审查并按 expected head 依次合并。 - 已读取当前权威设计及有效归档;现有功能和历史协议保持原义。 -- #186 已审查并合并为 `dfb89fe989eaef66c99f14ae6a953b07204b7170`。 - 主线 tree `e1d65b624266527bef5f66d2843fa1a40c4ddc5f` 与实际通过测试的 tree 相同。 -- [核心验收 36632579472](https://github.com/TokenNotIncluded/msg.lmm.best/actions/runs/36632579472): - 八份 ZIP 摘要、manifest 和 JUnit 真实 node-ID 已核验,2541 core + 8 conformance, +- 当前基线 main `c8ed2610ab774dba3085972683c1d2c8a9232222`, + tree `89f8e4119e625cb421b75be468392b2d7b187930` 与 #188 实际验收树一致。 +- [核心验收 36644965337](https://github.com/TokenNotIncluded/msg.lmm.best/actions/runs/36644965337): + 八份 ZIP 摘要、manifest 和 JUnit 真实 node-ID 已核验,2569 core + 8 conformance, 无失败、错误、跳过、重复或遗漏。 -- [质量 36632578929](https://github.com/TokenNotIncluded/msg.lmm.best/actions/runs/36632578929): - Ruff JSON 零诊断、stderr 为空、585 文件格式、Python 3.15.0rc2 编译通过。 -- [部署演练 36632578807](https://github.com/TokenNotIncluded/msg.lmm.best/actions/runs/36632578807): +- [质量 36644965239](https://github.com/TokenNotIncluded/msg.lmm.best/actions/runs/36644965239): + Ruff JSON 零诊断、stderr 为空、588 文件格式、Python 3.15.0rc2 编译通过。 +- [部署演练 36644965142](https://github.com/TokenNotIncluded/msg.lmm.best/actions/runs/36644965142): 同一 wheel 的锁定 server/client 安装、真实启动/重启、四传输、只读 doctor、 禁外发 worker、真实 OpenSSH/Git 撤权及正式隔离 selftest 42 项均通过。 - wheel/sdist 与核心构建产物逐字节一致;精确摘要见 PR #186。 + wheel/sdist 与核心构建产物逐字节一致;全部十份 ZIP、摘要与测量精确证据见 + [PR #188](https://github.com/TokenNotIncluded/msg.lmm.best/pull/188#issuecomment-5906443148)。 + 合并后独立主线部署/质量/恢复专项已 success,发布四文件与已验 #188 逐字节一致; + 主线 Rewrite 完整门槛仍排队,不能提前算通过。 ## 当前推进 -- 增加安装包的有界并发写入/读取测量、真实 SIGKILL 重启与全表/sequence 不变量检查。 -- 增加安装包的真实 PostgreSQL/Git/CAS 备份恢复计时和全表值比较;只允许恢复流程 +- 已完成安装包的有界并发写入/读取测量、真实 SIGKILL 重启与全表/sequence 不变量检查。 +- 已完成安装包的真实 PostgreSQL/Git/CAS 备份恢复计时和全表值比较;只允许恢复流程 明确规定的 runtime_config/quarantine 变化,验证删 marker 后仍隔离业务与幂等重放。 - 首次新演练 [36643217304](https://github.com/TokenNotIncluded/msg.lmm.best/actions/runs/36643217304) 在真实恢复时失败:私有 Git 打包引用后,ZIP 丢失空 refs 目录,Git 无法识别仓库。 本轮修复私有/用户仓库统一结构目录重建,保持只读 proof 校验不修改磁盘, - 并新增 packed-ref 恢复、只读拒绝和符号链接拒绝回归。新最终 tree 仍须重新完整验收。 + 并新增 packed-ref 恢复、只读拒绝和符号链接拒绝回归;#187 修复后的完整验收已通过。 +- #188 新增真实 COMMIT 故障回滚、主体奖励限额/版本拒绝,以及 R1–R5 各轮 + nonce/question/单轮 TTL/答案、跨主体与并发授勋回归,均已在完整门槛执行。 +- 当前补核对 #72:六种公开搜索/真实 CLI/历史签署内容状态、CLI PoP 真实重启重试; + `docs/BOUNTY_ACCEPTANCE.md` 给出原清单逐断言映射,新回归须云端实际通过后收口。 - 逐条核对 issue 中尚未刷新到当前源码的描述,保留原验收范围和历史证据。 ## issue 交接 diff --git a/docs/BOUNTY_ACCEPTANCE.md b/docs/BOUNTY_ACCEPTANCE.md new file mode 100644 index 00000000..1e02fe33 --- /dev/null +++ b/docs/BOUNTY_ACCEPTANCE.md @@ -0,0 +1,53 @@ +# Bounty / issue #72 acceptance + +Scope: the prefunded PoP market contract and its isolated official fixture. +The source base is main `c8ed2610ab774dba3085972683c1d2c8a9232222`. +CI provenance belongs to the exact tested tree; final results are recorded in +the PR and issue, rather than inferred from test names or this document. + +## Requirement to assertion map + +All node IDs below are relative to `tests/`. Parameterized nodes must all run. +The tests use actual PostgreSQL and the registered executor unless a row +explicitly describes a CLI parsing or pure-policy test. + +| Requirement | Test node or installed check | Actual assertion | +| --- | --- | --- | +| Integer, protected prefunding; publisher can be offline | `test_bounty.py::test_prefunded_bounty_pays_offline_publisher_and_stops_at_budget` | BountyEscrow is a LedgerAccount, has no Subject or credential; one funding leg, one payout, exact recipient/escrow balances and total supply; no publisher call needed to claim | +| Current catalog and immutable historical terms | `test_bounty.py::test_bounty_catalog_current_projection_and_historical_revision` | Underfunded, top-up, paid, closed projections; historical budget/state and terms Revision remain original while current fields report live values | +| Search and real CLI across all lifecycle states | `test_bounty_public_lifecycle.py::test_search_and_real_cli_preserve_terms_and_current_accounting` | Six transitions: underfunded/top-up/publisher pause/claims exhausted/inclusive expiry/close. Both public search APIs locate the exact Revision; real CLI get/store get agree on state, reason, budget, escrow and claims; historical signed Revision and content bytes are unchanged; all public table values and effect entrypoints prove reads cause no writes | +| Search semantics | Same six-transition test | Search returns Resource metadata/references, not accounting balances. Resource publication state is distinct from Bounty payout state; dereferencing through bounty.get/store.listing_get yields the current accounting projection. Existing published schemas are unchanged | +| Pause/resume/top-up/close conservation | `test_market_72.py::test_pause_preserves_budget_topup_does_not_resume_and_close_returns_remainder` | Publisher pause survives top-up; pending claim denied while paused, succeeds after resume; close refunds exact remainder; balances sum to supply | +| Claim cap cannot reopen through top-up | `test_market_72.py::test_exhausted_claim_limit_is_consistent_in_both_views_and_topup_cannot_reopen` | Current views remain claims_exhausted with positive escrow; resume denied; claimant sees own claim and outsider sees none | +| Expiry does not write during GET | `test_market_72.py::test_expired_listing_projection_does_not_mutate_state_or_emit_events` | Both views report expired; stored state and Event count remain unchanged; six-transition test additionally checks exact deadline and every table value | +| Current key/revocation and subject binding | `test_bounty.py::test_bad_proof_wrong_subject_expiry_and_top_up_do_not_spend`; `test_market_contracts.py::test_revoked_or_replaced_identity_key_cannot_spend_pending_challenge` | Wrong proof key/foreign subject/expired nonce denied; old revoked key and replacement non-primary key denied; no payout or consumed nonce | +| Canonical nonce fields and verifier version | `test_market_72.py::test_stored_challenge_fields_must_match_canonical_signed_payload`; `test_bounty_commit_boundary.py::test_unsupported_challenge_verifier_cannot_consume_budget_or_nonce` | Tampered stored nonce or unsupported version cannot spend; full row values unchanged in the version case | +| Eligibility and per-subject limit | `test_bounty.py::test_bad_proof_wrong_subject_expiry_and_top_up_do_not_spend`; `test_bounty.py::test_unknown_eligibility_constraint_fails_closed`; `test_bounty_commit_boundary.py::test_subject_limit_rejects_fresh_challenge_while_budget_remains` | Allowlist excludes other subject; unknown constraint rejected before funding; fresh valid challenge denied at subject cap even with remaining budget | +| Last reward concurrency and close race | `test_bounty.py::test_concurrent_last_reward_only_one_paid_claim`; `test_market_contracts.py::test_claim_close_race_and_event_failure_never_lose_or_duplicate_budget` | Exactly one payout at last reward; close always succeeds and escrow ends zero; all remaining funds belong to publisher/claimant, no lost supply | +| Claim/ledger/receipt/Event/Inbox/results transaction | `test_bounty_commit_boundary.py::test_deferred_claim_commit_failure_preserves_all_rows_and_retry`; event-failure node above | Real deferred PostgreSQL trigger fails at COMMIT after handler preparation; every public row value rolls back, then same request succeeds exactly once and replay changes no rows; injected Event failure also rolls back payout/nonce/minimal notice | +| CLI proof retries and actual restart | `test_bounty_public_lifecycle.py::test_real_cli_prove_replays_after_restart_without_second_nonce_or_reward` | Real signer/Registry/executor/PostgreSQL through CLI; new Application returns identical payout/receipt; one challenge/claim, every row value unchanged on retry | +| Mixed mutable/immutable terms fail closed | All nodes in `test_bounty_snapshot_integrity.py` | Reward/cap/subject cap/eligibility mismatch rejects claim and reads; doctor does not repair; management cannot move ledger funds | +| Real backup and persistent restore quarantine | `test_market_72.py::test_backup_restores_claim_nonce_inbox_receipt_and_replay_together` | Real backup/empty PostgreSQL restore retains exact claim, nonce, result, minimal Inbox and ledger rows; saved-result replay stays writes_paused, including after marker deletion; private reads remain quarantined | +| Official mint/fund entrypoints | `test_market_71.py::test_bank_fund_is_a_formal_command_without_unattended_flags`; `test_market_71.py::test_bank_fund_two_console_approvals_before_pin_or_writes`; installed `market_e2e` | Real daemon parser and MoneyAdmin through isolated TestRoot/TestConsoleIO; mint20 and bank fund20 require explicit approvals. Refusal precedes PIN/writes; no unattended flag | +| Official 20→10→5 fixture | `msg.admin.market_check::check_market_e2e`, run by installed official selftest | Bank prefunds10; current buyer IdentityKey PoP earns10; fixed text `msg.lmm.best store selftest` and hello.txt `delivery-ok\n` cost5; V4 Delivery waits for buyer's signed exact digest, then buyer5/bank15/all escrow0/supply20 | +| Published V3 automatic settlement preserved | `test_market_lifecycle.py::test_instant_delivery_settlement_and_claim_are_distinct`; `test_market_cli_bindings.py::test_explicit_buy_version_preserves_old_inputs_and_settlement_semantics` | Original published policies remain automatic; settled does not manufacture claimed or accepted. Explicit old versions 1/2/3 retain signed input and behavior | +| Current default V4 explicit acceptance | All nodes in `test_market_explicit_acceptance.py` | CLI selects V4; current buyer and exact delivered digest required; wrong/revoked signatures cannot settle or replay; one protected posting; restart preserves delivered escrow | +| Disabled/failed/unverified mail does not gate trade | `test_market_integration.py::test_explicitly_disabled_mail_never_calls_sender`; `test_market_delivery.py::test_verified_checkout_mail_is_minimal_and_not_claimed`; `test_market_delivery.py::test_unverified_checkout_has_no_order_information_until_owner_verifies`; installed `market_e2e` | Disabled sender never called; failed sender leaves settled business fact intact; unverified receives only minimal verification; verified notice has order reference/name snapshot/authenticated pickup, no product secret; local sink only | +| Send-time endpoint/current authorization | `test_market_delivery.py::test_send_time_revalidation_prevents_stale_or_cross_subject_delivery` | Disabled, unverified and foreign-subject endpoint variants cannot leak order facts | +| Bank has no special business privileges | `test_market_redemption.py::test_clearing_policy_is_pure_and_has_no_bank_or_role_exceptions`; `test_market_71.py::test_bank_fund_role_and_payment_rollback_together`; `test_market_cli_bindings.py::test_package_shortcut_binds_actual_id_and_preserves_seller_privacy` | Pure policy without bank/role override; failed bank funding preserves role/payment; another subject cannot inspect private package | +| Default/doctor/bootstrap/selftest guard | `test_market_contracts.py::test_default_doctor_is_read_only_and_selftest_cannot_seed_an_installed_server`; `test_market_recovery.py::test_market_doctor_cannot_mutate_or_synthesize_a_repair`; installed selftest | Default publishes no fake budget; doctor retains all table values; production selftest refuses seeding an installed server; official selftest is isolated | + +## Version and field boundary + +The original issue's automatic-settlement wording belongs to published V3. +The authoritative current design and default V4 require explicit signed buyer +acceptance. Both are tested; reverting V4 to automatic settlement would alter +the current contract. Delivery, claimed, accepted and settled are separate facts. + +PoP proves possession of the current IdentityKey for one subject/challenge. +It does not prove a human, trustworthiness or resistance to Sybil identities. +The bank remains an ordinary Subject for catalog, verification and permissions. + +This code acceptance does not authorize production mint/funding, Root/PIN, +SMTP/Webhook delivery or deployment. Real target-host/old-snapshot/current-pin +and cutover evidence remain separately tracked in #64–70 and #84. diff --git a/tests/test_bounty_public_lifecycle.py b/tests/test_bounty_public_lifecycle.py new file mode 100644 index 00000000..153a02c7 --- /dev/null +++ b/tests/test_bounty_public_lifecycle.py @@ -0,0 +1,126 @@ +"""Public discovery locates immutable terms; market reads report live accounting.""" + +from datetime import timedelta +from types import SimpleNamespace + +import pytest +from read_only_evidence import business_snapshot, readonly_evidence +from test_market_72 import fixture, proof +from test_market_cli_bindings import ExecutorClient, command +from test_service import NOW, call + +from msg.application import Application +from msg.core.codec import canonical, wire + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'transition,state,reason,budget,balance,claims', + [ + ('underfunded', 'paused', 'out_of_budget', 5, 5, 0), + ('top_up', 'active', None, 20, 20, 0), + ('pause', 'paused', 'publisher_paused', 20, 20, 0), + ('exhausted', 'paused', 'claims_exhausted', 20, 10, 1), + ('expire', 'paused', 'expired', 20, 20, 0), + ('close', 'closed', None, 20, 0, 0), + ], +) +async def test_search_and_real_cli_preserve_terms_and_current_accounting( + installed, monkeypatch, transition, state, reason, budget, balance, claims +): + app, root = installed + changes = {'budget_minor': 5} if transition in {'underfunded', 'top_up'} else {} + if transition == 'expire': + changes['expires_at'] = wire(NOW + timedelta(seconds=1)) + (key, bank), identity, created = await fixture(app, root, **changes) + lid = created.data['bounty']['listing_id'] + ref = created.resources[0] + original = dict(created.data['bounty']) + async with app.metadata.transaction(write=False) as tx: + revision_before = wire(await tx.revision(ref)) + content_before = await app.contents.read_bytes((await tx.revision(ref)).content) + publisher = ExecutorClient(app, key, bank) + if transition == 'top_up': + result = await command( + publisher, + 'bounty', + 'top-up', + canonical({'listing_id': lid, 'amount_minor': 15}).decode(), + ) + assert result.status == 'ok', wire(result) + elif transition in {'pause', 'close'}: + result = await command(publisher, 'bounty', transition, lid) + assert result.status == 'ok', wire(result) + elif transition == 'exhausted': + claimant_key, claimant = identity + result = await call( + app, + 'bounty.claim', + await proof(app, identity, lid), + key=claimant_key, + subject=claimant, + ) + assert result.status == 'ok', wire(result) + elif transition == 'expire': + app.clock = lambda: NOW + timedelta(seconds=1) + await app.load() + + anonymous = ExecutorClient(app, None, None) + async with readonly_evidence(app, monkeypatch): + for operation, parameters in ( + ('discovery.search', {'query': 'pop', 'type': 'listing'}), + ('discovery.lexical_search', {'scope': '/store', 'terms': 'pop', 'field': 'name'}), + ): + found = await call(app, operation, parameters) + assert found.status == 'ok', wire(found) + assert [item['id'] for item in found.data['items']] == [lid] + # Resource.state describes publication, not Bounty accounting. + # Search returns a reference, never fabricated budget/award facts. + assert found.data['items'][0]['revision'] == ref.revision + assert 'budget_minor' not in found.data['items'][0] + bounty = await command(anonymous, 'bounty', 'get', lid) + catalog = await command(anonymous, 'store', 'get', lid) + assert bounty.status == catalog.status == 'ok', (wire(bounty), wire(catalog)) + for shown in (bounty.data['bounty'], catalog.data['listing']): + assert (shown['state'], shown['pause_reason']) == (state, reason) + assert shown['budget_minor'] == budget + assert shown['escrow_balance_minor'] == balance + assert shown['paid_claims'] == claims + history = await call(app, 'store.listing_get', {'id': lid, 'revision': ref.revision}) + assert history.status == 'ok', wire(history) + shown = history.data['listing'] + assert shown['state'] == original['state'] + assert shown['budget_minor'] == original['budget_minor'] + assert shown['current_state'] == state and shown['current_budget_minor'] == budget + assert shown['terms_revision'] == ref.revision + async with app.metadata.transaction(write=False) as tx: + revision = await tx.revision(ref) + assert wire(revision) == revision_before + assert await app.contents.read_bytes(revision.content) == content_before + + +@pytest.mark.asyncio +async def test_real_cli_prove_replays_after_restart_without_second_nonce_or_reward(installed): + app, root = installed + _, (key, claimant), created = await fixture(app, root) + lid = created.data['bounty']['listing_id'] + client = ExecutorClient(app, key, claimant) + client.state = SimpleNamespace(signer=key, subject=claimant) + argv = ('bounty', 'prove', lid, '--request-id', 'cli-pop-restart') + paid = await command(client, *argv) + assert paid.status == 'ok' and not paid.replayed, wire(paid) + before = await business_snapshot(app) + restarted = Application(app.settings, clock=lambda: NOW) + await restarted.load() + try: + client = ExecutorClient(restarted, key, claimant) + client.state = SimpleNamespace(signer=key, subject=claimant) + replay = await command(client, *argv) + assert replay.status == 'ok' and replay.replayed, wire(replay) + assert replay.data == paid.data and replay.receipt == paid.receipt + assert await business_snapshot(restarted) == before + async with restarted.metadata.transaction(write=False) as tx: + assert tx.one('SELECT COUNT(*) FROM bounty_challenges')[0] == 1 + assert tx.one('SELECT COUNT(*) FROM bounty_claims')[0] == 1 + finally: + await restarted.close() From 6b7622dd42e7014cbfb27cea95ce2f42d2ae6361 Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 15:55:37 +0800 Subject: [PATCH 02/11] test(oauth): expose source ceiling and userinfo write boundaries --- .github/workflows/integration-regressions.yml | 4 +- tests/test_oauth_authority.py | 69 +++++++++++++++++++ 2 files changed, 72 insertions(+), 1 deletion(-) create mode 100644 tests/test_oauth_authority.py diff --git a/.github/workflows/integration-regressions.yml b/.github/workflows/integration-regressions.yml index 5e6cb99a..7c55a8fa 100644 --- a/.github/workflows/integration-regressions.yml +++ b/.github/workflows/integration-regressions.yml @@ -29,6 +29,7 @@ on: - 'src/msg/storage/session.py' - 'src/msg/storage/read_only.py' - 'tests/test_oauth.py' + - 'tests/test_oauth_authority.py' - 'tests/test_http_effect_order.py' - 'tests/test_read_entry_contract_matrix.py' - 'tests/test_read_query_keyset_boundary.py' @@ -65,6 +66,7 @@ on: - 'src/msg/storage/session.py' - 'src/msg/storage/read_only.py' - 'tests/test_oauth.py' + - 'tests/test_oauth_authority.py' - 'tests/test_http_effect_order.py' - 'tests/test_read_entry_contract_matrix.py' - 'tests/test_read_query_keyset_boundary.py' @@ -182,7 +184,7 @@ jobs: - name: Run recovery, keyset, effect-order, path and webhook regressions run: >- python -m pytest -q - tests/test_oauth.py + tests/test_oauth.py tests/test_oauth_authority.py tests/test_http_effect_order.py tests/test_legacy_routes_removed.py tests/test_read_query_keyset_boundary.py tests/test_read_entry_contract_matrix.py tests/test_recovery_runtime_generation.py diff --git a/tests/test_oauth_authority.py b/tests/test_oauth_authority.py new file mode 100644 index 00000000..e055b9a0 --- /dev/null +++ b/tests/test_oauth_authority.py @@ -0,0 +1,69 @@ +"""Derived access must retain current source authority and read-only semantics.""" + +import os +from dataclasses import replace + +import pytest +from read_only_evidence import business_snapshot, readonly_evidence +from test_oauth import device_tokens, oauth as oauth +from test_service import call + +from msg.core.codec import b64, unb64, wire +from msg.core.models import Scope + + +@pytest.mark.asyncio +@pytest.mark.parametrize('kind', ['oauth', 'api']) +@pytest.mark.parametrize('contraction', ['operations', 'scope']) +async def test_derived_credentials_recheck_current_source_ceiling(oauth, kind, contraction): + app, key, subject, _http = oauth + if kind == 'oauth': + issued = await device_tokens(oauth) + credential_id, _, value = issued['access_token'].partition('.') + token = (credential_id, unb64(value)) + else: + issued = await call( + app, + 'identity.token_create', + {'nonce': b64(os.urandom(32)), 'ttl': 60, 'recovery_secret': b64(os.urandom(32))}, + key=key, + subject=subject, + contract_version=3, + ) + assert issued.status == 'ok', wire(issued) + token = (issued.data['credential_id'], unb64(issued.data['token'])) + allowed = await call(app, 'discovery.get', {'id': '/main'}, subject=subject, token=token) + assert allowed.status == 'ok', wire(allowed) + async with app.metadata.transaction(write=True) as tx: + parent = await tx.credential(key.key_id) + owner = await tx.subject(subject) + # Keep the signer, subject/auth_version, revocation and deadlines intact. + # Only the authoritative source ceiling changes, just as in #76 vectors. + ceiling = ( + () + if contraction == 'operations' + else tuple( + replace(grant, scope=Scope(resource_id='t_store', descendants=True)) + for grant in parent.ceiling + ) + ) + await tx.save_credential(replace(parent, ceiling=ceiling), owner.auth_version) + before = await business_snapshot(app) + denied = await call(app, 'discovery.get', {'id': '/main'}, subject=subject, token=token) + assert denied.status == 'error' and denied.error.code == 'invalid_grant', wire(denied) + assert await business_snapshot(app) == before + + +@pytest.mark.asyncio +@pytest.mark.parametrize('method', ['GET', 'POST']) +async def test_userinfo_is_read_only_including_authoritative_oauth_state(oauth, monkeypatch, method): + app, _key, subject, http = oauth + issued = await device_tokens(oauth) + async with readonly_evidence(app, monkeypatch): + response = await http.request( + method, + '/oauth/userinfo', + headers={'Authorization': 'Bearer ' + issued['access_token']}, + ) + assert response.status_code == 200, response.text + assert response.json() == {'sub': subject, 'preferred_username': 'oauth-owner'} From 33dde9b234cb3105f8b0edd46c3621554be5424e Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 16:07:17 +0800 Subject: [PATCH 03/11] Bound HTTP Range parsing before numeric conversion --- src/msg/extensions/hosting.py | 2 +- src/msg/transports/http_routes.py | 6 ++- tests/test_hosting_same_origin.py | 9 ++++ tests/test_http_range_bounds.py | 70 +++++++++++++++++++++++++++++++ 4 files changed, 85 insertions(+), 2 deletions(-) create mode 100644 tests/test_http_range_bounds.py diff --git a/src/msg/extensions/hosting.py b/src/msg/extensions/hosting.py index 602b31d8..368a79c2 100644 --- a/src/msg/extensions/hosting.py +++ b/src/msg/extensions/hosting.py @@ -330,7 +330,7 @@ def hosted_range(request, size, etag): requested = request.headers.get('range') if not requested or request.headers.get('if-range') not in {None, etag}: return (0, size), 200, {} - match = re.fullmatch(r'bytes=(\d*)-(\d*)', requested) + match = re.fullmatch(r'bytes=(\d*)-(\d*)', requested) if len(requested) <= 128 else None require(match is not None and any(match.groups()), 'range_not_satisfiable') left, right = match.groups() if left: diff --git a/src/msg/transports/http_routes.py b/src/msg/transports/http_routes.py index de81cb19..fee84e1f 100644 --- a/src/msg/transports/http_routes.py +++ b/src/msg/transports/http_routes.py @@ -2431,7 +2431,11 @@ async def dispatch(request: Request): status = 200 requested_range = request.headers.get('range') if requested_range and (request.headers.get('if-range') in {None, etag}): - match = re.fullmatch(r'bytes=(\d*)-(\d*)', requested_range) + match = ( + re.fullmatch(r'bytes=(\d*)-(\d*)', requested_range) + if len(requested_range) <= 128 + else None + ) require(match is not None and any(match.groups()), 'range_not_satisfiable') left, right = match.groups() if left: diff --git a/tests/test_hosting_same_origin.py b/tests/test_hosting_same_origin.py index da692d4b..f30e98ed 100644 --- a/tests/test_hosting_same_origin.py +++ b/tests/test_hosting_same_origin.py @@ -91,6 +91,15 @@ async def test_same_domain_hosted_html_head_304_range_raw_and_no_write_route(ins partial = await http.get('/@web-owner/web/index.html', headers={'Range': 'bytes=0-14'}) assert partial.status_code == 206 and partial.content == html[:15] isolated(partial) + for malformed in ( + 'bytes=' + '9' * 5000 + '-', + 'bytes=0-' + '9' * 5000, + 'bytes=-' + '9' * 5000, + ): + refused = await http.get('/@web-owner/web/index.html', headers={'Range': malformed}) + assert refused.status_code == 416 + assert refused.json()['error'] == 'range_not_satisfiable' + isolated(refused) raw = await http.get(f'/_read/{published}/raw') assert raw.status_code == 200 assert 'attachment' in raw.headers['content-disposition'] diff --git a/tests/test_http_range_bounds.py b/tests/test_http_range_bounds.py new file mode 100644 index 00000000..1f0329a7 --- /dev/null +++ b/tests/test_http_range_bounds.py @@ -0,0 +1,70 @@ +"""Malformed Range fields have bounded parsing and stable client errors.""" + +import pytest +from starlette.requests import Request + +from msg.core.errors import Failure +from msg.extensions.hosting import hosted_range + + +@pytest.mark.parametrize( + 'value', + ['bytes=' + '9' * 5000 + '-', 'bytes=0-' + '9' * 5000, 'bytes=-' + '9' * 5000], + ids=['start', 'end', 'suffix'], +) +def test_hosted_range_rejects_oversized_numbers_with_contract_error(value): + request = Request({'type': 'http', 'headers': [(b'range', value.encode())]}) + with pytest.raises(Failure, match='range_not_satisfiable'): + hosted_range(request, 12, '"etag"') + + +@pytest.mark.parametrize( + 'value,expected', + [('bytes=0-4', (0, 5)), ('bytes=5-', (5, 12)), ('bytes=-5', (7, 12)), ('bytes=-99', (0, 12))], +) +def test_hosted_range_keeps_normal_prefix_open_and_suffix_ranges(value, expected): + request = Request({'type': 'http', 'headers': [(b'range', value.encode())]}) + interval, status, _ = hosted_range(request, 12, '"etag"') + assert interval == expected and status == 206 + + +def test_stale_if_range_ignores_range_before_parsing(): + request = Request({ + 'type': 'http', + 'headers': [(b'range', b'bytes=' + b'9' * 5000 + b'-'), (b'if-range', b'"stale"')], + }) + assert hosted_range(request, 12, '"etag"') == ((0, 12), 200, {}) + + +@pytest.mark.asyncio +async def test_raw_http_rejects_oversized_range_without_read_effects(installed, monkeypatch): + import httpx + from read_only_evidence import readonly_evidence + from test_service import call, register + + from msg.transports.http import create_app + + app, _ = installed + key, subject, _ = await register(app, 'range-reader') + created = await call( + app, + 'content.post_create', + {'parent': '/main', 'body': 'range content'}, + key=key, + subject=subject, + ) + assert created.status == 'ok' + async with httpx.AsyncClient( + transport=httpx.ASGITransport(app=create_app(app)), base_url=app.settings.service_url + ) as http: + async with readonly_evidence(app, monkeypatch): + for value in ( + 'bytes=' + '9' * 5000 + '-', + 'bytes=0-' + '9' * 5000, + 'bytes=-' + '9' * 5000, + ): + result = await http.get( + '/_id/' + created.resources[0].id + '/raw', headers={'Range': value} + ) + assert result.status_code == 416 + assert result.json()['error']['code'] == 'range_not_satisfiable' From 16fc693b837fc983b19629dd7f534167de63e17e Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 16:08:44 +0800 Subject: [PATCH 04/11] fix(oauth): enforce live source ceilings and keep userinfo reads side-effect free --- src/msg/security/oauth.py | 31 ++++++++++++++++++++++++++----- src/msg/transports/oauth_http.py | 9 ++++++--- tests/test_oauth_authority.py | 5 ++++- 3 files changed, 36 insertions(+), 9 deletions(-) diff --git a/src/msg/security/oauth.py b/src/msg/security/oauth.py index 273d7cd7..b7121a5b 100644 --- a/src/msg/security/oauth.py +++ b/src/msg/security/oauth.py @@ -15,9 +15,10 @@ from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat -from msg.core.codec import b64, canonical, loads, parse_time, wire +from msg.core.codec import b64, canonical, decode, loads, parse_time, wire from msg.core.errors import Failure, require -from msg.core.models import Credential +from msg.core.models import CapabilityGrant, Credential +from msg.security.policy import constraints_subset, scope_subset from msg.security.quarantine import require_live_authority DEVICE_GRANT = 'urn:ietf:params:oauth:grant-type:device_code' @@ -67,6 +68,24 @@ def scopes_for(client, value): return scopes +async def require_ceiling(tx, ceiling, parent): + # A derived credential is a limit, never a durable copy of authority which + # its signing source has since lost. Reuse the same containment vocabulary + # as certificate/delegation validation, including scope and constraints. + for grant in ceiling: + require( + any([ + allowed.capability == grant.capability + and allowed.version == grant.version + and grant.operations <= allowed.operations + and await scope_subset(grant.scope, allowed.scope, tx) + and constraints_subset(grant.constraints, allowed.constraints) + for allowed in parent.ceiling + ]), + 'invalid_grant', + ) + + async def require_source(tx, body, now): parent = await tx.credential(body['parent']) subject = await tx.subject(body['subject']) @@ -79,6 +98,9 @@ async def require_source(tx, body, now): and (parent.expires_at is None or parent.expires_at > now), 'invalid_grant', ) + await require_ceiling( + tx, tuple(decode(CapabilityGrant, raw) for raw in body.get('ceiling', ())), parent + ) if body.get('custodial'): row = tx.one( 'SELECT signing_key_id,status FROM custodial_vault WHERE subject=?', @@ -98,6 +120,7 @@ async def require_binding(tx, credential, now, config): source = loads(api[0]) require(credential.source_credential_id == source['parent'], 'invalid_grant') await require_source(tx, source, now) + await require_ceiling(tx, credential.ceiling, await tx.credential(source['parent'])) row = tx.one('SELECT body FROM oauth_states WHERE id=?', ('access:' + credential.id,)) if row is None: require( @@ -112,6 +135,7 @@ async def require_binding(tx, credential, now, config): client = client_for(config, family['client_id']) require(set(family['scopes']) <= client.scopes and not family.get('revoked'), 'invalid_grant') await require_source(tx, family, now) + await require_ceiling(tx, credential.ceiling, await tx.credential(family['parent'])) class OAuthService: @@ -381,9 +405,6 @@ async def exchange(self, tx, args): return None, 'unsupported_grant_type' async def tokens(self, tx, source, *, family_id=None, expiry=None): - from msg.core.codec import decode - from msg.core.models import CapabilityGrant - now = self.app.clock() subject = await require_source(tx, source, now) if family_id is None: diff --git a/src/msg/transports/oauth_http.py b/src/msg/transports/oauth_http.py index 886e6469..6487252c 100644 --- a/src/msg/transports/oauth_http.py +++ b/src/msg/transports/oauth_http.py @@ -235,8 +235,11 @@ async def dispatch(self, request): ], }) # Rate limits commit independently, including rejected code guesses. - async with self.service.metadata.transaction(write=True) as tx: - self.oauth.rate(tx, request.client.host if request.client else '') + # Userinfo is a read: do not mutate retained OAuth state during GET or + # POST reads. Authentication still checks the current source/family. + if path != '/oauth/userinfo': + async with self.service.metadata.transaction(write=True) as tx: + self.oauth.rate(tx, request.client.host if request.client else '') if path == '/oauth/login': require(request.method == 'GET', 'method_not_allowed') async with self.service.metadata.transaction(write=True) as tx: @@ -346,7 +349,7 @@ async def dispatch(self, request): 'method_not_allowed', ) args = await form(request) if request.method == 'POST' else {} - async with self.service.metadata.transaction(write=True) as tx: + async with self.service.metadata.transaction(write=path != '/oauth/userinfo') as tx: self.oauth.fence(tx) if path == '/oauth/device_authorization': require(set(args) <= {'client_id', 'scope'}, 'invalid_request') diff --git a/tests/test_oauth_authority.py b/tests/test_oauth_authority.py index e055b9a0..251390e8 100644 --- a/tests/test_oauth_authority.py +++ b/tests/test_oauth_authority.py @@ -56,7 +56,9 @@ async def test_derived_credentials_recheck_current_source_ceiling(oauth, kind, c @pytest.mark.asyncio @pytest.mark.parametrize('method', ['GET', 'POST']) -async def test_userinfo_is_read_only_including_authoritative_oauth_state(oauth, monkeypatch, method): +async def test_userinfo_is_read_only_including_authoritative_oauth_state( + oauth, monkeypatch, method +): app, _key, subject, http = oauth issued = await device_tokens(oauth) async with readonly_evidence(app, monkeypatch): @@ -64,6 +66,7 @@ async def test_userinfo_is_read_only_including_authoritative_oauth_state(oauth, method, '/oauth/userinfo', headers={'Authorization': 'Bearer ' + issued['access_token']}, + **({'json': {}} if method == 'POST' else {}), ) assert response.status_code == 200, response.text assert response.json() == {'sub': subject, 'preferred_username': 'oauth-owner'} From f55374a23e698d8594ea10937bced7bdf90d5acc Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 16:38:26 +0800 Subject: [PATCH 05/11] Preserve custodial OAuth login under live policy ceilings --- src/msg/security/authentication.py | 12 ++- src/msg/security/oauth.py | 49 ++++++---- tests/test_oauth.py | 17 +++- tests/test_oauth_custodial_ceiling.py | 126 ++++++++++++++++++++++++++ 4 files changed, 184 insertions(+), 20 deletions(-) create mode 100644 tests/test_oauth_custodial_ceiling.py diff --git a/src/msg/security/authentication.py b/src/msg/security/authentication.py index df459a56..f5f69384 100644 --- a/src/msg/security/authentication.py +++ b/src/msg/security/authentication.py @@ -114,7 +114,9 @@ async def authenticate(self, request, session, *, entry): old = await session.credential(request.arguments['credential_id']) from msg.security.oauth import require_binding - await require_binding(session, old, now, self.oauth_config) + await require_binding( + session, old, now, self.oauth_config, custodial_ceiling=self.temporary_ceiling() + ) require( old.kind == 'token' and old.subject_id == request.subject, 'recovery_unavailable' ) @@ -193,7 +195,13 @@ async def authenticate(self, request, session, *, entry): ids = () from msg.security.oauth import require_binding - await require_binding(session, credential, now, self.oauth_config) + await require_binding( + session, + credential, + now, + self.oauth_config, + custodial_ceiling=self.temporary_ceiling(), + ) else: raise Failure('invalid_proof') if credential.revoked_at is not None: diff --git a/src/msg/security/oauth.py b/src/msg/security/oauth.py index b7121a5b..7286a686 100644 --- a/src/msg/security/oauth.py +++ b/src/msg/security/oauth.py @@ -80,13 +80,13 @@ async def require_ceiling(tx, ceiling, parent): and grant.operations <= allowed.operations and await scope_subset(grant.scope, allowed.scope, tx) and constraints_subset(grant.constraints, allowed.constraints) - for allowed in parent.ceiling + for allowed in parent ]), 'invalid_grant', ) -async def require_source(tx, body, now): +async def require_source(tx, body, now, *, custodial_ceiling): parent = await tx.credential(body['parent']) subject = await tx.subject(body['subject']) require( @@ -98,29 +98,36 @@ async def require_source(tx, body, now): and (parent.expires_at is None or parent.expires_at > now), 'invalid_grant', ) - await require_ceiling( - tx, tuple(decode(CapabilityGrant, raw) for raw in body.get('ceiling', ())), parent - ) if body.get('custodial'): row = tx.one( 'SELECT signing_key_id,status FROM custodial_vault WHERE subject=?', (subject.resource_id,), ) require(subject.kind == 'custodial' and row == (parent.id, 'active'), 'invalid_grant') + # Vault keys deliberately carry no request grants. Custodial sessions retain + # their captured token limits under the current temporary policy, while the + # vault key above remains only their live identity/revocation binding. + await require_ceiling( + tx, + tuple(decode(CapabilityGrant, raw) for raw in body.get('ceiling', ())), + custodial_ceiling if body.get('custodial') else parent.ceiling, + ) if body.get('session'): _, session = get(tx, body['session'], now) require(not session.get('revoked'), 'invalid_grant') - await require_source(tx, session, now) + await require_source(tx, session, now, custodial_ceiling=custodial_ceiling) return subject -async def require_binding(tx, credential, now, config): +async def require_binding(tx, credential, now, config, *, custodial_ceiling): api = tx.one('SELECT body FROM oauth_states WHERE id=?', ('api:' + credential.id,)) if api is not None: source = loads(api[0]) require(credential.source_credential_id == source['parent'], 'invalid_grant') - await require_source(tx, source, now) - await require_ceiling(tx, credential.ceiling, await tx.credential(source['parent'])) + await require_source(tx, source, now, custodial_ceiling=custodial_ceiling) + await require_ceiling( + tx, credential.ceiling, tuple(decode(CapabilityGrant, raw) for raw in source['ceiling']) + ) row = tx.one('SELECT body FROM oauth_states WHERE id=?', ('access:' + credential.id,)) if row is None: require( @@ -134,8 +141,10 @@ async def require_binding(tx, credential, now, config): require(credential.source_credential_id == family['parent'], 'invalid_grant') client = client_for(config, family['client_id']) require(set(family['scopes']) <= client.scopes and not family.get('revoked'), 'invalid_grant') - await require_source(tx, family, now) - await require_ceiling(tx, credential.ceiling, await tx.credential(family['parent'])) + await require_source(tx, family, now, custodial_ceiling=custodial_ceiling) + await require_ceiling( + tx, credential.ceiling, tuple(decode(CapabilityGrant, raw) for raw in family['ceiling']) + ) class OAuthService: @@ -265,7 +274,7 @@ async def pending_result(self, tx, value, client_id, kind): return None, 'authorization_pending' if body['status'] == 'denied': return None, 'access_denied' - await require_source(tx, body, now) + await require_source(tx, body, now, custodial_ceiling=self.app.temporary_ceiling()) body['status'] = 'consumed' save(tx, locator['user'], body) if kind == 'login': @@ -283,7 +292,9 @@ async def session(self, tx, cookie): id = state_id('session', cookie) expiry, body = get(tx, id, self.app.clock()) require(not body.get('revoked'), 'invalid_grant') - await require_source(tx, body, self.app.clock()) + await require_source( + tx, body, self.app.clock(), custodial_ceiling=self.app.temporary_ceiling() + ) source = { name: body[name] for name in ('subject', 'parent', 'auth_version', 'custodial', 'ceiling', 'auth_time') @@ -373,7 +384,7 @@ async def exchange(self, tx, args): ), 'invalid_grant', ) - await require_source(tx, body, now) + await require_source(tx, body, now, custodial_ceiling=self.app.temporary_ceiling()) body['consumed'] = True body['family'] = 'family:' + uuid4().hex save(tx, id, body) @@ -397,7 +408,7 @@ async def exchange(self, tx, args): require(not family.get('revoked'), 'invalid_grant') client = client_for(self.config, family['client_id']) require(set(family['scopes']) <= client.scopes, 'invalid_scope') - await require_source(tx, family, now) + await require_source(tx, family, now, custodial_ceiling=self.app.temporary_ceiling()) old['used'] = True save(tx, id, old) # A refresh does not extend the family's absolute lifetime. @@ -406,7 +417,9 @@ async def exchange(self, tx, args): async def tokens(self, tx, source, *, family_id=None, expiry=None): now = self.app.clock() - subject = await require_source(tx, source, now) + subject = await require_source( + tx, source, now, custodial_ceiling=self.app.temporary_ceiling() + ) if family_id is None: family_id = 'family:' + uuid4().hex expiry = now + timedelta(seconds=self.config.session_ttl) @@ -517,7 +530,9 @@ async def bearer(self, tx, value): 'invalid_token', ) try: - await require_binding(tx, credential, now, self.config) + await require_binding( + tx, credential, now, self.config, custodial_ceiling=self.app.temporary_ceiling() + ) except Failure as exc: if exc.code in {'recovery_runtime_stale', 'recovery_quarantined'}: raise diff --git a/tests/test_oauth.py b/tests/test_oauth.py index ecf441f8..29bf0965 100644 --- a/tests/test_oauth.py +++ b/tests/test_oauth.py @@ -364,7 +364,7 @@ async def test_api_key_private_signature_rotation_delivery_and_header(oauth, tmp @pytest.mark.asyncio -async def test_custodial_signup_login_and_device_consent_reuses_vault(oauth): +async def test_custodial_signup_login_and_device_consent_reuses_vault(oauth, monkeypatch): app, _, _, http = oauth await http.get('/oauth/signup') binder = http.cookies.get('msg_login') @@ -413,6 +413,21 @@ async def test_custodial_signup_login_and_device_consent_reuses_vault(oauth): assert response.status_code == 200, response.text async with app.metadata.transaction(write=False) as tx: assert tx.one('SELECT COUNT(*) FROM custodial_vault')[0] == 1 + key_id = tx.one('SELECT signing_key_id FROM custodial_vault')[0] + assert (await tx.credential(key_id)).ceiling == () + access = response.json()['access_token'] + assert ( + await http.get('/oauth/userinfo', headers={'Authorization': 'Bearer ' + access}) + ).status_code == 200 + # The vault never receives request grants to make login work. Current policy + # loss must invalidate already-issued custodial access without mutating reads. + from read_only_evidence import readonly_evidence + + monkeypatch.setattr(app, 'temporary_ceiling', lambda: ()) + monkeypatch.setattr(app.authenticator, 'temporary_ceiling', lambda: ()) + async with readonly_evidence(app, monkeypatch): + denied = await http.get('/oauth/userinfo', headers={'Authorization': 'Bearer ' + access}) + assert denied.status_code == 401 and denied.json()['error'] == 'invalid_token' @pytest.mark.parametrize( diff --git a/tests/test_oauth_custodial_ceiling.py b/tests/test_oauth_custodial_ceiling.py new file mode 100644 index 00000000..40d99bc5 --- /dev/null +++ b/tests/test_oauth_custodial_ceiling.py @@ -0,0 +1,126 @@ +"""A vault identity binding is not a request grant or an unlimited ceiling.""" + +from datetime import UTC, datetime +from types import SimpleNamespace + +import pytest + +from msg.core.codec import canonical, wire +from msg.core.errors import Failure +from msg.core.models import CapabilityGrant, Scope +from msg.security.oauth import require_binding, require_source + +NOW = datetime(2026, 9, 27, tzinfo=UTC) + + +def grant(*operations, descendants=True): + return CapabilityGrant( + capability='content', + version=1, + scope=Scope(resource_id='root', descendants=descendants), + operations=frozenset(operations), + constraints={}, + ) + + +class SourceSession: + def __init__(self, *, custodial=True): + self.parent = SimpleNamespace( + id='key', + subject_id='user', + revoked_at=None, + not_before=NOW, + expires_at=None, + ceiling=() if custodial else (grant('read'),), + ) + self.user = SimpleNamespace( + resource_id='user', + local_only=False, + auth_version=0, + kind='custodial' if custodial else 'registered', + ) + self.vault = ('key', 'active') + self.body = { + 'parent': 'key', + 'subject': 'user', + 'auth_version': 0, + 'custodial': custodial, + 'ceiling': wire((grant('read'),)), + } + self.api = False + + async def credential(self, _id): + return self.parent + + async def subject(self, _id): + return self.user + + def one(self, query, args): + if 'custodial_vault' in query: + return self.vault + if self.api and args[0] == 'api:t_api_test': + return (canonical(self.body).decode(),) + return None + + +@pytest.mark.asyncio +async def test_active_vault_with_empty_request_ceiling_preserves_captured_policy(): + tx = SourceSession() + assert await require_source(tx, tx.body, NOW, custodial_ceiling=(grant('read'),)) is tx.user + assert tx.parent.ceiling == () + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'fault', ['empty_policy', 'narrow_scope', 'revoked', 'inactive', 'foreign_vault'] +) +async def test_custodial_source_still_fails_closed_on_current_authority_loss(fault): + tx = SourceSession() + policy = (grant('read'),) + if fault == 'empty_policy': + policy = () + elif fault == 'narrow_scope': + policy = (grant('read', descendants=False),) + elif fault == 'revoked': + tx.parent.revoked_at = NOW + elif fault == 'inactive': + tx.vault = ('key', 'destroyed') + else: + tx.vault = ('other-key', 'active') + with pytest.raises(Failure, match='invalid_grant'): + await require_source(tx, tx.body, NOW, custodial_ceiling=policy) + + +@pytest.mark.asyncio +async def test_empty_registered_key_ceiling_never_uses_custodial_policy(): + tx = SourceSession(custodial=False) + tx.parent.ceiling = () + with pytest.raises(Failure, match='invalid_grant'): + await require_source(tx, tx.body, NOW, custodial_ceiling=(grant('read'),)) + + +@pytest.mark.asyncio +async def test_registered_source_remains_independent_of_custodial_policy(): + tx = SourceSession(custodial=False) + assert await require_source(tx, tx.body, NOW, custodial_ceiling=()) is tx.user + + +@pytest.mark.asyncio +@pytest.mark.parametrize('expanded', [False, True]) +async def test_derived_credential_cannot_exceed_captured_custodial_grants(expanded): + tx = SourceSession() + tx.api = True + credential = SimpleNamespace( + id='t_api_test', + source_credential_id='key', + ceiling=(grant('read', 'write') if expanded else grant('read'),), + ) + if expanded: + with pytest.raises(Failure, match='invalid_grant'): + await require_binding( + tx, credential, NOW, None, custodial_ceiling=(grant('read', 'write'),) + ) + else: + await require_binding( + tx, credential, NOW, None, custodial_ceiling=(grant('read', 'write'),) + ) From c92135bb96df5e46b02f8fc3246f2f2f6b5c0f5f Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 16:49:52 +0800 Subject: [PATCH 06/11] Preserve API binding state shape in source ceiling validation --- src/msg/security/oauth.py | 4 ++- tests/test_oauth_custodial_ceiling.py | 50 +++++++++++++++++++++++---- 2 files changed, 46 insertions(+), 8 deletions(-) diff --git a/src/msg/security/oauth.py b/src/msg/security/oauth.py index 7286a686..383943b5 100644 --- a/src/msg/security/oauth.py +++ b/src/msg/security/oauth.py @@ -125,8 +125,10 @@ async def require_binding(tx, credential, now, config, *, custodial_ceiling): source = loads(api[0]) require(credential.source_credential_id == source['parent'], 'invalid_grant') await require_source(tx, source, now, custodial_ceiling=custodial_ceiling) + # API bindings predate OAuth families and store no captured ceiling. + # Their credential grants remain bounded by the current signing source. await require_ceiling( - tx, credential.ceiling, tuple(decode(CapabilityGrant, raw) for raw in source['ceiling']) + tx, credential.ceiling, (await tx.credential(source['parent'])).ceiling ) row = tx.one('SELECT body FROM oauth_states WHERE id=?', ('access:' + credential.id,)) if row is None: diff --git a/tests/test_oauth_custodial_ceiling.py b/tests/test_oauth_custodial_ceiling.py index 40d99bc5..005ea1a9 100644 --- a/tests/test_oauth_custodial_ceiling.py +++ b/tests/test_oauth_custodial_ceiling.py @@ -1,6 +1,6 @@ """A vault identity binding is not a request grant or an unlimited ceiling.""" -from datetime import UTC, datetime +from datetime import UTC, datetime, timedelta from types import SimpleNamespace import pytest @@ -8,6 +8,7 @@ from msg.core.codec import canonical, wire from msg.core.errors import Failure from msg.core.models import CapabilityGrant, Scope +from msg.oauth_config import OAuthConfig from msg.security.oauth import require_binding, require_source NOW = datetime(2026, 9, 27, tzinfo=UTC) @@ -47,7 +48,7 @@ def __init__(self, *, custodial=True): 'custodial': custodial, 'ceiling': wire((grant('read'),)), } - self.api = False + self.api = self.oauth = False async def credential(self, _id): return self.parent @@ -59,7 +60,18 @@ def one(self, query, args): if 'custodial_vault' in query: return self.vault if self.api and args[0] == 'api:t_api_test': - return (canonical(self.body).decode(),) + return ( + canonical({ + name: self.body[name] for name in ('subject', 'parent', 'auth_version') + }).decode(), + ) + if self.oauth and args[0] == 'access:t_oauth_test': + return (canonical({'family': 'family:test'}).decode(),) + if self.oauth and args[0] == 'family:test': + return ( + wire(NOW + timedelta(hours=1)), + canonical({**self.body, 'client_id': 'msg-cli', 'scopes': ['msg.read']}).decode(), + ) return None @@ -109,18 +121,42 @@ async def test_registered_source_remains_independent_of_custodial_policy(): @pytest.mark.parametrize('expanded', [False, True]) async def test_derived_credential_cannot_exceed_captured_custodial_grants(expanded): tx = SourceSession() - tx.api = True + tx.oauth = True credential = SimpleNamespace( - id='t_api_test', + id='t_oauth_test', source_credential_id='key', ceiling=(grant('read', 'write') if expanded else grant('read'),), ) if expanded: with pytest.raises(Failure, match='invalid_grant'): await require_binding( - tx, credential, NOW, None, custodial_ceiling=(grant('read', 'write'),) + tx, + credential, + NOW, + OAuthConfig(enabled=True), + custodial_ceiling=(grant('read', 'write'),), ) else: await require_binding( - tx, credential, NOW, None, custodial_ceiling=(grant('read', 'write'),) + tx, + credential, + NOW, + OAuthConfig(enabled=True), + custodial_ceiling=(grant('read', 'write'),), ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize('contracted', [False, True]) +async def test_api_binding_without_captured_ceiling_uses_current_signing_key(contracted): + tx = SourceSession(custodial=False) + tx.api = True + credential = SimpleNamespace( + id='t_api_test', source_credential_id='key', ceiling=(grant('read'),) + ) + if contracted: + tx.parent.ceiling = () + with pytest.raises(Failure, match='invalid_grant'): + await require_binding(tx, credential, NOW, None, custodial_ceiling=()) + else: + await require_binding(tx, credential, NOW, None, custodial_ceiling=()) From 448834373d1a7e2ae9e22cce0b111bb8a9538ca8 Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 18:14:31 +0800 Subject: [PATCH 07/11] Integrate validated OAuth authority and bounded Range fixes --- .github/workflows/integration-regressions.yml | 4 +- src/msg/extensions/hosting.py | 2 +- src/msg/security/authentication.py | 12 +- src/msg/security/oauth.py | 70 ++++++-- src/msg/transports/http_routes.py | 6 +- src/msg/transports/oauth_http.py | 9 +- tests/test_hosting_same_origin.py | 9 + tests/test_http_range_bounds.py | 70 ++++++++ tests/test_oauth.py | 17 +- tests/test_oauth_authority.py | 72 ++++++++ tests/test_oauth_custodial_ceiling.py | 162 ++++++++++++++++++ 11 files changed, 408 insertions(+), 25 deletions(-) create mode 100644 tests/test_http_range_bounds.py create mode 100644 tests/test_oauth_authority.py create mode 100644 tests/test_oauth_custodial_ceiling.py diff --git a/.github/workflows/integration-regressions.yml b/.github/workflows/integration-regressions.yml index 5e6cb99a..7c55a8fa 100644 --- a/.github/workflows/integration-regressions.yml +++ b/.github/workflows/integration-regressions.yml @@ -29,6 +29,7 @@ on: - 'src/msg/storage/session.py' - 'src/msg/storage/read_only.py' - 'tests/test_oauth.py' + - 'tests/test_oauth_authority.py' - 'tests/test_http_effect_order.py' - 'tests/test_read_entry_contract_matrix.py' - 'tests/test_read_query_keyset_boundary.py' @@ -65,6 +66,7 @@ on: - 'src/msg/storage/session.py' - 'src/msg/storage/read_only.py' - 'tests/test_oauth.py' + - 'tests/test_oauth_authority.py' - 'tests/test_http_effect_order.py' - 'tests/test_read_entry_contract_matrix.py' - 'tests/test_read_query_keyset_boundary.py' @@ -182,7 +184,7 @@ jobs: - name: Run recovery, keyset, effect-order, path and webhook regressions run: >- python -m pytest -q - tests/test_oauth.py + tests/test_oauth.py tests/test_oauth_authority.py tests/test_http_effect_order.py tests/test_legacy_routes_removed.py tests/test_read_query_keyset_boundary.py tests/test_read_entry_contract_matrix.py tests/test_recovery_runtime_generation.py diff --git a/src/msg/extensions/hosting.py b/src/msg/extensions/hosting.py index 602b31d8..368a79c2 100644 --- a/src/msg/extensions/hosting.py +++ b/src/msg/extensions/hosting.py @@ -330,7 +330,7 @@ def hosted_range(request, size, etag): requested = request.headers.get('range') if not requested or request.headers.get('if-range') not in {None, etag}: return (0, size), 200, {} - match = re.fullmatch(r'bytes=(\d*)-(\d*)', requested) + match = re.fullmatch(r'bytes=(\d*)-(\d*)', requested) if len(requested) <= 128 else None require(match is not None and any(match.groups()), 'range_not_satisfiable') left, right = match.groups() if left: diff --git a/src/msg/security/authentication.py b/src/msg/security/authentication.py index df459a56..f5f69384 100644 --- a/src/msg/security/authentication.py +++ b/src/msg/security/authentication.py @@ -114,7 +114,9 @@ async def authenticate(self, request, session, *, entry): old = await session.credential(request.arguments['credential_id']) from msg.security.oauth import require_binding - await require_binding(session, old, now, self.oauth_config) + await require_binding( + session, old, now, self.oauth_config, custodial_ceiling=self.temporary_ceiling() + ) require( old.kind == 'token' and old.subject_id == request.subject, 'recovery_unavailable' ) @@ -193,7 +195,13 @@ async def authenticate(self, request, session, *, entry): ids = () from msg.security.oauth import require_binding - await require_binding(session, credential, now, self.oauth_config) + await require_binding( + session, + credential, + now, + self.oauth_config, + custodial_ceiling=self.temporary_ceiling(), + ) else: raise Failure('invalid_proof') if credential.revoked_at is not None: diff --git a/src/msg/security/oauth.py b/src/msg/security/oauth.py index 273d7cd7..383943b5 100644 --- a/src/msg/security/oauth.py +++ b/src/msg/security/oauth.py @@ -15,9 +15,10 @@ from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat -from msg.core.codec import b64, canonical, loads, parse_time, wire +from msg.core.codec import b64, canonical, decode, loads, parse_time, wire from msg.core.errors import Failure, require -from msg.core.models import Credential +from msg.core.models import CapabilityGrant, Credential +from msg.security.policy import constraints_subset, scope_subset from msg.security.quarantine import require_live_authority DEVICE_GRANT = 'urn:ietf:params:oauth:grant-type:device_code' @@ -67,7 +68,25 @@ def scopes_for(client, value): return scopes -async def require_source(tx, body, now): +async def require_ceiling(tx, ceiling, parent): + # A derived credential is a limit, never a durable copy of authority which + # its signing source has since lost. Reuse the same containment vocabulary + # as certificate/delegation validation, including scope and constraints. + for grant in ceiling: + require( + any([ + allowed.capability == grant.capability + and allowed.version == grant.version + and grant.operations <= allowed.operations + and await scope_subset(grant.scope, allowed.scope, tx) + and constraints_subset(grant.constraints, allowed.constraints) + for allowed in parent + ]), + 'invalid_grant', + ) + + +async def require_source(tx, body, now, *, custodial_ceiling): parent = await tx.credential(body['parent']) subject = await tx.subject(body['subject']) require( @@ -85,19 +104,32 @@ async def require_source(tx, body, now): (subject.resource_id,), ) require(subject.kind == 'custodial' and row == (parent.id, 'active'), 'invalid_grant') + # Vault keys deliberately carry no request grants. Custodial sessions retain + # their captured token limits under the current temporary policy, while the + # vault key above remains only their live identity/revocation binding. + await require_ceiling( + tx, + tuple(decode(CapabilityGrant, raw) for raw in body.get('ceiling', ())), + custodial_ceiling if body.get('custodial') else parent.ceiling, + ) if body.get('session'): _, session = get(tx, body['session'], now) require(not session.get('revoked'), 'invalid_grant') - await require_source(tx, session, now) + await require_source(tx, session, now, custodial_ceiling=custodial_ceiling) return subject -async def require_binding(tx, credential, now, config): +async def require_binding(tx, credential, now, config, *, custodial_ceiling): api = tx.one('SELECT body FROM oauth_states WHERE id=?', ('api:' + credential.id,)) if api is not None: source = loads(api[0]) require(credential.source_credential_id == source['parent'], 'invalid_grant') - await require_source(tx, source, now) + await require_source(tx, source, now, custodial_ceiling=custodial_ceiling) + # API bindings predate OAuth families and store no captured ceiling. + # Their credential grants remain bounded by the current signing source. + await require_ceiling( + tx, credential.ceiling, (await tx.credential(source['parent'])).ceiling + ) row = tx.one('SELECT body FROM oauth_states WHERE id=?', ('access:' + credential.id,)) if row is None: require( @@ -111,7 +143,10 @@ async def require_binding(tx, credential, now, config): require(credential.source_credential_id == family['parent'], 'invalid_grant') client = client_for(config, family['client_id']) require(set(family['scopes']) <= client.scopes and not family.get('revoked'), 'invalid_grant') - await require_source(tx, family, now) + await require_source(tx, family, now, custodial_ceiling=custodial_ceiling) + await require_ceiling( + tx, credential.ceiling, tuple(decode(CapabilityGrant, raw) for raw in family['ceiling']) + ) class OAuthService: @@ -241,7 +276,7 @@ async def pending_result(self, tx, value, client_id, kind): return None, 'authorization_pending' if body['status'] == 'denied': return None, 'access_denied' - await require_source(tx, body, now) + await require_source(tx, body, now, custodial_ceiling=self.app.temporary_ceiling()) body['status'] = 'consumed' save(tx, locator['user'], body) if kind == 'login': @@ -259,7 +294,9 @@ async def session(self, tx, cookie): id = state_id('session', cookie) expiry, body = get(tx, id, self.app.clock()) require(not body.get('revoked'), 'invalid_grant') - await require_source(tx, body, self.app.clock()) + await require_source( + tx, body, self.app.clock(), custodial_ceiling=self.app.temporary_ceiling() + ) source = { name: body[name] for name in ('subject', 'parent', 'auth_version', 'custodial', 'ceiling', 'auth_time') @@ -349,7 +386,7 @@ async def exchange(self, tx, args): ), 'invalid_grant', ) - await require_source(tx, body, now) + await require_source(tx, body, now, custodial_ceiling=self.app.temporary_ceiling()) body['consumed'] = True body['family'] = 'family:' + uuid4().hex save(tx, id, body) @@ -373,7 +410,7 @@ async def exchange(self, tx, args): require(not family.get('revoked'), 'invalid_grant') client = client_for(self.config, family['client_id']) require(set(family['scopes']) <= client.scopes, 'invalid_scope') - await require_source(tx, family, now) + await require_source(tx, family, now, custodial_ceiling=self.app.temporary_ceiling()) old['used'] = True save(tx, id, old) # A refresh does not extend the family's absolute lifetime. @@ -381,11 +418,10 @@ async def exchange(self, tx, args): return None, 'unsupported_grant_type' async def tokens(self, tx, source, *, family_id=None, expiry=None): - from msg.core.codec import decode - from msg.core.models import CapabilityGrant - now = self.app.clock() - subject = await require_source(tx, source, now) + subject = await require_source( + tx, source, now, custodial_ceiling=self.app.temporary_ceiling() + ) if family_id is None: family_id = 'family:' + uuid4().hex expiry = now + timedelta(seconds=self.config.session_ttl) @@ -496,7 +532,9 @@ async def bearer(self, tx, value): 'invalid_token', ) try: - await require_binding(tx, credential, now, self.config) + await require_binding( + tx, credential, now, self.config, custodial_ceiling=self.app.temporary_ceiling() + ) except Failure as exc: if exc.code in {'recovery_runtime_stale', 'recovery_quarantined'}: raise diff --git a/src/msg/transports/http_routes.py b/src/msg/transports/http_routes.py index de81cb19..fee84e1f 100644 --- a/src/msg/transports/http_routes.py +++ b/src/msg/transports/http_routes.py @@ -2431,7 +2431,11 @@ async def dispatch(request: Request): status = 200 requested_range = request.headers.get('range') if requested_range and (request.headers.get('if-range') in {None, etag}): - match = re.fullmatch(r'bytes=(\d*)-(\d*)', requested_range) + match = ( + re.fullmatch(r'bytes=(\d*)-(\d*)', requested_range) + if len(requested_range) <= 128 + else None + ) require(match is not None and any(match.groups()), 'range_not_satisfiable') left, right = match.groups() if left: diff --git a/src/msg/transports/oauth_http.py b/src/msg/transports/oauth_http.py index 886e6469..6487252c 100644 --- a/src/msg/transports/oauth_http.py +++ b/src/msg/transports/oauth_http.py @@ -235,8 +235,11 @@ async def dispatch(self, request): ], }) # Rate limits commit independently, including rejected code guesses. - async with self.service.metadata.transaction(write=True) as tx: - self.oauth.rate(tx, request.client.host if request.client else '') + # Userinfo is a read: do not mutate retained OAuth state during GET or + # POST reads. Authentication still checks the current source/family. + if path != '/oauth/userinfo': + async with self.service.metadata.transaction(write=True) as tx: + self.oauth.rate(tx, request.client.host if request.client else '') if path == '/oauth/login': require(request.method == 'GET', 'method_not_allowed') async with self.service.metadata.transaction(write=True) as tx: @@ -346,7 +349,7 @@ async def dispatch(self, request): 'method_not_allowed', ) args = await form(request) if request.method == 'POST' else {} - async with self.service.metadata.transaction(write=True) as tx: + async with self.service.metadata.transaction(write=path != '/oauth/userinfo') as tx: self.oauth.fence(tx) if path == '/oauth/device_authorization': require(set(args) <= {'client_id', 'scope'}, 'invalid_request') diff --git a/tests/test_hosting_same_origin.py b/tests/test_hosting_same_origin.py index da692d4b..f30e98ed 100644 --- a/tests/test_hosting_same_origin.py +++ b/tests/test_hosting_same_origin.py @@ -91,6 +91,15 @@ async def test_same_domain_hosted_html_head_304_range_raw_and_no_write_route(ins partial = await http.get('/@web-owner/web/index.html', headers={'Range': 'bytes=0-14'}) assert partial.status_code == 206 and partial.content == html[:15] isolated(partial) + for malformed in ( + 'bytes=' + '9' * 5000 + '-', + 'bytes=0-' + '9' * 5000, + 'bytes=-' + '9' * 5000, + ): + refused = await http.get('/@web-owner/web/index.html', headers={'Range': malformed}) + assert refused.status_code == 416 + assert refused.json()['error'] == 'range_not_satisfiable' + isolated(refused) raw = await http.get(f'/_read/{published}/raw') assert raw.status_code == 200 assert 'attachment' in raw.headers['content-disposition'] diff --git a/tests/test_http_range_bounds.py b/tests/test_http_range_bounds.py new file mode 100644 index 00000000..1f0329a7 --- /dev/null +++ b/tests/test_http_range_bounds.py @@ -0,0 +1,70 @@ +"""Malformed Range fields have bounded parsing and stable client errors.""" + +import pytest +from starlette.requests import Request + +from msg.core.errors import Failure +from msg.extensions.hosting import hosted_range + + +@pytest.mark.parametrize( + 'value', + ['bytes=' + '9' * 5000 + '-', 'bytes=0-' + '9' * 5000, 'bytes=-' + '9' * 5000], + ids=['start', 'end', 'suffix'], +) +def test_hosted_range_rejects_oversized_numbers_with_contract_error(value): + request = Request({'type': 'http', 'headers': [(b'range', value.encode())]}) + with pytest.raises(Failure, match='range_not_satisfiable'): + hosted_range(request, 12, '"etag"') + + +@pytest.mark.parametrize( + 'value,expected', + [('bytes=0-4', (0, 5)), ('bytes=5-', (5, 12)), ('bytes=-5', (7, 12)), ('bytes=-99', (0, 12))], +) +def test_hosted_range_keeps_normal_prefix_open_and_suffix_ranges(value, expected): + request = Request({'type': 'http', 'headers': [(b'range', value.encode())]}) + interval, status, _ = hosted_range(request, 12, '"etag"') + assert interval == expected and status == 206 + + +def test_stale_if_range_ignores_range_before_parsing(): + request = Request({ + 'type': 'http', + 'headers': [(b'range', b'bytes=' + b'9' * 5000 + b'-'), (b'if-range', b'"stale"')], + }) + assert hosted_range(request, 12, '"etag"') == ((0, 12), 200, {}) + + +@pytest.mark.asyncio +async def test_raw_http_rejects_oversized_range_without_read_effects(installed, monkeypatch): + import httpx + from read_only_evidence import readonly_evidence + from test_service import call, register + + from msg.transports.http import create_app + + app, _ = installed + key, subject, _ = await register(app, 'range-reader') + created = await call( + app, + 'content.post_create', + {'parent': '/main', 'body': 'range content'}, + key=key, + subject=subject, + ) + assert created.status == 'ok' + async with httpx.AsyncClient( + transport=httpx.ASGITransport(app=create_app(app)), base_url=app.settings.service_url + ) as http: + async with readonly_evidence(app, monkeypatch): + for value in ( + 'bytes=' + '9' * 5000 + '-', + 'bytes=0-' + '9' * 5000, + 'bytes=-' + '9' * 5000, + ): + result = await http.get( + '/_id/' + created.resources[0].id + '/raw', headers={'Range': value} + ) + assert result.status_code == 416 + assert result.json()['error']['code'] == 'range_not_satisfiable' diff --git a/tests/test_oauth.py b/tests/test_oauth.py index ecf441f8..29bf0965 100644 --- a/tests/test_oauth.py +++ b/tests/test_oauth.py @@ -364,7 +364,7 @@ async def test_api_key_private_signature_rotation_delivery_and_header(oauth, tmp @pytest.mark.asyncio -async def test_custodial_signup_login_and_device_consent_reuses_vault(oauth): +async def test_custodial_signup_login_and_device_consent_reuses_vault(oauth, monkeypatch): app, _, _, http = oauth await http.get('/oauth/signup') binder = http.cookies.get('msg_login') @@ -413,6 +413,21 @@ async def test_custodial_signup_login_and_device_consent_reuses_vault(oauth): assert response.status_code == 200, response.text async with app.metadata.transaction(write=False) as tx: assert tx.one('SELECT COUNT(*) FROM custodial_vault')[0] == 1 + key_id = tx.one('SELECT signing_key_id FROM custodial_vault')[0] + assert (await tx.credential(key_id)).ceiling == () + access = response.json()['access_token'] + assert ( + await http.get('/oauth/userinfo', headers={'Authorization': 'Bearer ' + access}) + ).status_code == 200 + # The vault never receives request grants to make login work. Current policy + # loss must invalidate already-issued custodial access without mutating reads. + from read_only_evidence import readonly_evidence + + monkeypatch.setattr(app, 'temporary_ceiling', lambda: ()) + monkeypatch.setattr(app.authenticator, 'temporary_ceiling', lambda: ()) + async with readonly_evidence(app, monkeypatch): + denied = await http.get('/oauth/userinfo', headers={'Authorization': 'Bearer ' + access}) + assert denied.status_code == 401 and denied.json()['error'] == 'invalid_token' @pytest.mark.parametrize( diff --git a/tests/test_oauth_authority.py b/tests/test_oauth_authority.py new file mode 100644 index 00000000..251390e8 --- /dev/null +++ b/tests/test_oauth_authority.py @@ -0,0 +1,72 @@ +"""Derived access must retain current source authority and read-only semantics.""" + +import os +from dataclasses import replace + +import pytest +from read_only_evidence import business_snapshot, readonly_evidence +from test_oauth import device_tokens, oauth as oauth +from test_service import call + +from msg.core.codec import b64, unb64, wire +from msg.core.models import Scope + + +@pytest.mark.asyncio +@pytest.mark.parametrize('kind', ['oauth', 'api']) +@pytest.mark.parametrize('contraction', ['operations', 'scope']) +async def test_derived_credentials_recheck_current_source_ceiling(oauth, kind, contraction): + app, key, subject, _http = oauth + if kind == 'oauth': + issued = await device_tokens(oauth) + credential_id, _, value = issued['access_token'].partition('.') + token = (credential_id, unb64(value)) + else: + issued = await call( + app, + 'identity.token_create', + {'nonce': b64(os.urandom(32)), 'ttl': 60, 'recovery_secret': b64(os.urandom(32))}, + key=key, + subject=subject, + contract_version=3, + ) + assert issued.status == 'ok', wire(issued) + token = (issued.data['credential_id'], unb64(issued.data['token'])) + allowed = await call(app, 'discovery.get', {'id': '/main'}, subject=subject, token=token) + assert allowed.status == 'ok', wire(allowed) + async with app.metadata.transaction(write=True) as tx: + parent = await tx.credential(key.key_id) + owner = await tx.subject(subject) + # Keep the signer, subject/auth_version, revocation and deadlines intact. + # Only the authoritative source ceiling changes, just as in #76 vectors. + ceiling = ( + () + if contraction == 'operations' + else tuple( + replace(grant, scope=Scope(resource_id='t_store', descendants=True)) + for grant in parent.ceiling + ) + ) + await tx.save_credential(replace(parent, ceiling=ceiling), owner.auth_version) + before = await business_snapshot(app) + denied = await call(app, 'discovery.get', {'id': '/main'}, subject=subject, token=token) + assert denied.status == 'error' and denied.error.code == 'invalid_grant', wire(denied) + assert await business_snapshot(app) == before + + +@pytest.mark.asyncio +@pytest.mark.parametrize('method', ['GET', 'POST']) +async def test_userinfo_is_read_only_including_authoritative_oauth_state( + oauth, monkeypatch, method +): + app, _key, subject, http = oauth + issued = await device_tokens(oauth) + async with readonly_evidence(app, monkeypatch): + response = await http.request( + method, + '/oauth/userinfo', + headers={'Authorization': 'Bearer ' + issued['access_token']}, + **({'json': {}} if method == 'POST' else {}), + ) + assert response.status_code == 200, response.text + assert response.json() == {'sub': subject, 'preferred_username': 'oauth-owner'} diff --git a/tests/test_oauth_custodial_ceiling.py b/tests/test_oauth_custodial_ceiling.py new file mode 100644 index 00000000..005ea1a9 --- /dev/null +++ b/tests/test_oauth_custodial_ceiling.py @@ -0,0 +1,162 @@ +"""A vault identity binding is not a request grant or an unlimited ceiling.""" + +from datetime import UTC, datetime, timedelta +from types import SimpleNamespace + +import pytest + +from msg.core.codec import canonical, wire +from msg.core.errors import Failure +from msg.core.models import CapabilityGrant, Scope +from msg.oauth_config import OAuthConfig +from msg.security.oauth import require_binding, require_source + +NOW = datetime(2026, 9, 27, tzinfo=UTC) + + +def grant(*operations, descendants=True): + return CapabilityGrant( + capability='content', + version=1, + scope=Scope(resource_id='root', descendants=descendants), + operations=frozenset(operations), + constraints={}, + ) + + +class SourceSession: + def __init__(self, *, custodial=True): + self.parent = SimpleNamespace( + id='key', + subject_id='user', + revoked_at=None, + not_before=NOW, + expires_at=None, + ceiling=() if custodial else (grant('read'),), + ) + self.user = SimpleNamespace( + resource_id='user', + local_only=False, + auth_version=0, + kind='custodial' if custodial else 'registered', + ) + self.vault = ('key', 'active') + self.body = { + 'parent': 'key', + 'subject': 'user', + 'auth_version': 0, + 'custodial': custodial, + 'ceiling': wire((grant('read'),)), + } + self.api = self.oauth = False + + async def credential(self, _id): + return self.parent + + async def subject(self, _id): + return self.user + + def one(self, query, args): + if 'custodial_vault' in query: + return self.vault + if self.api and args[0] == 'api:t_api_test': + return ( + canonical({ + name: self.body[name] for name in ('subject', 'parent', 'auth_version') + }).decode(), + ) + if self.oauth and args[0] == 'access:t_oauth_test': + return (canonical({'family': 'family:test'}).decode(),) + if self.oauth and args[0] == 'family:test': + return ( + wire(NOW + timedelta(hours=1)), + canonical({**self.body, 'client_id': 'msg-cli', 'scopes': ['msg.read']}).decode(), + ) + return None + + +@pytest.mark.asyncio +async def test_active_vault_with_empty_request_ceiling_preserves_captured_policy(): + tx = SourceSession() + assert await require_source(tx, tx.body, NOW, custodial_ceiling=(grant('read'),)) is tx.user + assert tx.parent.ceiling == () + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'fault', ['empty_policy', 'narrow_scope', 'revoked', 'inactive', 'foreign_vault'] +) +async def test_custodial_source_still_fails_closed_on_current_authority_loss(fault): + tx = SourceSession() + policy = (grant('read'),) + if fault == 'empty_policy': + policy = () + elif fault == 'narrow_scope': + policy = (grant('read', descendants=False),) + elif fault == 'revoked': + tx.parent.revoked_at = NOW + elif fault == 'inactive': + tx.vault = ('key', 'destroyed') + else: + tx.vault = ('other-key', 'active') + with pytest.raises(Failure, match='invalid_grant'): + await require_source(tx, tx.body, NOW, custodial_ceiling=policy) + + +@pytest.mark.asyncio +async def test_empty_registered_key_ceiling_never_uses_custodial_policy(): + tx = SourceSession(custodial=False) + tx.parent.ceiling = () + with pytest.raises(Failure, match='invalid_grant'): + await require_source(tx, tx.body, NOW, custodial_ceiling=(grant('read'),)) + + +@pytest.mark.asyncio +async def test_registered_source_remains_independent_of_custodial_policy(): + tx = SourceSession(custodial=False) + assert await require_source(tx, tx.body, NOW, custodial_ceiling=()) is tx.user + + +@pytest.mark.asyncio +@pytest.mark.parametrize('expanded', [False, True]) +async def test_derived_credential_cannot_exceed_captured_custodial_grants(expanded): + tx = SourceSession() + tx.oauth = True + credential = SimpleNamespace( + id='t_oauth_test', + source_credential_id='key', + ceiling=(grant('read', 'write') if expanded else grant('read'),), + ) + if expanded: + with pytest.raises(Failure, match='invalid_grant'): + await require_binding( + tx, + credential, + NOW, + OAuthConfig(enabled=True), + custodial_ceiling=(grant('read', 'write'),), + ) + else: + await require_binding( + tx, + credential, + NOW, + OAuthConfig(enabled=True), + custodial_ceiling=(grant('read', 'write'),), + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize('contracted', [False, True]) +async def test_api_binding_without_captured_ceiling_uses_current_signing_key(contracted): + tx = SourceSession(custodial=False) + tx.api = True + credential = SimpleNamespace( + id='t_api_test', source_credential_id='key', ceiling=(grant('read'),) + ) + if contracted: + tx.parent.ceiling = () + with pytest.raises(Failure, match='invalid_grant'): + await require_binding(tx, credential, NOW, None, custodial_ceiling=()) + else: + await require_binding(tx, credential, NOW, None, custodial_ceiling=()) From 95f5c6ba654c984b080e280fcf92de3c16f32746 Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 18:27:46 +0800 Subject: [PATCH 08/11] Verify personal statement history, current key revocation and signed note lifecycle Add thirteen real PostgreSQL vectors across SOUL, AGENTS, Notes and Legacy. Verify exact history/content proofs, current authority loss, full-row rollback for body tampering/foreign signatures and note archive/restore proof conservation. --- tests/test_personal_signed_boundaries.py | 191 +++++++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 tests/test_personal_signed_boundaries.py diff --git a/tests/test_personal_signed_boundaries.py b/tests/test_personal_signed_boundaries.py new file mode 100644 index 00000000..0b1d0ac9 --- /dev/null +++ b/tests/test_personal_signed_boundaries.py @@ -0,0 +1,191 @@ +"""Independent statements keep historical proof without retaining current authority.""" + +from dataclasses import replace + +import pytest +from read_only_evidence import business_snapshot, readonly_evidence +from test_legacy_directive import signed_will +from test_personal_content_signatures import signed_arguments +from test_service import NOW, call, register + +from msg.core.codec import canonical, wire +from msg.security.crypto import verify + +KINDS = ['soul', 'agents', 'note', 'legacy'] + + +async def statement(app, key, owner, kind, body, *, ref=None): + options = { + 'resource_id': ref.id if ref else None, + 'parent_revision': ref.revision if ref else None, + } + if kind == 'legacy': + args, manifest = await signed_will( + app, key, owner, final_message=body, visibility='private', **options + ) + return 'identity.legacy_put', args, manifest + args, manifest = await signed_arguments(app, key, owner, kind=kind, body=body, **options) + if kind == 'note': + args.pop('kind') + args['name'] = 'private-plan' + return 'identity.note_put', args, manifest + return 'identity.personal_put', args, manifest + + +async def saved_statement(app, key, owner, kind): + op, args, manifest = await statement(app, key, owner, kind, 'privatehistoryneedle first') + saved = await call(app, op, args, key=key, subject=owner, contract_version=2) + assert saved.status == 'ok', wire(saved) + return op, saved, manifest + + +@pytest.mark.asyncio +@pytest.mark.parametrize('kind', KINDS) +async def test_statement_history_is_private_and_provable_after_signing_key_revocation( + installed, monkeypatch, kind +): + app, _ = installed + key, owner, _ = await register(app, 'statement-owner') + other_key, other, _ = await register(app, 'statement-outsider') + op, first, original_manifest = await saved_statement(app, key, owner, kind) + ref = first.resources[0] + async with app.metadata.transaction(write=False) as tx: + original = await tx.revision(ref) + original_bytes = await app.contents.read_bytes(original.content) + assert (await tx.resource(ref.id)).mode == 0o600 + verify(key.public_key, canonical(original_manifest), original.signature, purpose='revision') + _, args, new_manifest = await statement( + app, key, owner, kind, 'privatehistoryneedle second', ref=ref + ) + second = await call( + app, + op, + args, + key=key, + subject=owner, + expected=((ref.id, first.data['generation']),), + contract_version=2, + ) + assert second.status == 'ok', wire(second) + async with app.metadata.transaction(write=False) as tx: + current = await tx.revision(second.resources[0]) + verify(key.public_key, canonical(new_manifest), current.signature, purpose='revision') + assert current.parents == (ref.revision,) + assert await tx.revision(ref) == original + queries = [ + {'id': ref.id}, + {'id': ref.id, 'view': 'meta'}, + {'id': ref.id, 'view': 'history'}, + {'id': ref.id, 'revision': ref.revision}, + ] + async with readonly_evidence(app, monkeypatch): + for query in queries: + own = await call(app, 'discovery.get', query, key=key, subject=owner) + assert own.status == 'ok', wire(own) + for reader, subject in [(None, None), (other_key, other)]: + denied = await call(app, 'discovery.get', query, key=reader, subject=subject) + assert denied.status == 'error' and denied.error.code == 'permission_denied' + assert 'privatehistoryneedle' not in str(wire(denied)) + found = await call( + app, + 'discovery.search', + {'query': 'privatehistoryneedle'}, + key=other_key, + subject=other, + ) + assert found.status == 'ok' and ref.id not in str(found.data) + assert 'privatehistoryneedle' not in str(found.data) + _, pending, _ = await statement( + app, key, owner, kind, 'third statement', ref=second.resources[0] + ) + async with app.metadata.transaction(write=True) as tx: + await tx.put(replace(await tx.credential(key.key_id), revoked_at=NOW)) + before = await business_snapshot(app) + denied = await call( + app, + op, + pending, + key=key, + subject=owner, + expected=((ref.id, second.data['generation']),), + contract_version=2, + ) + assert denied.status == 'error' and denied.error.code == 'credential_revoked' + async with readonly_evidence(app, monkeypatch): + for query in queries: + denied = await call(app, 'discovery.get', query, key=key, subject=owner) + assert denied.status == 'error' and denied.error.code == 'credential_revoked' + assert await business_snapshot(app) == before + async with app.metadata.transaction(write=False) as tx: + assert await tx.revision(ref) == original + assert await app.contents.read_bytes(original.content) == original_bytes + verify(key.public_key, canonical(original_manifest), original.signature, purpose='revision') + + +@pytest.mark.asyncio +@pytest.mark.parametrize('kind', KINDS) +@pytest.mark.parametrize('fault', ['body_tamper', 'foreign_content_key']) +async def test_invalid_statement_content_proof_rolls_back_every_authoritative_fact( + installed, kind, fault +): + app, _ = installed + key, owner, _ = await register(app, 'proof-owner') + foreign_key, _, _ = await register(app, 'proof-foreign') + op, first, _ = await saved_statement(app, key, owner, kind) + _, args, manifest = await statement( + app, key, owner, kind, 'signed update', ref=first.resources[0] + ) + if fault == 'body_tamper': + args['final_message' if kind == 'legacy' else 'body'] = 'changed after signing' + code = 'invalid_signature' + else: + args['content_signature'] = wire(foreign_key.sign(canonical(manifest), purpose='revision')) + code = 'content_signer_mismatch' + before = await business_snapshot(app) + result = await call( + app, + op, + args, + key=key, + subject=owner, + expected=((first.resources[0].id, first.data['generation']),), + contract_version=2, + ) + assert result.status == 'error' and result.error.code == code, wire(result) + assert await business_snapshot(app) == before + + +@pytest.mark.asyncio +async def test_note_archive_restore_preserves_independent_revision_and_content(installed): + app, _ = installed + key, owner, _ = await register(app, 'signed-note-lifecycle') + _, saved, manifest = await saved_statement(app, key, owner, 'note') + ref = saved.resources[0] + async with app.metadata.transaction(write=False) as tx: + original = await tx.revision(ref) + proof = tx.one( + 'SELECT signature,signed_envelope FROM personal_revision_proofs WHERE revision_id=?', + (ref.revision,), + ) + generation = saved.data['generation'] + transitions = [('identity.note_archive', 'archived'), ('identity.note_restore', 'active')] + for operation, state in transitions: + result = await call( + app, + operation, + {'name': 'private-plan', 'expected_revision': ref.revision}, + key=key, + subject=owner, + expected=((ref.id, generation),), + ) + assert result.status == 'ok' and result.data['state'] == state, wire(result) + generation = result.data['generation'] + async with app.metadata.transaction(write=False) as tx: + assert await tx.revision(ref) == original + assert (await tx.resource(ref.id)).revision == ref.revision + assert tx.one( + 'SELECT signature,signed_envelope FROM personal_revision_proofs WHERE revision_id=?', + (ref.revision,), + ) == proof + assert await app.contents.read_bytes(original.content) == b'privatehistoryneedle first' + verify(key.public_key, canonical(manifest), original.signature, purpose='revision') From 14022b6cd4cda74d1e7d6f388f0bf769fc8cdbe0 Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 18:29:47 +0800 Subject: [PATCH 09/11] Apply cloud Ruff formatting suggestion for the personal proof assertion --- tests/test_personal_signed_boundaries.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tests/test_personal_signed_boundaries.py b/tests/test_personal_signed_boundaries.py index 0b1d0ac9..9bdcd4fb 100644 --- a/tests/test_personal_signed_boundaries.py +++ b/tests/test_personal_signed_boundaries.py @@ -183,9 +183,12 @@ async def test_note_archive_restore_preserves_independent_revision_and_content(i async with app.metadata.transaction(write=False) as tx: assert await tx.revision(ref) == original assert (await tx.resource(ref.id)).revision == ref.revision - assert tx.one( - 'SELECT signature,signed_envelope FROM personal_revision_proofs WHERE revision_id=?', - (ref.revision,), - ) == proof + assert ( + tx.one( + 'SELECT signature,signed_envelope FROM personal_revision_proofs WHERE revision_id=?', + (ref.revision,), + ) + == proof + ) assert await app.contents.read_bytes(original.content) == b'privatehistoryneedle first' verify(key.public_key, canonical(manifest), original.signature, purpose='revision') From 9224deea034fd0823c74e193d068d3213007dea6 Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 18:45:09 +0800 Subject: [PATCH 10/11] Use current credential persistence API in revocation test setup --- tests/test_personal_signed_boundaries.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/test_personal_signed_boundaries.py b/tests/test_personal_signed_boundaries.py index 9bdcd4fb..35bb1c8e 100644 --- a/tests/test_personal_signed_boundaries.py +++ b/tests/test_personal_signed_boundaries.py @@ -99,7 +99,10 @@ async def test_statement_history_is_private_and_provable_after_signing_key_revoc app, key, owner, kind, 'third statement', ref=second.resources[0] ) async with app.metadata.transaction(write=True) as tx: - await tx.put(replace(await tx.credential(key.key_id), revoked_at=NOW)) + subject = await tx.subject(owner) + await tx.save_credential( + replace(await tx.credential(key.key_id), revoked_at=NOW), subject.auth_version + ) before = await business_snapshot(app) denied = await call( app, From 597cb1d62cc5655fe7f41c8518c50f330b0df17a Mon Sep 17 00:00:00 2001 From: LIghtJUNction Date: Wed, 30 Sep 2026 18:48:11 +0800 Subject: [PATCH 11/11] Include all personal proof boundary cases in both PostgreSQL locale gates Preserve the concurrent save_credential setup fix at 9224deea. Keep the complete core gate and run these thirteen cases in the targeted dual-locale workflow as well. --- .github/workflows/integration-regressions.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/integration-regressions.yml b/.github/workflows/integration-regressions.yml index 7c55a8fa..d33eb096 100644 --- a/.github/workflows/integration-regressions.yml +++ b/.github/workflows/integration-regressions.yml @@ -30,6 +30,7 @@ on: - 'src/msg/storage/read_only.py' - 'tests/test_oauth.py' - 'tests/test_oauth_authority.py' + - 'tests/test_personal_signed_boundaries.py' - 'tests/test_http_effect_order.py' - 'tests/test_read_entry_contract_matrix.py' - 'tests/test_read_query_keyset_boundary.py' @@ -67,6 +68,7 @@ on: - 'src/msg/storage/read_only.py' - 'tests/test_oauth.py' - 'tests/test_oauth_authority.py' + - 'tests/test_personal_signed_boundaries.py' - 'tests/test_http_effect_order.py' - 'tests/test_read_entry_contract_matrix.py' - 'tests/test_read_query_keyset_boundary.py' @@ -185,6 +187,7 @@ jobs: run: >- python -m pytest -q tests/test_oauth.py tests/test_oauth_authority.py + tests/test_personal_signed_boundaries.py tests/test_http_effect_order.py tests/test_legacy_routes_removed.py tests/test_read_query_keyset_boundary.py tests/test_read_entry_contract_matrix.py tests/test_recovery_runtime_generation.py