Skip to content

feat: harden live macOS runtime and governance - #1317

Draft
seonghobae wants to merge 31 commits into
fix/workspace-document-registry-migrationfrom
codex/naruon-live-audit
Draft

seonghobae wants to merge 31 commits into
fix/workspace-document-registry-migrationfrom
codex/naruon-live-audit

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

Current repair status — Draft / prerequisite-first

Exact head: af362d58190c0bf2ed122d718473fe3c2bd503c4 (tree 027eb9d28c2a3677dd11d2ffd878fd4ef1c3fe29).
Normal merge parents: original owner 1b422f15e6e5f56be679f691c8ff925c9a420fb1 and migration owner #1503 19d5860bc27e860acba940390f5792721cd99e5e. This retargets onto #1503's fix/workspace-document-registry-migration branch without force push, dropped predecessor delta, or deleted migration IDs.

Current repair:

  • retain the actual provider-lookup connection across account import leases and per-item commits, including a one-slot pool;
  • invalidate uncertain acquisition/unlock and complete cleanup even after repeated cancellation;
  • prevent SQL on a replacement connection after actual lease-holder termination;
  • namespace new persisted body/attachment graph identities by account without changing original Message-IDs or deleting global constraints;
  • join both migration histories with no-DDL 0020_merge_import_registry, use Alembic's native graph inspection, and mark the open-PR local-runtime ADR Proposed.

Committed-head verification: 242 passed in 11.41s, exit 0, after fresh and repeated actual PostgreSQL migrations; task-only DB container/network cleanup confirmed. This includes real concurrent same-owner/other-user/other-organization imports and signed backend ASGI import/unsigned rejection. The old API suite also contains mocks; its test names alone do not prove signed DB behavior. Focused Ruff and git diff --check passed; the worktree remained clean.

Command from backend/, after task-isolated DB provisioning:

uv sync --locked
uv run --locked python scripts/migrate_db.py
uv run --locked python scripts/migrate_db.py
uv run --locked python -m pytest -q -W error -ra --tb=short \
  tests/test_email_import_lease_postgres.py tests/test_email_import_service.py \
  tests/test_emails_api.py tests/test_alembic_migrations.py \
  tests/test_workspace_document_migration.py tests/test_email_read_state_migration_postgres.py \
  tests/test_legacy_document_scope_postgres.py tests/test_attachment_parser.py \
  tests/test_bootstrap_db.py tests/test_container_dependency_pin_contract.py tests/test_data_api.py

JUnit import_af362d5.xml: SHA-256 238da7ad1c6e772cd087a3576f7c81d4d81780a6f0f0b4b3986a229910da965c (local receipt, not uploaded hosted evidence).
Decision, RED receipts, replay source and limits.

Remaining gates: #1503 predecessor integration; #1562 migrated-PostgreSQL Application CI prerequisite (owner request); fresh required checks and qualifying independent review on the exact head. Local success is not hosted approval, protected merge, deployed provider/browser behavior, or p95/100% coverage acceptance. Existing default-branch dependency findings also remain required security work, not waived by these tests.

Caller precondition remains settled/read-only work before import: the pending-ORM guard cannot detect already-flushed or raw-SQL uncommitted writes. The actual provider-lookup caller is verified; do not generalize to arbitrary external transactions.

Preserved earlier PR record

The following is historical proposal/validation material retained for provenance. Earlier counts and live-mail/model/browser observations do not transfer to the current head and have not been rerun in this repair.

Summary

  • Add macOS Colima Compose runtime selection: mlx-lm -> llama.cpp -> Ollama.
  • Route EmbeddingGemma through a separate llama.cpp endpoint and prevent tenant API keys from being sent to local runtimes.
  • Disable MLX thinking for local structured/translation/draft calls so reasoning cannot exhaust the response before content.
  • Harden NUL/control-character handling, grounded project extraction defaults, compound relational naming, Keyverse OIDC boundaries, and privileged workflow archive materialization.
  • Document PR/check/review iteration and non-admin merge policy in ADRs.

Historical validation

  • Backend: 1739 passed, 33 skipped after semantic-segment embedding update (a57f8a75); current-head CI reruns after each branch update
  • Frontend: 427 passed; lint and typecheck passed
  • Full product UI smoke: 10 routes, critical interactions, and accessibility checks passed
  • Live Playwright: 4 passed, including real MLX draft and EmbeddingGemma search
  • Live mail smoke: import/search/frontend visibility/summary/draft passed
  • MLX chat: 200 with non-empty content
  • EmbeddingGemma: 200 with 768-dimensional vectors
  • actionlint, shell syntax checks, and test_pr_governance_gate.sh: passed

Security boundary

Privileged workflow logic is materialized only from a trusted full SHA. PR head content is handled as data; archive traversal, links, devices, and workspace escapes are rejected. No admin merge or branch-protection bypass is used.

Summary by CodeRabbit

  • New Features
    • Added support for local LLM runtimes, including Ollama, MLX, and llama.cpp.
    • Added separate chat and embedding endpoints, including EmbeddingGemma support.
    • Improved email import, semantic search, and grounded answers with segmented, token-aware embeddings.
    • Project graph extraction now defaults to the orchestrated approach.
  • Bug Fixes
    • Improved handling of oversized embedding inputs and local structured responses.
    • Strengthened authentication input validation and database migration safety.
    • Improved macOS runtime detection, container connectivity, and email import verification.
  • Documentation
    • Added guidance for OIDC setup, local AI configuration, database naming, and security practices.

Live mail validation update

The actual iCloud mail directory supplied by the operator was exercised through the running Colima stack without copying private mail into the repository. Real-mail import completed with zero failed items, same-owner sequential imports released all PostgreSQL advisory locks, and inbox/search visibility was verified through both the backend API and the same-origin frontend cookie proxy. Private message contents and credentials were not included.

Semantic embedding update

Import embeddings now use the content-graph parser's heading/paragraph/structured-field segments first. Only an oversized semantic segment is physically split for provider safety; its vectors are pooled back to the segment, then to the existing email/attachment source centroid. The persisted content segments remain the Ontology and Project Graph citation units.


Open in Devin Review

@coderabbitai

coderabbitai Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR adds separate local chat and embedding routing, semantic embedding chunking, stricter authentication and governance validation, safer email import locking, canonical read-state migrations, updated macOS Compose detection, and new policy documentation.

Changes

Local runtime and embedding integration

Layer / File(s) Summary
Provider and embedding contracts
backend/core/config.py, backend/services/llm_provider_selection.py, backend/api/search.py, backend/services/batch_embedding_service.py
Provider settings now support dedicated embedding endpoints and local-environment providers.
Semantic embedding processing
backend/services/embedding.py, backend/services/email_import_service.py, backend/tests/test_embedding.py, backend/tests/test_email_import_service.py
Email content is segmented, token-bounded, embedded, and pooled by source.
Local LLM and macOS wiring
backend/services/llm_service.py, scripts/naruon_compose.sh, docker-compose.macos.yml, README.md
Local providers use JSON-object responses with disabled thinking. macOS setup detects and configures local runtimes.

Input, identity, and live-smoke safety

Layer / File(s) Summary
Identity and local HTTP validation
backend/api/auth.py, backend/tests/test_auth_real.py, backend/tests/test_local_http.py, docs/adr/0005-keyverse-oidc-trust-boundary.md
Claims reject control characters and invalid ASCII. Local HTTP validation rejects NUL characters.
Import locking and smoke handling
backend/services/email_import_service.py, backend/scripts/private_mail_http_smoke.py, AGENTS.md
Import locks use validated hashed owner keys and dedicated connections. Mail smoke tests use an environment secret and API visibility checks.

Governance, schema, and extractor defaults

Layer / File(s) Summary
Trusted governance materialization
.github/workflows/pr-governance.yml, backend/tests/test_release_governance.py, docs/adr/0006-privileged-workflow-archive-safety.md
Trusted base SHAs and archive members receive explicit validation before governance code runs.
Canonical email schema migration
backend/alembic/versions/0018_email_read_state_ownership.py, backend/alembic/versions/0019_merge_email_read_state_ownership.py, backend/tests/test_alembic_migrations.py
Read-state ownership targets email_records, preserves compatible columns, and reconciles migration heads.
Project-graph selector and repository policy
backend/core/config.py, backend/services/project_graph/extractor_registry.py, docs/architecture/kg-extractor-seam.md, docs/adr/0002-compound-snake-case-database-names.md
The default extractor is now orchestrator. Deterministic extraction remains a reference fallback.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes two primary changes: hardening the live macOS runtime and governance controls.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/naruon-live-audit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

@github-actions

github-actions Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

PR governance metadata gate is not ready for 1b422f15e6e5f56be679f691c8ff925c9a420fb1:

  • Required check strix is FAILURE on the current head.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 11, 2026

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head cf8c21e4b0843aef49f8c15d4f0f93816178dc09.

  • Head SHA: cf8c21e4b0843aef49f8c15d4f0f93816178dc09

  • Workflow run: 31542616436

  • Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (6 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (6 files)"]
  R1 --> V1["required checks"]
  Evidence --> S2["Workflow: pr-governance.yml"]
  S2 --> I2["GitHub Actions review job"]
  I2 --> R2["Review risk: Workflow: pr-governance.yml"]
  R2 --> V2["actionlint plus required checks"]
  Evidence --> S3["Backend (30 files)"]
  S3 --> I3["API and service runtime"]
  I3 --> R3["Review risk: Backend (30 files)"]
  R3 --> V3["backend tests"]
  Evidence --> S4["Docs (7 files)"]
  S4 --> I4["operator or user guidance"]
  I4 --> R4["Review risk: Docs (7 files)"]
  R4 --> V4["docs review"]
Loading

@opencode-agent

opencode-agent Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: 714a2e5d49095f3124025dda9b096ae12e9266c5
  • Workflow run: 32025069092
  • Workflow attempt: 1
  • Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 714a2e5d49095f3124025dda9b096ae12e9266c5.

  • Head SHA: 714a2e5d49095f3124025dda9b096ae12e9266c5

  • Workflow run: 32025069092

  • Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (6 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (6 files)"]
  R1 --> V1["required checks"]
  Evidence --> S2["Workflow: pr-governance.yml"]
  S2 --> I2["GitHub Actions review job"]
  I2 --> R2["Review risk: Workflow: pr-governance.yml"]
  R2 --> V2["actionlint plus required checks"]
  Evidence --> S3["Backend (32 files)"]
  S3 --> I3["API and service runtime"]
  I3 --> R3["Review risk: Backend (32 files)"]
  R3 --> V3["backend tests"]
  Evidence --> S4["Docs (7 files)"]
  S4 --> I4["operator or user guidance"]
  I4 --> R4["Review risk: Docs (7 files)"]
  R4 --> V4["docs review"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head cf8c21e4b0843aef49f8c15d4f0f93816178dc09.

  • Head SHA: cf8c21e4b0843aef49f8c15d4f0f93816178dc09

  • Workflow run: 31542616436

  • Workflow attempt: 2

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (6 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (6 files)"]
  R1 --> V1["required checks"]
  Evidence --> S2["Workflow: pr-governance.yml"]
  S2 --> I2["GitHub Actions review job"]
  I2 --> R2["Review risk: Workflow: pr-governance.yml"]
  R2 --> V2["actionlint plus required checks"]
  Evidence --> S3["Backend (30 files)"]
  S3 --> I3["API and service runtime"]
  I3 --> R3["Review risk: Backend (30 files)"]
  R3 --> V3["backend tests"]
  Evidence --> S4["Docs (7 files)"]
  S4 --> I4["operator or user guidance"]
  I4 --> R4["Review risk: Docs (7 files)"]
  R4 --> V4["docs review"]
Loading

seonghobae commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Actual iCloud mail directory validation remains scoped to the running Colima stack; no private message content or credentials are included.

The import embedding path has now been corrected to use content-graph semantic segments (heading/paragraph/structured fields) as primary units, preserving heading context. Only oversized segments are split to the physical provider ceiling, then pooled back to the semantic segment and finally to the existing email/attachment centroid. The contextual-orchestrator batch path receives the same bounded physical inputs.

Validation on the local semantic update: backend 1739 passed, 33 skipped; focused embedding/import/batch suites 79 passed; Ruff and git diff --check passed. Local commit 9c1250c; remote PR head a57f8a7. Current-head GitHub Checks and independent review must rerun for this head before normal protected Merge.

coderabbitai[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Contributor Author

Current-head repair record (2026-08-12 UTC):

  • new head: 44563f82f0e144f0bfeebd6ede15871c2cfec381
  • restored published Alembic revision 0011_email_read_state byte-for-byte from the exact base and moved canonical email_records ownership/schema work into new 0018_email_read_state_ownership, with 0019_merge_email_read_state_ownership preserving a single graph head;
  • embedding inputs now use the selected model tokenizer with a conservative 256-token ceiling and token-count-weighted pooling, including the contextual-orchestrator batch path;
  • both local runtime branches assign and validate converted host URLs before export;
  • focused Python AST and shell syntax validation passed locally from the exact remote files.

This addresses the three current actionable review findings. The central trusted-uv coverage repair remains the prerequisite for a fresh exact-head OpenCode review; no predecessor review or check evidence is being transferred.

Copy link
Copy Markdown
Contributor Author

Follow-up current-head repair (2026-08-12 UTC):

  • head advanced to 0b5f4478e7084c7856caafbebf65757f83f12839;
  • fixed the exact CI failures from run 31565044989: the merge revision identifier is now within Alembic's 32-character version_num limit, and the batch embedding regression asserts the new tokenizer-token ceiling rather than the obsolete character ceiling;
  • no security boundary, migration ownership rule, or provider credential behavior was weakened.
    Fresh exact-head Checks are required before review/merge evaluation.

coderabbitai[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Contributor Author

Current-head CI repair (head a6f36eb35f9cedbc9125f249375227be4aa0ef66):

  • restored the migration regression test path to the existing 0019_merge_email_read_state_ownership.py file while retaining the Alembic-safe revision id 0019_merge_read_state_ownership;
  • changed the long-input batch regression expectation to the production token-weighted pooling formula instead of an unweighted character-era average.

The branch is being rechecked from this exact head. No approval or merge decision is inferred from this repair.

Copy link
Copy Markdown
Contributor Author

TDD red phase for the next review repairs (head 844a3cbae7644b2945afcbb7691cee31b2c891b3): added failing contract coverage for the canonical email_records migration target and for preserving Unicode when a token boundary bisects a UTF-8 character. The next commit will implement only those root-cause fixes after this exact-head run reports the expected failures.

Copy link
Copy Markdown
Contributor Author

TDD green implementation at exact head d225700ddf17767a337949d37c3fb0e1914e7eb0: the published read-state migration now targets the canonical email_records table and has execution-based upgrade/downgrade coverage; token slices now use full-text offsets so UTF-8 boundaries cannot produce replacement characters. The exact-head CI rerun is in progress.

Copy link
Copy Markdown
Contributor Author

TDD red phase for the remaining local-tokenizer review (head 8cc7e16964449ff06bde62b45046231856eab305): added a provider-boundary regression requiring local embeddinggemma calls to use the model-native /tokenize and /detokenize endpoints before embedding. The implementation will follow the documented llama.cpp contract; current tiktoken fallback remains only for known local-test/offline paths until the green implementation lands.

@seonghobae
seonghobae dismissed stale reviews from coderabbitai[bot] and opencode-agent[bot] August 22, 2026 07:46

Stale review: all review-thread comments on this PR are resolved and the reviewer's cited commit predates the current head, which passes all non-metadata-gate required checks (verified via gh pr checks and the reviewThreads GraphQL query — 0 unresolved threads). Dismissing as superseded per AGENTS.md stale-review guidance.

@opencode-agent opencode-agent Bot added area: security Security boundary, hardening, or vulnerability prevention priority: medium Normal-priority or P2 work status: needs-review Open pull request requiring current-head review or checks type: feature New or expanded product capability labels Aug 22, 2026
@seonghobae

Copy link
Copy Markdown
Contributor Author

Root-repair handoff at 1b422f15e6e5f56be679f691c8ff925c9a420fb1; import runtime reproduction remains pending. Preserve this PR's dedicated-connection and NUL-safe owner-key delta.

Acquisition lines285–298 catches Exception but not asyncio.CancelledError, so cancellation while acquiring bypasses this helper's explicit connection cleanup. Release lines326–338 does not inspect the unlock response and closes normally after uncertainty. Test actual acquisition cancellation, require confirmed unlock, and invalidate uncertain ownership before cleanup can wait or pool the connection. Budget the separate lease connection alongside the already-used work session, including supported one-slot configurations.

Add migrated-PostgreSQL tests around real item commits/rollbacks, same-pool readers, independent-replica reacquisition, cancelled acquisition, failed unlock, and retained imported bytes/identities. Keep tests/test_email_import_service.py and tests/test_emails_api.py coverage; mocked query ordering alone is insufficient.

#1469 exact worker investigation reproduced the original pool-return defect and records cleanup-order pitfalls. Its305-pass/100%-worker-coverage receipt is not this import path's GREEN. No source takeover, copied implementation, approval, closure, or protected-readiness claim.

Normally integrate migration owner #1503 at 19d5860 without discarding #1317 history. Preserve both migration branches with a no-DDL merge revision and retain owner-scoped mail and graph identities.

Record actual bounded-pool, cancellation, lost-connection and signed API evidence in doctoring; keep ADR Proposed and hosted CI prerequisite #1562 explicit. Candidate verification: 242 focused tests passed against fresh and repeated PostgreSQL migrations; no protected merge or release claimed.
@seonghobae
seonghobae changed the base branch from develop to fix/workspace-document-registry-migration September 6, 2026 08:07

Copy link
Copy Markdown
Contributor Author

Supplemental read-only agent review of af362d58190c0bf2ed122d718473fe3c2bd503c4 against first parent 1b422f15e6e5f56be679f691c8ff925c9a420fb1 reported no actionable findings within cancellation cleanup, replacement-connection SQL, the documented settled/read-only caller precondition, and account-scoped graph identities. It also reported three isolated in-memory repeated-cancellation probes passing. The agent did not rerun or independently verify the 242-test PostgreSQL suite, edit the checkout, or mutate GitHub. This comment records bounded review provenance only; it is not a qualifying protected approval or current hosted gate evidence. Existing prerequisite and product acceptance gaps remain as stated in the PR body.

seonghobae added a commit that referenced this pull request Sep 6, 2026
Advance baseline to 1.32 with #1317 af362d5's own 242-test migrated PostgreSQL receipt and #1566 5b5a49c's 47-test source-only playbook receipt. Preserve historical records, full owner deltas, unresolved CI/review/release gates and the Noema domain boundary.
spec.loader.exec_module(revision)
calls = []
operations = SimpleNamespace(
get_bind=lambda: object(),
finish_projection.set()
expected_error = RuntimeError
with pytest.raises(expected_error):
await import_task
assert await _replica_can_import(observer_engine, owner_key) is False
acquire_task.cancel()
with pytest.raises(asyncio.CancelledError):
await acquire_task
else RuntimeError
)
with pytest.raises(expected_error):
await release_task
@seonghobae seonghobae removed the status: needs-review Open pull request requiring current-head review or checks label Sep 6, 2026
@seonghobae seonghobae added status: draft Draft pull request enhancement New feature or request labels Sep 6, 2026 — with ChatGPT Codex Connector
@seonghobae

Copy link
Copy Markdown
Contributor Author

2026-10-03 actual PostgreSQL/browser verification

Protected develop@042b0c70531b229af3acbd0421a2f23098d848b3 still fails fresh Alembic installation on the retired emails index. Private composition of #1694 with dependency owner #1828 fixes that startup seam, but actual same-origin browser import then fails with PostgreSQL CharacterNotInRepertoireError at the raw NUL owner-lock key. This reproduces an existing, not newly invented, product blocker.

I tested the complete existing #1317 source af362d58190c0bf2ed122d718473fe3c2bd503c4 with an explicit private overlay of #1828's canonical dependency artifacts/contracts 74f9d6dd3b34d1179ca5f2aa3c5c294d22c7d2b2. This is private integration evidence, not source adoption, protected shipment, hosted review or deployment. Import service, lease tests, models and migration bytes match #1317's immutable objects. No duplicate import implementation/PR was created.

Real execution:

  • isolated PostgreSQL 17 + pgvector, fresh managed migrations and idempotent rerun;
  • existing eleven-module owner acceptance command: 242 passed, 0 skipped, exit 0 with warnings fatal;
  • real Chrome -> Next same-origin /api/* -> signed FastAPI -> actual PostgreSQL, no API mocking or auth override;
  • anonymous mail rejected 401; synthetic member HttpOnly cookie session accepted 200; EML import imported=1/failed=0; list/detail/search read actual stored source; second owner lists zero rows;
  • browser sends no Authorization/public identity headers; no pageerror and no warning/error-class server logs;
  • owned test DB and server processes cleaned up.

Receipt directory (operator-local, not a hosted artifact): /Users/seonghobae/.hermes/cache/scratch/naruon-usable-20261003/; live-owner-integration-manifest.json, live-browser.json, owner-postgres.xml, owner-postgres.log, live-mail.png. The test used synthetic mail and ephemeral member HMAC credentials only; no customer secrets/mail or real provider execution.

#1381's MERGED object 754cea8d... is not current develop ancestry, so it cannot be credited as delivered. Existing #1503 workspace prerequisite and canonical #1828 security dependency owner remain the correct integration routes. Keep this PR's complete lease/graph protection; do not replace it with a NUL-only leaf fix. Fresh exact-integrated-head hosted checks, independent approval and normal protected merge remain required. Related to #1428.

@seonghobae

Copy link
Copy Markdown
Contributor Author

October 3, 2026 — real DB prerequisite reconciliation, not merged acceptance

The exact private #1317 af362d5 lineage with #1828 published64eff330 dependency bytes passes241 existing owner regression cases plus1 signed import/list/detail/search cross-user/cross-org isolation case on disposable PostgreSQL17, zero skips. Parent verified1885 unchanged source files, emitted JUnit and final command-log hashes. Repeated diagnostic log labels are explicitly marked superseded; no full historical-log preservation claim.

Fresh native bootstrap exits0 but emits unclosed asyncpg connection/transport/socket ResourceWarning, so strict acceptance FAILS. A diagnostic disposal wrapper is not a source repair. Migration and historical-shape upgrade/repeat preserve existing body/read-state values, but raw SQL omitting email_records.is_read still fails because no DB server_default exists. #1782's existing default repair remains needed; explicit is_read test seeds do not clear it.

Native Alembic inspection of immutable #1782 migration addition gives heads0018_email_record_read_state +0020_merge_import_registry. Adding current#1503 organization/personal successors gives0020_merge_import_registry +0021_workspace_personal_owner_binding. Current#1503head9151c755 is not ancestor of#1317; original19d5860b is. No existing IDs/edges were deleted/reparented/stamped and no upgrade-heads workaround used. Current registry semantic deltas cannot be claimed adopted from two migration files alone.

New owned scratch proposals are addressing only bootstrap executable cleanup and narrow existing-owner default/graph composition, preserving canonical source/dependency/security/CI owners. No duplicate PR, production edit, commit/push or protected merge. Complete source review and actual fresh/existing DB acceptance are required before any owner adoption. Existing CI isolatedcapacity/HOLD remains independent.

@seonghobae

seonghobae commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor Author

October 3, 2026 — whole composed-source review FAILED; owner escalation

Frozen private tree c37aa3db957aca344316b920404ef1bd49882db8, protected-base whole patch f057c32065e63613d3c27ca3cf93d4f5d78bbfbb5eaedffdf77ff223c0b98a3d. Independent review covered all111 changed paths and1898 source files. Parent verified exact source and517 DB artifacts: final real PostgreSQL17.11 full backend1966pass/3conditional-skips, bootstrap12/storage24/signed routes4 pass. These functional passes do not clear the following source findings. Review was static; synthetic reproduction is pending, not a claim of observed production disclosure.

P1 — TENANT-KEY-CROSSES-GLOBAL-EMBEDDING-OVERRIDE

Locations: backend/services/embedding.py:255-258, backend/services/llm_provider_selection.py:121-132, backend/api/search.py:304-310, backend/services/email_import_service.py:1215-1220
A tenant-config runtime deliberately has embedding_base_url=None, but generate_embeddings treats an explicit None base URL as permission to select the global OPENAI_EMBEDDING_BASE_URL. With OPENAI_BASE_URL unset (normal default OpenAI tenant configuration), ALLOW_LOCAL_LLM_PROVIDERS=true and a configured local embedding URL, _local_environment_provider returns None because chat base_url is unset; tenant fallback retains its real API key and base_url=None. Search/import then pass that None into generate_embeddings, which selects the local global embedding endpoint and sends the real tenant key via tokenizer Authorization and AsyncOpenAI. The new embedding override crosses the tenant/local credential boundary despite the test asserting runtime.embedding_base_url is None. The SSRF allowlist permits the configured local endpoint and therefore does not prevent this credential disclosure.
Owner fix direction: Owner must separate default endpoint resolution from explicitly selected tenant/provider endpoints and keep local-only overrides paired with local-only credentials. Do not send tenant credentials to a process-level local embedding override.
Regression direction: Use the actual resolver plus search/import embedding consumer with tenant key, no chat base URL, local global embedding URL and local allowance. Assert external/default OpenAI remains selected and no local request carries tenant credentials; pair local-placeholder and explicit external-provider positives.

P2 — NEW-RUNTIME-ENV-CREDENTIAL-SOURCE

Locations: backend/core/config.py:135, backend/core/config.py:Settings.model_config and __init__, backend/services/llm_provider_selection.py:136-151, AGENTS.md:Config & secrets (KV, not env)
The candidate adds OPENAI_API_KEY to pydantic Settings loaded from environment/dotenv and reads get_secret_value at runtime to construct a provider. This is a new credential source, not bootstrap transport into the credential registry. The repository rule expressly disallows new runtime secret reads from environment; the sole added exception is LIVE_E2E_SESSION_SECRET in the private smoke script, not OPENAI_API_KEY in production provider resolution. Calling the path local_environment and requiring ALLOW_LOCAL_LLM_PROVIDERS does not constrain the value to a nonsecret placeholder.
Owner fix direction: Use the credential registry for actual credentials, or an explicitly nonsecret fixed local-provider placeholder without runtime environment secret sourcing. Obtain the source owner decision; do not broaden the exception during this review.
Regression direction: Prove local selection cannot consume an arbitrary environment API credential and registry-backed provider positives still work.

P2 — IMPORT-EMBEDDING-WINDOW-BOUND-REMOVED

Locations: backend/services/email_import_service.py:552-565, backend/services/email_import_service.py:1173-1188, backend/services/email_import_service.py:1215-1221, backend/services/embedding.py:266-285 and embeddings.create input
Protected import processed each source in MAX_EMBEDDING_CHUNKS_PER_WINDOW slices and accumulated a single source vector. The new semantic path sends all semantic segments at once, materializes all physical chunk strings/ranges/token weights, and retains all returned vectors before pooling. The direct provider path submits the entire request_texts list in one embeddings call. Only the partial-batch remainder remains windowed. This removes the existing bounded-request/vector-memory contract for large supported sources, permits provider batch-limit rejection, and can allocate source-size-times-1536 Python vector objects. Semantic segmentation does not itself bound segment count or tokens. CHANGELOG still describes bounded windows.
Owner fix direction: Keep semantic provenance and source centroids while windowing physical provider requests and streaming weighted pooling; preserve partial-batch order and no duplicate embedding.
Regression direction: Use many semantic segments and one oversized segment with an independent literal request-count ceiling. Assert every normal/direct/batch/remainder request is bounded and source pooling equals a small unwindowed mathematical control.

P2 — LOCAL-NATIVE-TOKENIZER-404-SILENT-FALLBACK

Locations: backend/services/embedding.py:63-70, backend/services/embedding.py:95-99, backend/services/embedding.py:267-281, docs/adr/0001-local-llm-and-orchestrator-boundary.md:decision9
For any unknown configured model, including a local EmbeddingGemma/llama.cpp endpoint, native tokenizer 404 unconditionally falls back to cl100k_base. The code makes no local-versus-remote distinction at that catch. Consequently 256 fallback-token chunks need not fit the local provider tokenizer/context, contradicting the explicit local provider-authoritative ceiling/fail-closed contract. The added remote-unknown-model 404 positive does not prove local rejection.
Owner fix direction: Retain explicitly justified remote compatibility separately; reject missing native tokenizer for unknown local model or use an exact versioned model tokenizer contract.
Regression direction: Pair unknown-local 404 rejection with a valid local native tokenizer positive and remote fallback control; demonstrate a text whose native token count exceeds fallback count.

P2 — CURRENT-CANDIDATE-DOC-CONTRACT-DRIFT

Locations: docs/doctoring/starlette-httpx2-testclient-dependency.md:Decision and boundary, CHANGELOG.md:Unreleased, docs/adr/0004-input-safety-and-evidence-first-judgment.md:semantic embedding decision
The newly included TestClient decision instructs pinning httpx2==2.5.0 and attributes this change to that wheel, and the changed Unreleased entry repeats 2.5.0. Actual reviewed pyproject/requirements/uv.lock select httpx2==2.13.1 from the security owner. ADR0004 also specifies a 256-character physical ceiling whereas embedding.py implements 256 tokenizer tokens. These are current instructions presented as the decision for this composition, not clearly superseded historical receipts, and disagree with executable/dependency definitions.
Owner fix direction: Owner must identify the old 2.5.0 evidence as historical and describe the actual reviewed dependency and token contract; preserve historical receipts unchanged.
Regression direction: Check documented current pin against the actual manifests and use token-versus-character positive controls for the documented physical boundary.

The source-repair/re-review cycle limit has been reached. No further autonomous product edits or publication from this failed candidate are authorized by the review process. Existing canonical source owners must decide the bounded provider/embedding repair direction. All old findings/attempts and immutable source are preserved; no duplicate PR, source write, commit/push, formal approval, protected merge or deployment.

Same-origin browser rehearsal also remains failed/unverified: initial build emitted symbol warning and Podman fixture transport failed before DB/browser start. A separate harness-only recovery is running without product edits. It cannot clear this source verdict. Personal orgless signed positive and full Linux frontend acceptance remain unresolved.

October 3, 2026 — synthetic final-transport reproduction completed

All five findings are confirmed within a bounded synthetic diagnostic, not customer/provider execution. Preserved run2 completed20 cases/exit0/harness-errors0 and matched1898 frozen source hashes. Actual resolver plus unchanged search/import consumer bodies and installed AsyncOpenAI serialization send the synthetic tenant Authorization to127.0.0.1 when tenant-selected URL isNone and global embedding override islocal. Default-tenant external, explicit external-provider and local-placeholder positives distinguish the destination/key pairing. No real API keys, external connect/DNS or provider calls; one import-time socket.bind was blocked by the audit hook. Exit0 means defect observation completed, NOT safe product acceptance.

Synthetic Settings environment credential reached local_environment with registry lookup0. Normal/direct physical requests33 exceed independent ceiling32; partial remainder32+1 is separately bounded. Unknown-local tokenizer404 continued embedding; actual model tokenizer mismatch remains unmeasured. Current documentation sayshttpx2 2.5.0/256characters versus actual2.13.1/256tokens; executed400characters/201tokens/one-chunk control confirms the distinction. Earlier run1 placeholder rejection and later instrumentation run3/4 timeouts remain failed; current diagnostic.py replay reliability is unresolved. Preserved run2 is the completed evidence; historical window-removal attribution was not reverified by this diagnostic's failed protected-blob lookup.

Browser harness recovery also STOPPED after its second failure. Podman CLI transport works and genuine build cache removes the no-cache message, but warm pnpm build emitted13WARN rows and unexpectedly recreated/downloaded dependencies despite the requested no-repeat-network boundary. That violation is disclosed, logs retained, and no third rerun. No DB/app/Chromium/UI journey or screenshot executed in either recovery; no browser PASS. No product source edits or unrelated resource cleanup.

The complete failed111-path review/patch/binding plus run2 diagnostic, final DB result/JUnit and browser stop log are preserved outside scratch in a hash-verified owner-escalation packet. Source repair cycles remain exhausted. Existing source owner/expert must take a bounded next repair under normal review/security/protection gates; no further autonomous product edits, owner replacement, credential changes, commit/push/merge/deploy. GitHub auth-status returned1 but normal existing gh PR read/write is working; no credentials changed or route bypassed.

@seonghobae

Copy link
Copy Markdown
Contributor Author

October 4, 2026 KST — verified read-only repair contract ready for existing owner

The expert handoff is complete and parent-verified, not implemented or approved. Contract SHA256 34375f05303138a5200b142396fa5002f9b90de815692f45cd55669a13e841de; all1898 frozen candidate source hashes and33 focused source hashes match, dependency read receipts are bound. Durable location: /Users/seonghobae/orca/reports/naruon-usable-20261003/source-escalation/expert-contract/repair-contract.ko.md.

The contract distinguishes unconfigured/default/explicit endpoint states; binds credentials to the selected endpoint without ambient global override; reuses existing encrypted LLMProvider/TenantConfig rather than inventing a registry API; uses a nonsecret local placeholder until authenticated local ownership is explicitly established; fails closed for unknown-local native tokenizer absence; limits final physical requests to32 and preserves token-weighted physical→semantic plus equal-semantic→source pooling. Completed-prefix batch work must not be submitted again on partial fallback. Exact search/import/grounded-answer/local-engine/orchestrator consumer RED→GREEN matrix and documentation corrections are specified, not executed.

Crucially, narrow leakage containment is not approval of Naruon-owned provider/model routing. Existing#1540/contextual-orchestrator#1023 released-client/schema requirement remains, last successful release query returns0. Preserve#1480/#1525/#1384 authority and existing branch writers; no invented CO API, copied wire schema or default routing workaround. Owner decisions required: precise source adopter, endpoint/default compatibility, authenticated-local credential binding and tokenizer/batch authority. Source repair budget remains exhausted; this handoff does not reset it. The existing owner must settle that design/ownership boundary before a new implementation phase.

Actual prior DB1966pass/3conditional-skips does not clear the five failed source findings. Latest browser attempts remain unexecuted/failed. No candidate or canonical source edits, product execution, provider/DB access, adoption, commit/push, formal approval, merge or deployment were performed by this contract task. Current canonical1828head64eff330 remains clean; owner coordination reply is still unreceived.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: security Security boundary, hardening, or vulnerability prevention enhancement New feature or request priority: medium Normal-priority or P2 work status: draft Draft pull request type: feature New or expanded product capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant