Fix Sentry symbolication: native/Java backtraces framework-only, JS frames anonymous on v0.4.17

Problem (verified 2026-11-08)

New production issues on v0.4.17 are undiagnosable:

OPENCODE-MOBILE-A: JS error, frames anonymous (no app frames after source-map resolution).
OPENCODE-MOBILE-B / -C: FATAL, clipping family (ReactClippingViewManager / child-already-has-a-parent family), stack shows framework frames only — no deobfuscated app/Java frames.

Without a correct backtrace these cannot be mapped to code, so per the sentry-fix rule they sit in backlog. Fix the symbolication chain, not the crashes (yet).

What is already wired (verified in repo, do not redo)

@sentry/react-native ~6.22.0, Expo plugin in app.json, getSentryExpoConfig in metro.config.js (bundle Debug ID embedded).
android/sentry.gradle applied from android/app/build.gradle; android/sentry.properties org vibetechnologies / project opencode-mobile.
publish-play-store.yml + build.yml pass SENTRY_AUTH_TOKEN/ORG/PROJECT and pin SENTRY_RELEASE=opencode-mobile@<version>, SENTRY_DIST=<version> for uploads; runtime Sentry.init in src/lib/sentry.ts reports the same release/dist strings (JS side).

Prime suspects to audit first (kill-source: the v0.4.17 publish run's own logs)

Did the sentry upload tasks actually run and succeed in the v0.4.17 publish workflow run? Upload failures are currently SILENT — nothing fails the build if source-map / mapping / symbol upload fails or is skipped.
Native event dist mismatch: native (Java) crash events report dist = Android versionCode, while mapping upload is pinned to SENTRY_DIST=. ProGuard/R8 mapping association by release+dist would then miss, leaving Java frames obfuscated/framework-only.
Native debug symbols (.so) upload: confirm a task actually uploads them for the release AAB; if RN's sentry.gradle does not cover it, wire the Sentry Android Gradle plugin / sentry-cli debug-files upload in CI.
JS anonymous frames: confirm the uploaded source map's Debug ID matches the shipped bundle's Debug ID for the Play build (not just local builds).

Scope

Audit with evidence from the actual publish-run logs (Actions API) and the gradle/SDK config; state which link in the chain is broken before changing anything.
Fix the broken link(s): config/CI only unless a small src/lib/sentry.ts change (e.g. dist alignment) is proven necessary.
Make symbolication failures LOUD: publish workflow must fail if source-map, mapping, or symbol upload fails or is skipped for the production build.
Add a verifiable check: after the fix, a CI step proves the uploaded artifacts exist for the release (e.g. sentry-cli releases files list / debug-files check) — record the proof on this issue.

Hard limits

applicationId cc.agentlabs.opencode never changes.
No dependency upgrades (work with @sentry/react-native 6.22 / bundled sentry-cli as-is).
No release/tag in this task: merge to main only; the fix takes effect on the next release, which the daily sentry-fix loop or Den triggers.
No speculative crash fixes — this issue is symbolication only.

Claim — Muse, task lead (OpenCode Mobile board), working issue #232 under dev_workflow.

Scope: audit the Sentry symbolication chain against the v0.4.17 publish run (Actions API logs) and repo gradle/SDK config; post the audit verdict here before implementing; fix only the proven-broken link(s); make upload failures fail the publish workflow; add a CI verification step proving uploaded artifacts exist for the release. Config/CI only unless a small src/lib/sentry.ts change is proven necessary.

Hard limits honored: applicationId cc.agentlabs.opencode unchanged; no dependency upgrades; merge to main only — NO tag, NO release.

Stage: 0 claim → audit next.

Audit verdict (Stage 1/2 — investigation track) — read before implementing

Evidence: publish run 42026918221 (tag v0.4.17, head 0bbfa70, 2026-11-05, conclusion success), full job log; the retained AAB artifact itself; config on main vs tag v0.4.17 (byte-identical for all audited files). Sentry API unreachable from this VM (token in Bitwarden, no bw here) — server-side artifact state unproven where noted. Design gap recorded: no verification step exists anywhere in the pipeline, and native-symbol upload was never wired — absent uploads ship green.

Suspect 1 — upload tasks in the v0.4.17 run: MIXED

JS source maps: RAN and SUCCEEDED. Gradle task createBundleReleaseJsAndAssets_SentryUpload_opencode-mobile@0.4.17_0.4.17 uploaded bundle + map, release opencode-mobile@0.4.17, dist 0.4.17, debug id 94db72d2-3545-42e8-b9c3-da4281336440. (Server replied "processing pending"; nothing ever re-checked.)
ProGuard/R8 mapping: never ran — no such task exists. Root cause under Suspect 2.
Native debug symbols: never uploaded. Root cause under Suspect 3.
The run concluded success and published: the absences are silent by construction.

Suspect 2 — native dist mismatch: REAL in config, MOOT for v0.4.17

Uploads pin dist = versionName (SENTRY_DIST=0.4.17); native events carry dist = versionCode (155 for this build — the run bumped 55+100). SDK defaults confirm the split (sentry.gradle extractReleasesInfo defaults dist to versionCode; RNSentryModuleImpl.fetchNativeRelease() returns versionCode).
But minify is OFF (enableMinifyInReleaseBuilds defaults false, never set in gradle.properties; release minifyEnabled follows it). No R8 run, no mapping.txt exists (AAB has none), nothing to associate at any dist. Java frames are not obfuscated, so B/C's framework-only stacks are consistent with genuine RN-framework crashes carrying no app Java frames — a missing mapping is not what hides them.
No fix for this suspect. If minify is ever enabled, the mapping must be uploaded UUID-associated (Sentry Android Gradle Plugin) or with a versionCode dist — never the current versionName pin. Recorded here so it does not dangle.
Not changing src/lib/sentry.ts: JS release/dist already match the JS uploads exactly, and JS symbolication is Debug-ID-based. (One residual ambiguity — whether JS forwards its dist into native init — needs one B/C event's raw JSON from Sentry; not needed for this fix.)

Suspect 3 — native debug symbols (.so): BROKEN — proven

Nothing uploads them to Sentry. sentry.gradle creates only the JS upload task + SentryCollectModules; no Sentry Android Gradle Plugin anywhere; no sentry-cli debug-files step in publish-play-store.yml or build.yml. The AAB's 52 .so.sym files go to Google Play, not Sentry. Sentry holds no debug files for these ELF build IDs, so true native frames cannot be symbolicated.

Suspect 4 — JS Debug ID mismatch: DISPROVEN

Downloaded the retained AAB (the exact file uploaded to Play): its bundle contains sentry-dbid-94db72d2-3545-42e8-b9c3-da4281336440 — identical to the uploaded source map's Debug ID. OPENCODE-MOBILE-A's anonymous frames are not a Debug ID mismatch. Unproven only: whether the uploaded artifact bundle finished server-side processing, and event A's own debug_meta (needs the Sentry API).

Would an upload failure fail the workflow today?

A hard JS-upload failure would (plain ./gradlew bundleRelease, no continue-on-error / || true). A skipped or absent upload cannot fail anything: the upload task is onlyIf-gated, mapping/.so uploads have no task at all, and no verification step exists (verify-release-bundle.mjs only greps the AAB noise gate).

Fix scope (only the proven-broken links)

publish-play-store.yml: after Build AAB, upload native debug symbols via bundled sentry-cli debug-files upload over the unstripped merged native libs (association by ELF build ID — dist-independent). Step must fail the job on error.
publish-play-store.yml: add a verification step (no continue-on-error) proving server-side artifacts for the release: source bundle+map listed for opencode-mobile@<version>, and native debug files present for the project. This is what makes every silent-skip mode loud.
scripts/verify-release-bundle.mjs: extend (without altering the existing noise-gate checks) to cross-check the AAB bundle's sentry-dbid-… against the built source map's Debug ID; fail on mismatch.
No change: JS upload path, src/lib/sentry.ts, ProGuard/dist config.

Lane: simple fix (CI/tooling, no contract change). Stage 3 implementation next.

Context pack — Stage 0 claim: comment 6717155701. Stage 1/2: this verdict. Owner: Muse, task lead. Next action: Stage 3 implementation subagent (branch + PR, no merge by implementer).

Fix shipped (Stage 3) — PR #233 (#233), branch fix/issue-232-sentry-symbolication, head bbd1d2b18bff16966f70ef82021d7a1d35d56453 (verified via API readback), base main aebc4efd. 2 files: .github/workflows/publish-play-store.yml (+81/−1) and scripts/verify-release-bundle.mjs. No test files added (test-count delta 0).

Context pack update — Current stage: 4 (review). Owner: Muse, task lead. Completed: Stage 0 claim (6717155701), Stage 1/2 audit verdict (6717204966), Stage 3 PR #233. Next action: Stage 4 independent review per claude-code-review; merge only on a read-back PASS verdict. Residual (unproven until next publish): live server-side Sentry checks — no valid token exists on this VM; the new CI verification step is the proof mechanism at the next release.

Stage 4 review: FAIL (verdict on PR #233, comment 6717301647; head reviewed bbd1d2b). One validated blocking finding: in "Verify Sentry artifacts for release", grep -q "index.android.bundle" is subsumed by the map filename — with only index.android.bundle.map listed, the bundle check passes, so the step would ship green in exactly the partial-upload state it exists to catch. Criteria 1, 3, 4, 5, 6 PASS; test-count delta 0. Fix round dispatched: correct the bundle match (must exclude the .map entry) + fixture re-verification, then re-review.

Fix Sentry bundle check to exclude source map line (issue #232)

Stage 4 re-review: FAIL (PR #233 comments 6717343921 + addendum 6717352721; head 55d955b). The round-1 grep fix is wrong against the REAL sentry-cli 2.53.0 output: releases files list prints a bordered table with path-prefixed names (~/index.android.bundle), not one bare filename per line. Proven with the real binary + mock API using the step's exact check block: bundle+map listed → exit 1 "JS bundle not listed" (false failure — would break every publish). Map-only / bundle-only / empty all exit 1.

Diagnosis (anomaly rule): both review rounds trace to one root cause — the check was written against an assumed list format; the real table format was only exercised in re-review. Fix round 2 must validate against the real binary + mock release-files API, all four cases, with the step's exact commands. Criteria 1/3/4/5 and test-count delta 0 remain PASS.

fix(ci): token-exact Sentry JS bundle check for real sentry-cli table…

Fix round 2 landed (PR #233 head a89a27b, comment 6717386759): bundle check is now token-exact (([^.]|$)), proven against the real sentry-cli 2.53.0 + mock API for bundle+map / map-only / bundle-only / empty. One residual proven by the fix agent, accepted as in-scope: the releases files list table has a Source Map column that renders the bundle artifact's Sourcemap header verbatim — production bundle artifacts carry that header, so the whole-output substring map check can pass when no map artifact was uploaded (bundle-only payload + header → exit 0 in the supplementary run). Fix round 3: extract the Name column from the table and exact-match artifact names against it; the header column then cannot false-match. Harness must add case (c2): bundle-only WITH the Sourcemap header → exit 1.

fix(ci): verify Sentry source map by exact artifact name, not table s…

Merge PR #233: Sentry native symbol upload + release artifact verific…

Close-out — MERGED (Stage 5 + 5.5)

PR fix(ci): Sentry native symbol upload + release artifact verification (#232) #233 merged: merge commit 4a23a05, read back as main HEAD; parents aebc4ef + reviewed head 99cd1d6. No tag created (latest tag remains v0.4.17), no release — the fix takes effect at the next publish, per the task limits.
Stage 4: PASS on head 99cd1d6 (PR comment 6717447516), after two FAIL rounds that caught real defects in the verification step (substring grep; then the real sentry-cli table format + the Source Map header column false-pass). Test-count delta: 0 (43 test files, identical sets). CI on the merged head: Android build, iOS CI, gitleaks — all success.
What shipped (2 files): publish-play-store.yml — native .so debug-files upload step after Build AAB (hard-fail), plus a "Verify Sentry artifacts for release" step proving server-side presence: bundle + source map listed for opencode-mobile@<version> (Name-column exact match) and native debug files present via the dsyms API — any absence fails the publish. scripts/verify-release-bundle.mjs — Debug ID cross-check (AAB sentry-dbid-… vs built source map), existing checks untouched.
CI verification proof (the proof the issue asked to record): the verification logic was exercised end-to-end against the real bundled sentry-cli 2.53.0 + a mock release-files API by both the fix agents and the independent reviewer — bundle+map → exit 0; map-only / bundle-only / bundle-only-with-Sourcemap-header / empty / dsyms-404 / dsyms-empty / CLI-500 → exit 1. The live proof runs in the publish workflow itself at the next release: that step IS the check, and it fails the publish if artifacts are missing. Server-side state for past releases remains unverifiable from this VM (Sentry token lives in Bitwarden; no client here) — recorded as the standing residual, not a blocker to this fix.
Stage 5.5 Design/PRD sync: checked — no update needed. This project has no Notion PRD/Design doc (verified 2026-11-01); the change is CI/tooling only and alters no product behavior, contract, or documented surface.
Lane: standard (CI/tooling) — closes at merge per the 2026-11-08 Stage 6 scoping; no Stage 6 gate. Remaining residuals, owned elsewhere: the crashes themselves (OPENCODE-MOBILE-A/-B/-C) stay backlog until a symbolicated backtrace maps them to code; the conditional ProGuard note is recorded in the audit verdict (6717204966) for if minify is ever enabled.

Symbolication verdict at v0.4.18 — NOT proven (publish run 43565249322 FAILED at 'Verify Sentry artifacts for release').

What succeeded in the run:

Build AAB: BUILD SUCCESSFUL in 20m 13s.
Gradle sourcemap upload for release opencode-mobile@0.4.18, dist 0.4.18: 'Uploaded files to Sentry', bundle + source map, debug id e602adf5-6615-4c44-b5a6-9dcab755a199.
Native debug symbols: 88 .so files uploaded ('Uploaded 88 missing debug information files').
Noise-gate check on the shipped bundle: passed.

What failed:

'Verify Sentry artifacts for release' (06:34:33Z): sentry-cli releases files list for opencode-mobile@0.4.18 returned 'error: Release not found. Ensure that you configured the correct release, project, and organization.'
Consequence: the AAB never reached Play (Publish to Play Store step never ran, rollout step skipped). Play production is still v0.4.17. GitHub release v0.4.18 exists with both APKs (build run 43565249363 success).

Diagnosis note: upload and verify agree on release name (opencode-mobile@0.4.18), org (vibetechnologies, via android/sentry.properties + repo secret) and project (opencode-mobile) — the plugin reads SENTRY_RELEASE/SENTRY_DIST env vars (sentry.gradle lines 338-340) and the workflow set them. Leading suspicion: the deprecated 'releases files list' query is incompatible with the artifact-bundle upload the plugin now performs ('Upload type: artifact bundle'), so the release lookup 404s even though the upload reported success. Needs a diagnosis task before any re-publish — re-running the same workflow would fail identically.

Claim + diagnosis — Play publish blocked on v0.4.18 verify step (follow-up to #232)

Claim: I am the owning lead for this follow-up. Work Log row 418ac25e-b49f-8128-a1de-e0f1c8e3e049 (In progress). Branch: fix/sentry-verify-artifact-bundle-232. Nobody-else check: no open PR touches the publish workflow verify step (open: #234, #215, #211, #200, #199, #196, #195, #192, #191, #185, #182, #161).

Diagnosis (confirmed from the full log of run 43565249322)

Upload succeeded — as an artifact bundle, with no Release object created:

Gradle task createBundleReleaseJsAndAssets_SentryUpload_opencode-mobile@0.4.18_0.4.18 ran sentry-cli react-native gradle --release opencode-mobile@0.4.18 --dist 0.4.18 → Uploaded files to Sentry, Upload type: artifact bundle, Bundle ID 19297063-1541-58db-9866-a9b97df2b49a. Report lists ~/index.android.bundle + ~/index.android.bundle.map (debug id e602adf5-6615-4c44-b5a6-9dcab755a199).
Native: 88 .so debug files uploaded (Uploaded 88 missing debug information files, sentry-cli 2.53.0).
The full 3722-line run log contains zero releases new / releases finalize / set-commits / Created release: nothing in the workflow ever creates a Sentry Release object.

Verify failed:

Step Verify Sentry artifacts for release (06:34:33Z): sentry-cli releases files list printed the deprecation notice, then error: Release not found. Ensure that you configured the correct release, project, and organization.

Root cause: the upload path (artifact bundle via sentry-cli react-native gradle) never creates a Sentry Release object, while the verify step queries the legacy release-files endpoint, which 404s when the release does not exist. Refinement of the earlier suspicion: the 404 is about the missing Release object, not about file visibility — the endpoint never gets to list files. Org / project / token are proven correct by the successful uploads using the same credentials.

Design gap: PR #233's verify step assumed a Release object that the artifact-bundle upload path never creates.

Note on live reproduction: no Sentry token is available in this session (bws reports "Missing access token" — the token is Den-held via Bitwarden), so the live-API reproduction happens in CI: the re-triggered publish run executes the fixed verify step with the repo secret. That run is the live proof.

Fix direction

Verify via a query that works with artifact-bundle uploads — candidates: create the release before listing (releases new + list), the artifact-bundle API, or sourcemaps explain by debug id. Implementation will pick the query from sentry-cli 2.53.0 help + Sentry API docs and prove it live on the re-triggered publish. The gate must still hard-fail when artifacts are missing — no weakening.

Context pack

Current stage: diagnosis done → Stage 3 implementation (PR on fix/sentry-verify-artifact-bundle-232)
Issue: Fix Sentry symbolication: native/Java backtraces framework-only, JS frames anonymous on v0.4.17 #232 (closed; verdict comment 6873226348 already records NOT proven at v0.4.18)
Work Log row: 418ac25e-b49f-8128-a1de-e0f1c8e3e049
Acceptance: verify step fixed via PR (Stage 4 PASS, CI green, merge SHA read back) → publish re-triggered for the EXISTING v0.4.18 tag (no re-tag) → publish run success + AAB in Play + rollout started → symbolication verdict recorded here
Open residuals: none yet; the query choice is Stage 3's decision, proven by the re-triggered publish run
Exact next action: dispatch Stage 3 implementation subagent

Merge pull request #245 from dzianisv/fix/sentry-verify-artifact-bund…

Merge — PR #245 (Stage 5 + 5.5)

PR #245 merged: merge commit e7385a349ef028bb0fd5938c71ec2a8820d85306, now main HEAD; parents b5bbf1a (v0.4.18 tag) + 42ad8b28 (reviewed head). Stage 4 verdict comment 6873805421 (PASS, head 42ad8b28). CI green on the head: typecheck, test, build (19m46s), iOS simulator build, gitleaks.

Stage 5.5 — Design/PRD sync: checked, no update needed. The Fix bugs runbook promises only that the verify step FAILS the publish when the JS bundle, its source map, or native debug files are missing in Sentry for the release. The merged change keeps that promise exactly (same gate, different query: artifact-bundle API instead of the legacy release-files endpoint). No design doc names the old query mechanism, and the PRD is untouched.

Next: re-trigger the publish for the existing v0.4.18 tag via workflow_dispatch (production / inProgress / 0.20 — the tag path's rollout; a re-run of 43565249322 would reuse the old workflow file). No new tag. Verdict on symbolication follows from that run.

Symbolication verdict at v0.4.18 — PROVEN (publish path)

Publish run 43572233118 (workflow_dispatch on main e7385a34, production / inProgress / 0.20, versionCode 158): publish job SUCCESS — every step green, including the fixed Verify Sentry artifacts for release.

Live proof of the new query (08:00:07Z): matched artifact bundle 574ab16e-27ef-5f0e-9035-8b726c1a1b4f (fileCount=2, date 2026-11-14T07:45:31Z — the fresh upload from this run), found ~/index.android.bundle (size=0, debugId d9cbfded-5899-48bf-9d57-a6f82d05c118) and ~/index.android.bundle.map (size 14,442,822, same debugId). Shared debug id = the pair can symbolicate. Native dsyms: 88 .so files uploaded in the same run (step green).

Play: AAB uploaded, edit 17651488377991504144 committed to production at 20% (inProgress). Rollout line recorded on the GitHub release body (v0.4.18): Rollout: fraction 0.20 — versionCode 158 — status inProgress.

Notes:

The run-level conclusion reads "failure" only because the non-blocking cua-mid-stream job failed (Den disabled it as a publish gate 2026-11-06; it still runs on dispatch and fails for lack of credits). The publish job itself is green.
The bundle file entry reports size=0 — Sentry reports the bundle container entry with size 0; the map is 14.4 MB. The Stage 4 filed note about asserting fileSize > 0 does not apply to the bundle entry; a follow-up could assert the map's size > 0.
Residual (unchanged from the original Fix Sentry symbolication: native/Java backtraces framework-only, JS frames anonymous on v0.4.17 #232 acceptance): the FIRST crash report on v0.4.18 should carry app/native frames — confirmed only when a real crash arrives. The upload/verify side is now proven; the end-to-end frame proof awaits the first post-release crash (3-day Sentry watch per Staged Play rollout for future releases (follow-up to #227) #230).

Work Log row 418ac25e-b49f-8128-a1de-e0f1c8e3e049 → Done. Fix: PR #245, merge e7385a34.

Navigation Menu

Repository navigation

Fix Sentry symbolication: native/Java backtraces framework-only, JS frames anonymous on v0.4.17 #232

Problem (verified 2026-11-08)

What is already wired (verified in repo, do not redo)

Prime suspects to audit first (kill-source: the v0.4.17 publish run's own logs)

Scope

Hard limits

Audit verdict (Stage 1/2 — investigation track) — read before implementing

Suspect 1 — upload tasks in the v0.4.17 run: MIXED

Suspect 2 — native dist mismatch: REAL in config, MOOT for v0.4.17

Suspect 3 — native debug symbols (.so): BROKEN — proven

Suspect 4 — JS Debug ID mismatch: DISPROVEN

Would an upload failure fail the workflow today?

Fix scope (only the proven-broken links)

Close-out — MERGED (Stage 5 + 5.5)

Claim + diagnosis — Play publish blocked on v0.4.18 verify step (follow-up to #232)

Diagnosis (confirmed from the full log of run 43565249322)

Fix direction

Context pack

Merge — PR #245 (Stage 5 + 5.5)

Symbolication verdict at v0.4.18 — PROVEN (publish path)

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Fix Sentry symbolication: native/Java backtraces framework-only, JS frames anonymous on v0.4.17 #232

Description

Problem (verified 2026-11-08)

What is already wired (verified in repo, do not redo)

Prime suspects to audit first (kill-source: the v0.4.17 publish run's own logs)

Scope

Hard limits

Activity

dzianisv commented on Nov 8, 2026

dzianisv commented on Nov 8, 2026

Audit verdict (Stage 1/2 — investigation track) — read before implementing

Suspect 1 — upload tasks in the v0.4.17 run: MIXED

Suspect 2 — native dist mismatch: REAL in config, MOOT for v0.4.17

Suspect 3 — native debug symbols (.so): BROKEN — proven

Suspect 4 — JS Debug ID mismatch: DISPROVEN

Would an upload failure fail the workflow today?

Fix scope (only the proven-broken links)

dzianisv commented on Nov 8, 2026

dzianisv commented on Nov 8, 2026

dzianisv commented on Nov 8, 2026

dzianisv commented on Nov 8, 2026

dzianisv commented on Nov 8, 2026

Close-out — MERGED (Stage 5 + 5.5)

dzianisv commented on Nov 14, 2026

dzianisv commented on Nov 14, 2026

Claim + diagnosis — Play publish blocked on v0.4.18 verify step (follow-up to #232)

Diagnosis (confirmed from the full log of run 43565249322)

Fix direction

Context pack

dzianisv commented on Nov 14, 2026

Merge — PR #245 (Stage 5 + 5.5)

dzianisv commented on Nov 14, 2026

Symbolication verdict at v0.4.18 — PROVEN (publish path)

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions