Skip to content

Conversation

@antonis
Copy link
Contributor

@antonis antonis commented Dec 15, 2025

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

Based on: #5470

📜 Description

This PR migrates tests that were originally created in Detox to Maestro to unify all the E2E tests.

💡 Motivation and Context

This is a followup to #4914 migrating the implementation to v7

💚 How did you test it?

CI, Manual

📝 Checklist

  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • All tests passing
  • No breaking changes

🔮 Next steps

#skip-changelog

krystofwoldrich and others added 4 commits June 11, 2025 20:15
…gration

- Resolved conflicts in workflow files, keeping Maestro tests
- Updated dependency versions to match RN 0.80.2
- Merged sample app changes with new tab structure
- Kept Detox test scripts for compatibility while maintaining Maestro migration
- Updated gradle and xcode configurations
@antonis antonis added the ready-to-merge Triggers the full CI test suite label Dec 15, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 15, 2025

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 401.33 ms 420.54 ms 19.22 ms
Size 43.94 MiB 48.91 MiB 4.97 MiB

Baseline results on branch: capture-app-start-errors-v7

Startup times

Revision Plain With Sentry Diff
99179a4+dirty 357.38 ms 375.64 ms 18.26 ms
60d1e83+dirty 406.16 ms 422.83 ms 16.67 ms
f26d7a8+dirty 380.15 ms 407.84 ms 27.68 ms

App size

Revision Plain With Sentry Diff
99179a4+dirty 43.94 MiB 48.90 MiB 4.96 MiB
60d1e83+dirty 43.94 MiB 48.90 MiB 4.96 MiB
f26d7a8+dirty 43.94 MiB 48.90 MiB 4.96 MiB

Previous results on branch: antonis/v7-detox-to-maestro-migration

Startup times

Revision Plain With Sentry Diff
16753e9+dirty 459.78 ms 503.98 ms 44.20 ms

App size

Revision Plain With Sentry Diff
16753e9+dirty 43.94 MiB 48.90 MiB 4.96 MiB

@github-actions
Copy link
Contributor

github-actions bot commented Dec 15, 2025

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 490.18 ms 508.02 ms 17.84 ms
Size 43.75 MiB 48.08 MiB 4.33 MiB

Baseline results on branch: capture-app-start-errors-v7

Startup times

Revision Plain With Sentry Diff
99179a4+dirty 405.13 ms 415.16 ms 10.03 ms
60d1e83+dirty 396.72 ms 390.04 ms -6.68 ms
f26d7a8+dirty 571.36 ms 637.92 ms 66.56 ms

App size

Revision Plain With Sentry Diff
99179a4+dirty 43.75 MiB 48.08 MiB 4.32 MiB
60d1e83+dirty 43.75 MiB 48.08 MiB 4.32 MiB
f26d7a8+dirty 43.75 MiB 48.08 MiB 4.32 MiB

Previous results on branch: antonis/v7-detox-to-maestro-migration

Startup times

Revision Plain With Sentry Diff
16753e9+dirty 401.16 ms 433.20 ms 32.04 ms

App size

Revision Plain With Sentry Diff
16753e9+dirty 43.75 MiB 48.08 MiB 4.32 MiB

@github-actions
Copy link
Contributor

github-actions bot commented Dec 15, 2025

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1225.42 ms 1226.08 ms 0.66 ms
Size 3.44 MiB 4.59 MiB 1.15 MiB

Baseline results on branch: capture-app-start-errors-v7

Startup times

Revision Plain With Sentry Diff
60d1e83+dirty 1207.79 ms 1207.35 ms -0.44 ms
99179a4+dirty 1213.96 ms 1215.34 ms 1.38 ms
f26d7a8+dirty 1209.49 ms 1207.54 ms -1.95 ms

App size

Revision Plain With Sentry Diff
60d1e83+dirty 3.41 MiB 4.67 MiB 1.26 MiB
99179a4+dirty 3.44 MiB 4.67 MiB 1.23 MiB
f26d7a8+dirty 3.44 MiB 4.67 MiB 1.23 MiB

Previous results on branch: antonis/v7-detox-to-maestro-migration

Startup times

Revision Plain With Sentry Diff
16753e9+dirty 1214.38 ms 1222.09 ms 7.71 ms

App size

Revision Plain With Sentry Diff
16753e9+dirty 3.44 MiB 4.67 MiB 1.23 MiB

@github-actions
Copy link
Contributor

github-actions bot commented Dec 15, 2025

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1207.83 ms 1212.93 ms 5.10 ms
Size 3.44 MiB 4.59 MiB 1.15 MiB

Baseline results on branch: capture-app-start-errors-v7

Startup times

Revision Plain With Sentry Diff
60d1e83+dirty 1201.87 ms 1204.61 ms 2.74 ms
99179a4+dirty 1210.16 ms 1213.57 ms 3.40 ms
f26d7a8+dirty 1227.33 ms 1220.67 ms -6.66 ms

App size

Revision Plain With Sentry Diff
60d1e83+dirty 3.41 MiB 4.67 MiB 1.26 MiB
99179a4+dirty 3.44 MiB 4.67 MiB 1.23 MiB
f26d7a8+dirty 3.44 MiB 4.67 MiB 1.23 MiB

Previous results on branch: antonis/v7-detox-to-maestro-migration

Startup times

Revision Plain With Sentry Diff
16753e9+dirty 1200.13 ms 1202.62 ms 2.49 ms

App size

Revision Plain With Sentry Diff
16753e9+dirty 3.44 MiB 4.67 MiB 1.23 MiB

The set-dsn scripts were moved from scripts/detox/ to scripts/ during the migration
…NewsScreen

- Add scrollUntilVisible for 'Capture message' button (might be off-screen)
- Add proper waiting and scrolling for SpaceflightNewsScreen to trigger auto-load
- Wait for 'Load More Articles' button to appear after autoLoadCount threshold
@antonis
Copy link
Contributor Author

antonis commented Dec 19, 2025

The captureReplay e2e test failures on iOS are not related with this PR that touches only the sample app E2E tests and will be handled separately in a PR from the base branch.

@antonis antonis marked this pull request as ready for review December 19, 2025 12:08
@antonis antonis changed the title chore(sample-e2e): v7: Migrate from Detox to Maestro ref(sample-e2e): v7: Migrate from Detox to Maestro Dec 19, 2025
@antonis antonis merged commit 33d1129 into capture-app-start-errors-v7 Dec 22, 2025
85 of 88 checks passed
@antonis antonis deleted the antonis/v7-detox-to-maestro-migration branch December 22, 2025 13:00
antonis added a commit that referenced this pull request Jan 29, 2026
* ref(ios): Extract Cocoa SDK init into standalone file (#4442)

* ref(android): Extracts Android native initialization to standalone structures (#4445)

* Extract Android SDK Init

* Update tests

* Adds changelog

* Fix lint issues

* Rename RNSentryStart instance for clarity

* Converts RNSentryStart to utility class

* Update CHANGELOG.md

---------

Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>

* feat(experimental): Add native `startWithConfigureOptions` for Apple platforms (#4444)

* feat: Read `sentry.options.json` during cocoa init (#4447)

* Adds utility class for converting `JsonObject` to `WritableMap` (#4479)

* Convert json object to writable map

* Make class/methods package-private(default)

* feat: Automatically load `sentry.options.json` file (#4476)

* feat(experimental): Initialize Android SDK from json configuration (#4451)

* misc: Add `sentry.options.json` example to the changelog (#4509)

* feat(init): Load options from `sentry.options.json` in JS (#4510)

* release: 6.7.0-alpha.0

* misc(sample): Change RN Sample to use native file init by default (#4522)

* chore(sample-rn): Remove duplicate init options from code (#4532)

* chore(sample-rn): Always use fhe file option (including auto init) (#4533)

* internal(sample-rn): Add Detox for integration/e2e tests of the rn sample (#4535)

* internal(sample-rn): Add header and message envelope tests (#4536)

* fix(sample-e2e): Fix type errors missing sentry/core and afterAll (#4564)

* chore(samples): Add package scripts for native builds, dsn and testing (#4561)

* test(e2e): Verify captured Errors Screen transaction (#4584)

* test(e2e): Add auto init from JS tests (#4588)

* test(e2e): Add app start crash test for iOS (#4593)

* test(e2e): Avoid race conditions when waiting for captured message (#4595)

* chore(sample-e2e): Move Detox related files to e2e-detox dir

* fix: remove unused SentryPackage import

Removed unused import that was causing CI lint failure

* fix: update RNSentryStartTest for Sentry Android SDK v7 API changes

- Removed assertions for packages getter (not available in v7)
- Removed assertion for enableTracing property (removed in v7)
- Added comments explaining the API changes

* fix: remove unused addPackages method to fix PMD lint error

The method was a no-op after v7 API changes, so removing it entirely
to avoid unused parameter warnings from PMD.

* fix: update RNSentrySDKTest for Sentry Android SDK v7 API changes

- Removed assertions for enableTracing property (removed in v7)
- Removed assertions for packages getter (not available in v7)
- Added comments explaining the API changes

* fix: use relative path for RNSentrySDK+Test.h import in bridging header

The file is in the parent directory, so use ../ prefix to fix the import path

* fix: remove deprecated enableTracing property in iOS RNSentryStart

The property is deprecated in v7. Tracing is already disabled by setting
tracesSampleRate and tracesSampler to nil.

* fix: remove enableTracing assertions from iOS tests

The enableTracing property is deprecated in Sentry Cocoa SDK v7.
Tracing is already verified to be disabled by checking that
tracesSampleRate and tracesSampler are nil.

* Update Podspec

* Fix lint issue

* ref(sample-e2e): v7: Migrate from Detox to Maestro (#5473)

* chore(sample-e2e): Migrate from Detox to Maestro

* fix set dsn script path

* fix: Update script paths after detox-to-maestro migration

The set-dsn scripts were moved from scripts/detox/ to scripts/ during the migration

* Update script paths

* Fix tests

* Fix test failure

* Fix idle issue

* fix(e2e): Fix Maestro flows for captureMessage and captureSpaceflightNewsScreen

- Add scrollUntilVisible for 'Capture message' button (might be off-screen)
- Add proper waiting and scrolling for SpaceflightNewsScreen to trigger auto-load
- Wait for 'Load More Articles' button to appear after autoLoadCount threshold

* chore(e2e-sample): Increase Maestro driver startup timeout

(cherry picked from commit ee429b5)

* increase timeout

* Increase timeouts on Android too

---------

Co-authored-by: Krystof Woldrich <krystof.woldrich@sentry.io>

* chore: Merge Android UI profiling on the capture startup crashes branch (#5544)

* chore: Merge Android UI profiling on the capture startup crashes branch

* Fix logger compilation issue

* Properly check logging values

* fix SR iOS issue (#5560)

* Remove duplicate changelog entry

* feat(expo): Add RNSentrySDK APIs support to @sentry/react-native/expo plugin (#4633)

* useNativeInit Android implementation

* Adds changelog

* useNativeInit iOS implementation

* Fix indentation

* Extend test cases with realistic data

* Adds code sample in the changelog

* Fix CHANGELOG.md

Co-authored-by: LucasZF <lucas-zimerman1@hotmail.com>

* Warn if RESentySDK.init/start wasn't injected

* Make useNativeInit opt-in

* Make Android failure warning more clear

Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>

* Make Android no update warning more clear

Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>

* Use path.basename to get last path component

* Update tests to account for the new warnings

* Explicitly check for kotlin

* Add filename in the warning message

* Import only if init injection succeeds

* Explicitly check for Objective-C

* Add filename in the warning

* Make iOS file not found warning more clear

* Import only if init injection succeeds

* Reset test mock config in a function

* Lint issue

* Add missing quote

Co-authored-by: LucasZF <lucas-zimerman1@hotmail.com>

* Remove unneeded async

Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>

* Set useNativeInit = false by default

* dynamically fill white spaces

* Add unsupported language in warning message

* Add objcpp in detected languages

Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>

* Update tests for objcpp

* ref(expo-plugin): Split utils to logger, version and utils (#4906)

Co-authored-by: Antonis Lilis <antonis.lilis@gmail.com>

* Update changelog

* fix(ios): Add Swift module support for RNSentrySDK native init

Fixes Swift compilation errors when using the useNativeInit Expo plugin feature.

Changes:
- Updated RNSentry.h to use angle bracket import for RNSentrySDK, properly
  exposing it through the module system
- Added DEFINES_MODULE to RNSentry.podspec to enable Swift module generation
- Fixed Expo plugin to insert import after first import statement (supports
  modern Expo AppDelegate structure without UIKit import)

This enables Swift code to successfully import RNSentry and call
RNSentrySDK.start() when using native initialization.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix test

* Update changelog

* Fix native tests

* Fix lint issue

* Fix native tests

* Revert unneeded changes

* Fix sample app build

---------

Co-authored-by: LucasZF <lucas-zimerman1@hotmail.com>
Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix native ios tests

* Mark area

Co-authored-by: LucasZF <lucas-zimerman1@hotmail.com>

* fix(android): Fix ConcurrentModificationException (#5588)

* fix(android): Fix ConcurrentModificationException when disabling native crash handling

When enableNativeCrashHandling is set to false, the code was iterating over
the integrations list with a for-each loop while calling remove() directly,
which causes a ConcurrentModificationException at runtime.

Fixed by using Java 8's removeIf() method which safely handles iteration
and removal in a single operation. This is more concise and follows modern
Java best practices.

Added unit tests to verify the fix and ensure integrations are properly
removed without throwing exceptions.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Lint fix

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(android): Fix crash when spotlight is enabled without defaultSidecarUrl

The code attempted to read defaultSidecarUrl without checking if the key
exists in the options map. This caused a NoSuchKeyException crash during
startup when spotlight was set to true in sentry.options.json without
providing defaultSidecarUrl.

Added key existence check to match iOS implementation behavior and prevent
the crash while maintaining backward compatibility.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* test(e2e): Add auto init from JS tests for Android (#5583)

* test(e2e): Add auto init from JS tests for Android

Implements Android E2E testing infrastructure to verify both manual native initialization and auto initialization from JavaScript, matching the iOS implementation and resolving issue #4912.

Key additions:
- Jest configs for android.auto and android.manual test modes
- Build scripts that toggle SENTRY_DISABLE_NATIVE_START at compile time
- Test scripts to run auto and manual test suites separately
- App start crash testing via flag file mechanism
- TestControlModule to enable/disable crash-on-start from JS
- Comprehensive E2E test documentation

Unlike iOS which uses launch arguments at runtime, Android requires separate builds with different build configurations to control native initialization.

Closes #4912

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(e2e): Add scrolling to find crash control buttons in Android test

The crash control buttons are off-screen, so the Maestro flow needs to
scroll to find them before tapping. This matches the pattern used in
other Android E2E tests.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(e2e): Make Android crash flag auto-expire after one crash

The crash flag file was persisting across app launches, causing the app
to crash indefinitely. Now the flag auto-deletes when read, allowing:
1. First launch: Enable flag
2. Second launch: Read flag, delete it, then crash
3. Third launch: Start normally and send crash report

This solves the chicken-and-egg problem where the app couldn't reach
JavaScript to clear the flag because it kept crashing before JS loaded.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(e2e): Handle wrapped exceptions in Android crash test

Android wraps exceptions thrown in Application.onCreate() with:
"Unable to create application... RuntimeException: <original message>"

Updated the test to check if ANY exception in the chain contains our
intentional crash message, rather than expecting an exact match on the
first exception.

Test now passes locally and should pass in CI.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Clean up notes for now

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(android): Fix crash when dsn or devServerUrl are missing from options

Fixes a crash on Android startup when initializing from sentry.options.json
without dsn or devServerUrl fields. The code was calling getString() on
ReadableMap without checking if the keys exist first, which throws
NoSuchKeyException for missing keys.

Both fields are optional in configuration files, so the code now checks
for key existence before accessing values, returning null when keys are
missing. This matches the pattern used throughout the rest of the file
and is already handled correctly by the null-checks in the breadcrumb
filter logic.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>
Co-authored-by: Krystof Woldrich <krystof.woldrich@sentry.io>
Co-authored-by: getsentry-bot <bot@sentry.io>
Co-authored-by: getsentry-bot <bot@getsentry.com>
Co-authored-by: LucasZF <lucas-zimerman1@hotmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Triggers the full CI test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants