Skip to content

[Compatibility]: Restore Mac2 live readiness on Xcode 27.2 #79

Open

Description

@houlianpi

Status

Refining. This issue records a release-blocking compatibility gap. Do not implement until the Integration Lead selects the owned fix boundary and assigns one isolated worktree.

Problem

Independent clean native-macOS acceptance for #76 cannot reach Live Ready on the current supported host:

  • native macOS 26.7.3 arm64;
  • Xcode 27.2 (27B52);
  • Appium 3.7.0;
  • appium-mac2-driver 4.3.1;
  • OMP 18.18.3;
  • exact Candidate run 43127702644, attempt 1, artifact 13904340707, commit fdaac4168fe737195167f1d72c063191763c4e60.

The authenticated Candidate, offline dependency installation, package inventory, canonical Skill, Contract identity, OMP discovery, MCP initialize, and static Mac2 readiness all pass. WebDriverAgentMac builds and is locally signed, but the Apple-supplied XCTRunner host aborts before main with DYLD code 1 because @rpath/XCTest.framework does not resolve.

Xcode's generated test specification contains the correct platform developer framework paths in TestingEnvironmentVariables, but those paths are not effective for the LaunchServices-spawned UI-test host. Its loader searches bundle-relative and /System/Developer locations, not the active Xcode bundle. This is not a missing framework, network failure, demonstrated TCC denial, or demonstrated code-sign rejection.

User outcome

On this exact native macOS/Xcode target, the standard supported AutoGenesis Appium/Mac2 path starts WebDriverAgentMac and reaches Live Ready without copying or embedding Apple private test frameworks, modifying system paths, weakening installation security boundaries, or requiring users to patch generated runner binaries.

Evidence and eliminated hypotheses

  • Exact Candidate service digest and 185-file internal checksum closure passed.
  • Network-denied offline installation of all 84 locked distributions passed; metadata and pip check passed.
  • OMP discovered one project Skill, one stdio MCP server, 34 tools, three workflow prompts, and the matching Contract 2.1.0 identity.
  • Xcode built and spawned the runner; crash reports consistently showed fatalDyldError=1, EXC_CRASH/SIGABRT, DYLD Library missing, @rpath/XCTest.framework/Versions/A/XCTest before testmanagerd pairing.
  • Installed XCTest exists, includes arm64, and passes strict Apple signature validation.
  • External-network denial cannot cause the pre-main loader failure.
  • No process-specific TCC denial or privacy prompt occurred.
  • Ordinary and testing-environment DYLD path injection did not become effective for the host.
  • The Apple-prebuilt XCTRunner host has insufficient Mach-O header capacity for one additional rpath; WDA linker header padding cannot affect that prebuilt host.
  • A transaction-only external Frameworks symlink is rejected by bundle signing policy.
  • Embedding a 22-image Apple XCTest runtime closure was rejected as non-representative release acceptance and is out of scope.
  • Every bounded experiment failed closed and returned tracked DerivedData, system staging, diagnostic reports, processes, listeners, and Calculator state to baseline.

Detailed sanitized acceptance evidence is linked from #76.

Minimum sufficient design

Choose one supported, maintainable compatibility boundary that makes the unmodified user path work. Candidate directions to evaluate during refinement, not pre-approved implementation choices:

  1. adopt an upstream Mac2/Xcode combination with demonstrated Xcode 27.2 runtime launch support;
  2. contribute or consume an upstream Mac2/WebDriverAgentMac fix that preserves standard Apple test-host loading and signing;
  3. define an explicitly supported Xcode toolchain constraint if Xcode 27.2 cannot be supported safely.

A local acceptance-only binary patch, private-framework embedding, system /System/Developer mutation, TCC automation, or undocumented user workaround is not an acceptable product fix.

Scope

  • Reproduce the pre-main XCTest loader failure from an exact clean built artifact.
  • Select and document the owned compatibility boundary.
  • Implement the smallest standard-path fix or exact supported-toolchain constraint.
  • Add focused compatibility verification for WDA startup and Calculator Live Ready.
  • Update affected support/readiness documentation and release evidence.

Non-goals

  • No change to [Release]: Bundle exact dependency wheelhouses in Candidate artifacts #76 wheelhouse identity, dependency lock, Candidate checksum/provenance, or offline installation behavior.
  • No public release, protected-branch change, or support expansion beyond native macOS arm64.
  • No Apple private-framework redistribution or embedding.
  • No system-path mutation, TCC bypass, broad filesystem authority, or generated runner patch offered to users.
  • No claim that WDA startup alone proves Calculator, exact BDD, or recording acceptance.

Interfaces affected

To be frozen during refinement. Potentially the supported Appium/Mac2/Xcode compatibility matrix, package/toolchain pins, native macOS readiness checks, and Candidate acceptance environment. No public interface change is approved yet.

Acceptance scenarios

Scenario: Standard Mac2 path launches on the approved Xcode target
  Given a clean native macOS arm64 environment using the approved Xcode and Mac2 versions
  And AutoGenesis was installed only from the exact authenticated built Candidate
  When the standard project-scoped Appium/Mac2 configuration starts WebDriverAgentMac
  Then the unmodified signed runner pairs with testmanagerd
  And WDA exposes its loopback status endpoint
  And no Apple private framework is copied or embedded
  And no system path, TCC setting, or generated runner binary is modified

Scenario: Calculator reaches Live Ready
  Given the matching Contract identity and connected project-scoped MCP Runtime
  When one disposable Mac2 session targets system Calculator
  Then the intended Calculator window and bounded UI state are observed
  And the session, task-owned application, server, and attributable temporary state are cleaned

Scenario: Unsupported toolchain fails honestly
  Given the selected Xcode/Mac2 combination is outside the approved matrix
  When readiness evaluates the host
  Then it reports the exact unsupported constraint before starting WDA
  And it does not recommend private-framework embedding, binary patching, TCC bypass, or system-path mutation

Required verification

  • Clean built-artifact reproduction on native macOS arm64.
  • WDA /status, runner/testmanagerd pairing, unchanged standard runner identity/signing behavior, and no private-framework copying.
  • One real Calculator Live Ready session plus exact cleanup evidence.
  • Negative unsupported-toolchain behavior if the decision is a version constraint.
  • Independent correctness/security review covering process execution, filesystem scope, signing, downloads/dependency changes, TCC boundaries, and cleanup.
  • Separate clean-environment acceptor before unblocking [Release]: Bundle exact dependency wheelhouses in Candidate artifacts #76.

Security impact

High risk: process execution, code signing, Xcode helper filesystem behavior, loopback services, application automation, and privacy boundaries. The fix must not normalize private framework redistribution, system mutation, hidden TCC changes, broad unsandboxed writes, or acceptance-only patched binaries. Logs/evidence must remain sanitized and contain no local user paths or unrelated ambient content.

Rollback

Revert the selected dependency/toolchain/readiness change atomically. Restore the prior support matrix, keep #76 blocked, and do not cite a patched or private-framework-embedded run as acceptance evidence.

Dependencies and ownership

Definition of Ready gaps

  • Observed failure, target environment, security boundary, and acceptance outcome are explicit.
  • Non-goals and rejected unsafe workarounds are explicit.
  • Select upstream fix/dependency/toolchain-constraint direction.
  • Freeze affected interfaces and exact supported version matrix.
  • Assign implementation file ownership and clean verification environment.
  • Confirm whether a dependency update is authorized.

Activity

  1. added
    acceptance:pendingAcceptance evidence has not passed yet
    area:runtimeMCP runtime and automation execution behavior
    risk:highRequires explicit risk and rollback review
    on Nov 12, 2026
  2. added
    security-reviewIndependent security review is required
    type:compatibilityMCP client or platform compatibility qualification
    priority:p0Must be addressed before the next Windows Runtime compatibility release
    on Nov 12, 2026
  3. houlianpi commented on Nov 12, 2026

    @houlianpi
    Author

    Refinement evidence:\n\n- npm latest is Mac2 4.5.1; comparison from 4.3.1 contains no XCTest loader/host-runtime fix. Relevant changes are HTTP framing, session behavior, window APIs, and request-base-path handling.\n- Upstream Xcode 27 issue appium/appium-mac2-driver#410 was resolved by the 4.1.1 deployment-target build fix already present in our 4.3.1. That fix does not address the observed Xcode 27.2 pre-main XCTest loader failure.\n- Mac2 4.5.1 includes Xcode 27.0 in its CI matrix, but its functional workflow skips Accessibility seeding and IntegrationTests whenever SIP is enabled. Therefore the matrix proves install/doctor/build surfaces on hosted runners, not successful WDA runtime launch.\n- Upstream #410 records process-local Xcode 26.6 as the standard workaround without changing global Xcode selection.\n\nRecommended product decision: freeze AutoGenesis 1.1.1 Skills Preview native-macOS acceptance to a process-local Xcode 26.6 toolchain on the existing macOS arm64 host; keep global Xcode 27.2 untouched. Verify exact Xcode identity before Appium startup and fail readiness honestly outside the approved matrix. Treat Xcode 27.2 WDA launch support as deferred upstream compatibility work, not a local private-framework or runner-patching feature.\n\nProposed verification environment: current native macOS 26.7.3 arm64 host, OMP 18.18.3, Appium 3.7.0, Mac2 4.3.1, process-local DEVELOPER_DIR bound to authenticated Xcode 26.6, exact Candidate run/artifact from #76. Required proof remains WDA status, Calculator Live Ready, direct 2 + 3 = 5, exact BDD, read-only recording preview, and identity-bound cleanup.

  4. houlianpi commented on Nov 12, 2026

    @houlianpi
    Author

    Human Product Owner decision: wait for upstream standard-path Xcode 27.2 support. Do not download/install Xcode 26.6, change the global or process-local Xcode toolchain, upgrade Mac2 speculatively, patch or re-sign generated runners, or copy/embed Apple private XCTest runtime assets.\n\nFrozen compatibility interface while blocked:\n- current host remains native macOS 26.7.3 arm64 with Xcode 27.2;\n- Mac2 4.3.1 remains the accepted Candidate dependency until an authenticated upstream release explicitly addresses or is independently proven to close this runtime loader failure;\n- readiness must not report Xcode 27.2 Live Ready from build/doctor/CI-matrix evidence alone;\n- no support claim is added for Xcode 27.2 until clean built-artifact WDA launch, Calculator Live Ready, direct/BDD/read-only-recording journeys, cleanup, review, and separate acceptance pass.\n\nUnblock condition: an upstream Mac2 release or supported upstream fix provides a standard unmodified XCTest host path on Xcode 27.2. At that point, pin exact package integrity, update the Candidate lock atomically if authorized, verify on this same host, and resume #76. Blocker owner: upstream appium-mac2-driver compatibility.

  5. houlianpi commented on Nov 12, 2026

    @houlianpi
    Author

    Dependency status from PR #80: exact-head CI now passes Candidate wheelhouse acquisition and network-denied built-artifact installation on the GitHub-hosted macos-26 arm64 / CPython 3.11 environment, which satisfies #76’s macOS 15-or-newer package target. This does not satisfy #79: no Xcode 27.2 Mac2 Live Ready host, native Calculator/BDD/recording flow, or durable clean-host acceptance evidence was exercised. Per the Human Product Owner’s merge-first decision, #79 remains blocked/deferred and release remains prohibited.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    acceptance:pendingAcceptance evidence has not passed yetarea:runtimeMCP runtime and automation execution behaviorpriority:p0Must be addressed before the next Windows Runtime compatibility releaserisk:highRequires explicit risk and rollback reviewsecurity-reviewIndependent security review is requiredstatus:blockedWaiting on an external prerequisite or decisiontype:compatibilityMCP client or platform compatibility qualification

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions