IBKRGatewayManager

IBKRGatewayManager keeps an Interactive Brokers Gateway VM running for the QuantStrategyLab trading system. It handles the VM's deployment, remote sync, Docker rollout, and the 2FA watcher that keeps the Gateway logged in without an operator clicking through dialogs by hand. The InteractiveBrokersPlatform repository depends on this gateway being up and authenticated, but this repository never decides which strategy trades or submits an order itself.

Chinese README

Investing involves risk. This project does not provide investment advice and is for education, research, and engineering review only.

What this repository is

IBKRGatewayManager is a QuantStrategyLab IBKR gateway operations utility. It manages deployment, remote sync, Docker rollout, and 2FA watcher setup for the IBKR gateway VM.

It supports the system but does not decide which strategy should be live. Strategy eligibility remains in the strategy and snapshot repositories; broker execution remains in the platform repositories.

Design boundary

• Keep contracts stable and versioned where downstream repositories depend on them.
• Prefer backward-compatible changes unless a coordinated migration is planned.
• Keep secrets and environment-specific settings outside the shared library code.
• Document changes that affect multiple platforms or strategy packages.
• A compact, generic Gateway dialog never causes a click-through. If the API is unavailable, the watcher may perform one bounded container restart without acknowledging the dialog; an unresolved dialog still fails closed for operator review.

QSL architecture role

• Layer: runtime-ops.
• Responsibility: IBKR gateway VM lifecycle and 2FA operations utility.
• Owns: gateway deployment, remote sync, Docker rollout, watcher setup.
• Consumes: InteractiveBrokersPlatform operational requirements.
• Must not: decide strategy eligibility or submit trading orders.

Gateway target configuration

Deployment workflows require the repository-level Actions variable IB_GATEWAY_TARGETS_JSON. It is a JSON object keyed by a non-sensitive target label (or a list whose entries include name). Every target supplies its own GCP project, Workload Identity provider, service account, VM location, and deployment settings. The repository has no default gateway or cloud account.

Setting maintenance_enabled=false for a target pauses scheduled redeployment only; it does not disable the Gateway health and 2FA timers already installed on the VM. A scheduled workflow can succeed when every target was skipped, so its result does not verify the Gateway API, login, or account identity. Use the read-only diagnostics and separate account-level evidence; do not enable scheduled redeployment just to obtain a green workflow.

{
  "gateway-a": {
    "gcp_project_id": "<gcp-project-id>",
    "gcp_workload_identity_provider": "projects/<number>/locations/global/workloadIdentityPools/<pool>/providers/<provider>",
    "gcp_workload_identity_service_account": "<service-account>@<gcp-project-id>.iam.gserviceaccount.com",
    "gce_user": "<vm-user>",
    "gce_instance_name": "<vm-name>",
    "gce_zone": "<gcp-zone>",
    "deploy_path": "/opt/ibkr-gateway",
    "mode": "paper",
    "container_name": "<container-name>",
    "compose_project_name": "<compose-project>",
    "compose_service_name": "<compose-service>",
    "cloud_run_egress_cidr": "<trusted-egress-cidr>",
    "allow_connections_from_localhost_only": "no",
    "ssh_private_key_secret_name": "<secret-manager-secret-name>"
  }
}

Use Secret Manager secret names rather than secret values. Full deployments require target-specific names for the SSH key and Gateway username and password. VNC is optional: omit vnc_server_password_secret_name to leave the VNC server disabled. TOTP auto-fill is off by default; an explicit opt-in also requires a target-specific TOTP secret name. When auto-fill is off, the workflow does not fetch or send the TOTP seed to the VM. Do not commit target JSON, credentials, account identifiers, addresses, or private keys. A missing required target configuration fails before any cloud authentication or remote operation.

Optional ib_gateway_base_version selects the Gateway base image version for one target. The default is 10.50.1e; change it only after checking that target's compatibility. Other targets keep their configured version.

Full deployments install the VM runtime .env with mode 0600. The Gateway uses the configured username and password; when ibkr_2fa_autofill is explicitly enabled, the 2FA bot also reads the target-specific TOTP seed and fills a compatible challenge. This path does not use VNC. Verify API and account identity before enabling platform traffic. The VM stores the Gateway password and, when enabled, the TOTP seed; restrict access to that host and its deployment identity.

When moving a VM to another project, optional gcp_secret_project_id keeps Secret Manager reads in the existing project; otherwise secrets come from gcp_project_id. The configured deployment service account needs access to both the VM and those specific secrets. Change the target only after the new VM is ready and the old Gateway has stopped.

Project and instance names work for IAP administration, and a platform service can reach the Gateway over the VPC's internal address. The Gateway VM still needs a separate route to IBKR on the public internet. Removing its external IP requires another outbound route, such as Cloud NAT, with its own charges.

The manual read-only metadata workflow can optionally compare the protected target's SSH key with instance/project SSH-key metadata. Enable inspect_ssh_key_binding only together with match_current_gateway and inspect_ssh_policy; leave inspect_connections off. It reports only a fixed match, no-match, or unknown result, never opens SSH, and does not establish guest-side SSH health. OS Login makes metadata-key comparison unknown; a no-match does not prove the Gateway is unreachable.

With match_current_gateway and inspect_connections, the same bounded IAP SSH inspection also reads the container's actual published TCP mapping and checks the mapped container port, the host API port, and port 5000 in the host and selected container namespaces. This avoids confusing a host port with a container port. Output is fixed statuses only, without addresses or process details. It sends no API/HTTP request and changes no runtime configuration. A listener on port 5000 does not identify Client Portal Gateway or prove a valid session; its absence does not rule out another port or container. Native socket observations do not establish authentication, cash completeness or spending authority. The original two-argument helper and its six-field output remain supported; the extended response is accepted only as one complete field group.

The metadata workflow's inspect_readonly_api=true mode makes one bounded low-level API handshake on one exact resource-digest-bound protected target. It admits only API startup and does not synchronize accounts or orders. The separate inspect_contract_details flag defaults off and requires that mode. When explicitly authorized for the verified SOXL Gateway, it adds exactly one SOXL/SMART/USD security-definition request after the handshake. The outgoing SDK port admits only startup (71) and that one fixed request (9); submission, cancellation and all other requests remain forbidden. Connection and definition waits each have a 10-second limit within the existing 25-second native limit; there is no retry or reconnect.

Only fixed states, completion/counts, identity-match booleans and allowed error codes leave the native process. Incomplete records remain unknown, a rejected request is not a successful empty response, and a local definition response does not establish the Cloud Run path, account identity, market-data permission or trading readiness. All raw SDK output and callback errors stay captured; no login, restart, settings or installed dependency changes occur.

The separate manual Read Native Cash From Existing Gateway workflow performs one active, bounded read on that protected target. Its expected single account comes from IB_GATEWAY_EXPECTED_ACCOUNT_SELECTOR_JSON, copied from the existing runtime's private selector rather than inferred from a target label. It verifies the existing VM address and the native managed account before requesting data. It uses the container's installed ib_insync==0.9.86; it does not install, restart, log in, deploy a container or create cloud resources. Native outgoing messages are restricted to API startup, selected-account updates (including unsubscribe) and all-client open-order reads; no summary, order write or auto-bind request is admitted. A random nonzero client ID avoids the SDK's client-zero manual-order binding path; a collision fails without retry.

The workflow emits only fixed field states/counts and an evidence digest. Cash values and selected API orders stay in a new mode-0700 directory, with a create-only mode-0600 observation.json, inside the existing container's /tmp. Readback verifies those bytes locally. This is container-local evidence, not a durable cloud report; container replacement may remove it. Missing settlement fields, transfer obligations and durable storage remain explicit gaps. An empty API order result does not certify all account obligations or spending authority.

To inspect settlement-related field names already saved by a successful run, provide both its saved_capture_id and saved_evidence_sha256. This mode verifies the private file's permissions, exact digest, account, capture and completed-read markers before returning only tag/currency/count metadata. It does not construct an SDK client or reconnect to the broker. Numeric-row counts describe what the collector preserved as a decimal; they do not establish settlement semantics or permit a tag alias. A missing file or mismatched binding fails without fallback to a fresh broker request.

SettledCashByDate, SettledCashByDate-C and SettledCashByDate-S are preserved as bounded raw strings only in the private observation. A schedule is not a scalar decimal: the diagnostic does not drop its text or alias it to SettledCash. Public output shows only row counts and syntax shapes, collapsing all digit/letter runs so that dates and amounts remain private. Parsing the schedule, choosing a date/segment and establishing its financial meaning require their own verified source contract; this diagnostic grants no spending authority.

The public collector is gzip/base64 packaged for SSH stdin transport and checked against its exact source digest on the existing Linux host before execution. Account binding still travels on stdin, never command arguments. The capture UUID is emitted before transport so an interrupted read can later be inspected without issuing another broker request. No timeout is extended.

Saved readback also reports single-pair syntax, the shape of its numeric key and exact same-capture arithmetic between total, securities and commodities. A zero key is not assumed to mean today, and a calendar-shaped key is not assigned a broker timezone. Equality with captured ledger cash does not certify settlement semantics. Multiple entries, missing raw text and duplicates remain explicit; no entry is selected as current cash and no production cash adapter is changed.

With both saved binding inputs, the optional retain_saved_on_host flag copies the verified original bytes to the existing host user's private ~/.qsl-native-cash-evidence/<capture UUID>/observation.json. The copy uses the existing SSH/Docker access, retains the original SHA256, creates mode-0700/0600 paths and verifies byte readback. An existing identical copy is only read; a conflicting file is never overwritten. Docker's archive stays in host memory, is not extracted, and only one regular file is accepted. No raw bytes travel back through SSH. This survives container replacement while the host disk is retained; it is not a verified cloud report or a backup against VM/disk loss. The flag defaults off and requires saved-file mode; it never reconnects to the broker. A requested retention failure fails the job while preserving the separate saved-read result. No permission, storage service or Gateway deployment is created.

The same workflow accepts runtime_preflight=true for a metadata-only check of the protected matched host. It is exclusive with saved-capture and retention inputs. It checks existing Docker image source labels, installed distribution metadata in the running Gateway container, and available host memory/disk. It never pulls or launches an image, imports a broker/strategy package, reads container configuration, connects to a broker, or changes software or state. Only fixed states and resource counts leave Linux. Candidate image labels are not source verification, missing container packages do not describe other environments, and the result never grants runtime or trading readiness.

runtime_image_access=true separately checks the fixed original platform manifest using the VM's attached service-account token. Its private immutable locator comes from PILOT_RUNTIME_BASE_IMAGE; it is not a public workflow input. The check is exclusive with the other modes, rejects redirects and proxies, and verifies the returned bytes against the admitted digest. It never pulls layers, launches a runtime or changes permissions; denial stops the operation.

runtime_identity_diagnostic=true diagnoses that VM authentication stage alone. It reports the attached service-account count from the already-read VM metadata and makes one metadata token request with fixed HTTP/transport/response states. It never requests the registry, retries the request, changes identity or prints tokens, account identities or error bodies. It is exclusive with all other modes.

Repository layout

• tests/: unit, contract, and regression tests.
• .github/workflows/: CI, scheduled jobs, release, or deployment workflows.
• scripts/: operator scripts and local helpers.

Quick start

python -m pip install -r requirements.txt
python -m pytest -q

Useful docs

• No separate docs/ directory yet; start with this README and the workflow files.

QSL compatibility

• qsl.toml records tier, upgrade_ring, and compat.bundle for compatibility checks.
• This Gateway operations utility installs Python dependencies from requirements.txt.

Community and security

• See CONTRIBUTING.md for pull request scope, local verification, and documentation expectations.
• Follow CODE_OF_CONDUCT.md for maintainer and contributor conduct.
• Report credential, automation, broker, exchange, or cloud-resource vulnerabilities through SECURITY.md; do not open public issues for secrets or live-execution risk.

Contributing

See CONTRIBUTING.md. AI assistants and maintainers follow AGENTS.md.

License

See LICENSE.