Repository navigation
feat(api): list organization notifications and finish drift coverage - #1104
Merged
sdairs merged 1 commit intoOct 21, 2026
Merged
Conversation
This was referenced Oct 20, 2026
sdairs
changed the title
This was referenced Oct 20, 2026
sdairs
marked this pull request as ready for review
October 20, 2026 12:31
iskakaushik
approved these changes
Oct 20, 2026
sdairs
added a commit
that referenced
this pull request
Oct 21, 2026
…#1100) AWS backup-bucket payloads now include `iamRoleExternalId`, and the live OpenAPI adds organization notification listing and an additional service-create permission. Refresh the snapshot, regenerate operation metadata and the beta list, and model external IDs on the four AWS bucket types. The update request preserves omission, a replacement string, and explicit `null` (clear) independently. Its custom deserializer retains unknown-field tracking for strict CLI file input. Create/properties require the string under the repository's spec requiredness policy; response fields remain optional. The library README documents caller migration. Refs #1088. Stacked directly above #1099, preserving its Query API response-envelope fix and all earlier layers. BigQuery and notifications are separate API/CLI follow-ups; `spec_coverage_test` intentionally reports their remaining drift on this intermediate layer. Validation: three focused external-ID model tests and four backup client wiremock tests pass. No live resources or credentials were created. Stack dependency: #1099. Final API coverage: #1104; final CLI layer: #1105. Combined-stack validation at `dc45f645f20fa7b0d3639d732531d947726b68a5`: `cargo fmt --all --check`; both CLI Clippy configurations; `cargo check -p clickhousectl --no-default-features`; API/analyzer all-target Clippy; `cargo check --workspace --all-features`; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered. Exact-head CI at `ae6fffc5497229328a6f44458d4440a05cf72cf4`: format and MSRV pass; API build and Clippy pass. [The API test job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415838262) fails only in `vendored_openapi_snapshot_matches_rust_api`, with the expected 17 BigQuery/notifications findings covered by #1102/#1104. The dependent final API layer #1104 passes its full job. The rendering step is skipped after this expected test failure. Cloud integration decision requires authorization; no provisioning label was applied. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes AWS backup-bucket request shapes (required external ID on create) and PATCH semantics for IAM trust configuration; incorrect callers could break backup setup or fail API authorization checks after the new service-create permission is documented. > > **Overview** > Refreshes the vendored ClickHouse Cloud OpenAPI snapshot and regenerates operation metadata. **Service create** (`INSTANCE_CREATE`) now documents both `control-plane:organization:create-service` and `control-plane:service:manage`. The beta **organization notifications list** operation appears in the snapshot and in `meta` (`notification_get_list`); a full Rust client for it is left to a follow-up layer. > > **AWS backup buckets** gain `iam_role_external_id` (`iamRoleExternalId`) on response and request types. Responses accept missing or null external IDs; create/properties require a string. Patch uses `Option<Option<String>>` with a custom deserializer so callers can omit the field (preserve), send a string (replace), or send `null` (clear). The README describes migration for struct-literal callers. > > Adds focused **serde tests** and updates **wiremock backup-bucket client tests** for the new field and wire shapes. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ae6fffc. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
sdairs
added a commit
that referenced
this pull request
Oct 21, 2026
`cloud backup bucket create/update --file` now has explicit coverage and documentation for AWS `iamRoleExternalId`. Update input distinguishes omission (preserve), string (replace), and `null` (clear); unknown fields and invalid types still fail before HTTP. The existing typed file-input path carries the API field without adding a flag or duplicating request serialization. Fixes #1106. Refs #1088. Depends on the immediately preceding backup API layer, itself above #1099 in native stack #1095. No new library method is omitted. The AWS create example follows the live schema's required string field. Validation: builder tests cover create requiredness, all update states and strict unknown-field checks; real-binary wiremock coverage pins POST/PATCH bodies. The cloud-service guide includes clearing through stdin. No paid provisioning was attempted. Stack dependency: #1100. Final API coverage: #1104; final CLI layer: #1105. Combined-stack validation at `dc45f645f20fa7b0d3639d732531d947726b68a5`: `cargo fmt --all --check`; both CLI Clippy configurations; `cargo check -p clickhousectl --no-default-features`; API/analyzer all-target Clippy; `cargo check --workspace --all-features`; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered. Exact-head CI at `1b36b75c7b67bdc5ec7596c273b41f494770ca96`: [full CLI job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415845906), format, MSRV and Cloud integration decision pass. Live suites were not selected for this CLI-only layer. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Test and documentation only; no production CLI or API behavior changes in this diff. > > **Overview** > Adds **tests and docs** for AWS **`iamRoleExternalId`** on `cloud backup bucket create/update --file`, without changing CLI request-building logic in this diff. > > Unit tests now expect the field in AWS create/update JSON fixtures and add **`aws_backup_bucket_external_id_preserves_update_states_and_unknown_field_checks`**: update accepts omit/string/`null`, rejects bad types and typos, and create rejects missing or null external IDs. A **wiremock** test asserts the real binary sends matching **POST/PATCH** bodies for create (with external ID) and update (omit, replace, or clear). > > The **cloud-service** guide documents `iamRoleExternalId` in the AWS field table and explains preserve/replace/clear semantics with a stdin example. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 1b36b75. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
sdairs
added a commit
that referenced
this pull request
Oct 21, 2026
BigQuery ClickPipe settings now preserve the live API's CDC replication method, sync interval, parallelism, delay and pull window. Table mappings preserve the watermark column and events function in both request and tolerant response types. Explicit fallible write-back conversions report absent required wire fields. Add the spec-described `cdc` and `cdc_only` replication modes while retaining unknown future values. Keep the CLI's existing parser snapshot-only in this API layer; its CDC exposure is the next PR. The CLI settings literal is a compile-only adaptation, and a subprocess test pins omission of all newly added fields for existing snapshot requests. The library README documents Rust caller migration. Refs #1088. Depends on the preceding backup CLI layer in native stack #1095 above #1099. Notifications drift intentionally remains until its API follow-up. Validation: four focused model tests, snapshot wire-omission coverage, and combined-stack repository gates. Third-party BigQuery provisioning and live CDC were not attempted. Stack dependency: #1101. Final API coverage: #1104; final CLI layer: #1105. Combined-stack validation at `dc45f645f20fa7b0d3639d732531d947726b68a5`: `cargo fmt --all --check`; both CLI Clippy configurations; `cargo check -p clickhousectl --no-default-features`; API/analyzer all-target Clippy; `cargo check --workspace --all-features`; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered. Exact-head CI at `5bcc2976e5614040148dd848a207bd0286e8b1b8`: format, MSRV and [full CLI job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415854891) pass. API build and Clippy pass; [the API test job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415854963) fails only in `vendored_openapi_snapshot_matches_rust_api`, with the expected three notifications findings fixed by #1104. The rendering step is skipped after this expected test failure. Cloud integration decision requires authorization; no provisioning label was applied. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Expands ClickPipe BigQuery request/response models and write-back conversions used by API clients; CLI behavior stays snapshot-only but struct-literal callers must adopt new fields or `Default`. > > **Overview** > Extends the **clickhouse-cloud-api** BigQuery ClickPipe models to match the Cloud API’s CDC surface: `replicationMode` gains **`cdc`** and **`cdc_only`**, and settings/table mappings pick up optional CDC-related fields (`replicationMethod`, sync/pull tuning, `queryCdcWatermarkColumn`, `eventsFunction`, etc.) on both request and tolerant response types. > > Adds **`TryFrom`** conversions from BigQuery settings and table-mapping **response** types back to request types, with **`MissingRequiredFields`** when required wire fields are absent. **README** documents migration (`..Default::default()` for new struct fields). **Tests** cover serde round-trip, null/absent handling, and enum values. > > **clickhousectl** is only adapted to compile: BigQuery create still accepts **`snapshot`** only for `--replication-mode`, settings use **`..Default::default()`** so CDC fields are not sent on snapshot creates, and CLI request-shape tests assert those fields stay omitted. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 5bcc297. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
sdairs
added a commit
that referenced
this pull request
Oct 21, 2026
`cloud clickpipe create bigquery` now supports `cdc`/`cdc_only`, replication-method selection, and four timing/parallelism controls. Strict table-mapping JSON accepts the method-specific watermark or events function. Invalid mode/method combinations, missing per-table fields and invalid numeric values fail before reading credentials or sending HTTP; existing snapshot requests omit the new settings. Fixes #1107. Refs #1088. Depends on the immediately preceding BigQuery API layer in native stack #1095 above #1099. No new client method is omitted; BigQuery source updates remain unsupported by the existing API surface. Validation: 11 focused unit tests, nine focused subprocess tests, model write-back tests, and three command-tree checks passed. The ClickPipe configuration reference documents CDC inputs and constraints. No external BigQuery service or paid test service was provisioned. Stack dependency: #1102. Final API coverage: #1104; final CLI layer: #1105. Combined-stack validation at `dc45f645f20fa7b0d3639d732531d947726b68a5`: `cargo fmt --all --check`; both CLI Clippy configurations; `cargo check -p clickhousectl --no-default-features`; API/analyzer all-target Clippy; `cargo check --workspace --all-features`; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered. Exact-head CI at `65addfae0cb59bb91bbf879cf5a79039717009ac`: [full CLI job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415864220), format, MSRV and Cloud integration decision pass. Live suites were not selected for this CLI-only layer. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Adds new ingestion modes and validation on BigQuery pipe creation; mistakes are caught client-side, but misconfigured CDC could affect production replication behavior once deployed. > > **Overview** > **`cloud clickpipe create bigquery`** now supports **CDC** (`cdc` / `cdc_only`) alongside snapshot, using API-driven `--replication-mode` values instead of snapshot-only parsing. > > New create flags wire **replication method** (`query_based` / `events_based`) and **CDC tuning** (sync interval, pull parallelism, visibility delay, pull window). **`--table-mapping-json`** accepts **`queryCdcWatermarkColumn`** and **`eventsFunction`** (`appends` / `changes`), with validation that matches method to per-table fields and blocks CDC options in snapshot mode. > > Invalid combinations and out-of-range numbers **fail locally** before reading service-account files or calling the API; snapshot creates still omit the new settings. **Tests** cover parsing, request shape (both auth paths), and early rejection; **docs** add a BigQuery CDC section and update the source table blurb. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 65addfa. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
sdairs
added a commit
that referenced
this pull request
Oct 21, 2026
Add read-only `cloud org notifications` (Beta), exposing limit, cursor, severity, type, resource ID/type and inclusive RFC 3339 date filters. JSON preserves page entries and cursor metadata, including an empty opaque cursor; human output safely renders sparse entries. Route through a domain-local `CloudClient` wrapper and declare the notification workflow's permissions. Fixes #1108. Refs #1088. Depends on the immediately preceding notifications API layer, which closes the drift issue. This is the final new layer in native stack #1095 above #1099. All parameters of the new library method are exposed; no method is intentionally omitted. Validation: five focused unit tests and five real-binary subprocess tests pass, covering OAuth, filters, pagination, empty/sparse human output, errors and local date/range validation. Parent/leaf help was inspected and the authentication reference documents page-by-page reads. No paid provisioning or credentials creation was attempted. Stack dependency: #1104. Final API coverage: #1104; final CLI layer: #1105. Combined-stack validation at `dc45f645f20fa7b0d3639d732531d947726b68a5`: `cargo fmt --all --check`; both CLI Clippy configurations; `cargo check -p clickhousectl --no-default-features`; API/analyzer all-target Clippy; `cargo check --workspace --all-features`; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered. Exact-head CI at `dc45f645f20fa7b0d3639d732531d947726b68a5`: [full CLI job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415882754), format, MSRV and Cloud integration decision pass. The final live-spec dry run still reports zero actionable drift. Live suites were not selected for this CLI-only layer. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Read-only CLI and documentation with local validation; no writes, auth changes, or provisioning paths. > > **Overview** > Adds a read-only **Beta** `clickhousectl cloud org notifications` command to page through organization notifications via the Cloud API. > > The command exposes **limit**, **cursor**, **severity**, **--type**, **resource-id**, **resource-type**, and inclusive **RFC 3339** **from-date** / **to-date** filters. It is classified as a read (OAuth-friendly), registers **`NOTIFICATION_GET_LIST`** in org permissions, and calls a new **`CloudClient::list_notifications`** wrapper. **JSON** returns `result` with optional `limit` and `nextCursor`; **human** output prints a markdown table (sparse fields as `-`) and prints the next cursor as a safely escaped JSON string. > > Unit and subprocess tests cover flag parsing, validation, HTTP query forwarding, pagination, errors, and table rendering. **`docs/reference/authentication.md`** documents pagination and filter behavior alongside **`notification-types`**. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit dc45f64. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add the beta
Client::notification_get_listendpoint with all eight optional filters, opaque cursor forwarding, andApiResponse<Vec<Notification>>pagination metadata.Notificationtolerates missing/null fields and future type, severity and resource-kind strings. Export it through both public facade paths and add a read-only notification step to the organization integration lifecycle.Fixes #1088. Depends on the preceding BigQuery CLI layer in native stack #1095, preserving #1099 and the earlier drift work. Together the API layers cover all current findings: a fresh live dry run reports zero actionable drift. The three pre-existing API-role enum acknowledgements remain unchanged; no new exemption hides drift.
Validation: nine focused API cases pass, covering filters, OAuth, cursor encoding, empty/sparse responses, metadata, facade paths and errors. The full combined API/analyzer suite, including vendored spec coverage, passes. The live organization lifecycle was not run because it also creates credentials/resources; the added step itself is read-only.
Stack dependency: #1103. Final API coverage: #1104; final CLI layer: #1105.
Combined-stack validation at
dc45f645f20fa7b0d3639d732531d947726b68a5:cargo fmt --all --check; both CLI Clippy configurations;cargo check -p clickhousectl --no-default-features; API/analyzer all-target Clippy;cargo check --workspace --all-features; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered.Exact-head CI at
9431c677e0cf3f172dcbf48f81849c7191e81d38: full API/analyzer job, including vendored spec coverage and Python rendering/classifier tests, passes. Format and MSRV pass. Cloud integration decision requires authorization; the provisioning suites remain skipped. No provisioning label was applied.Note
Low Risk
Read-only beta client surface plus tolerant response models and tests; no auth, billing, or mutating API behavior.
Overview
Adds the beta
Client::notification_get_listendpoint so callers can page through an organization’s notification history asApiResponse<Vec<Notification>>, with optionallimit, opaquecursor, and filters (severity, type, resource id/type, RFC 3339 date range).Introduces the
Notificationresponse model (all fields optional; type/severity/resource kind stay open strings) and re-exports it on the publicmodelsfacade. README documents pagination defaults (100, max 250) and retention/access rules.Tests: wiremock coverage for query params, envelope metadata, sparse/null pages, and API errors; serde round-trip tolerance for
Notification; facade type parity; a read-only “list notifications” step in the org integration lifecycle (including optional cursor follow-up).Reviewed by Cursor Bugbot for commit 9431c67. Bugbot is set up for automated code reviews on this repo. Configure here.