From cc1b8fca3074f2303cbe3c731cf2f680a137f140 Mon Sep 17 00:00:00 2001 From: Wu Sheng Date: Tue, 16 Jun 2026 23:00:29 +0800 Subject: [PATCH 1/3] Clear CVE Dependabot alerts: Curator/ZooKeeper, assertj, e2e Java/Python fixtures Shipped (oap-server-bom + LICENSE + docs): - Apache Curator 4.3.0 -> 5.9.0 (curator-test too) and ZooKeeper 3.5.7 -> 3.9.5, bumped together (Curator 5.x is the line carrying the ZK 3.9.x client). Clears CVE-2023-44981. OAP is a ZK client only so the server-side bug was never reachable, but the bundled jar tripped Dependabot. Plugins use only stable Curator APIs; no source changes. Supported ZooKeeper server version is now 3.5+ (3.4.x dropped by Curator 5.x). LICENSE + cluster docs + application.yml updated. Build/test scope: - assertj-core 3.20.2 -> 3.27.7 (CVE-2026-24400, test scope only). e2e test fixtures (test/e2e-v2, never shipped): - guava -> 32.0.0-jre, kafka-clients -> 3.9.2, log4j-core -> 2.25.4, logback -> 1.2.13 (Java-8 line), json-path -> 2.9.0, flask -> 3.1.3, protobuf -> 4.25.8. The 17 Go-fixture alerts require a go1.24 toolchain (grpc 1.79.3 / x-crypto 0.45 need go 1.24, above skywalking-go's published go1.23 ceiling) and are handled separately: skywalking-go go1.24 enablement + e2e go fixture migration. --- dist-material/release-docs/LICENSE | 14 +++++++------- docs/en/changes/changes.md | 2 ++ docs/en/setup/backend/backend-cluster.md | 6 +++--- oap-server-bom/pom.xml | 6 +++--- .../src/main/resources/application.yml | 3 +-- pom.xml | 2 +- .../cases/airflow/mock/requirements-replay.txt | 4 ++-- .../e2e-mock-baseline-server/pom.xml | 2 +- .../java-test-service/e2e-mock-sender/pom.xml | 2 +- .../java-test-service/e2e-service-provider/pom.xml | 6 +++--- test/e2e-v2/java-test-service/pom.xml | 6 +++--- 11 files changed, 27 insertions(+), 26 deletions(-) diff --git a/dist-material/release-docs/LICENSE b/dist-material/release-docs/LICENSE index 614f378faa2d..0776a406b0b7 100644 --- a/dist-material/release-docs/LICENSE +++ b/dist-material/release-docs/LICENSE @@ -323,10 +323,10 @@ The text of each license is the standard Apache 2.0 license. https://mvnrepository.com/artifact/org.apache.commons/commons-compress/1.26.2 Apache-2.0 https://mvnrepository.com/artifact/org.apache.commons/commons-lang3/3.18.0 Apache-2.0 https://mvnrepository.com/artifact/org.apache.commons/commons-text/1.4 Apache-2.0 - https://mvnrepository.com/artifact/org.apache.curator/curator-client/4.3.0 Apache-2.0 - https://mvnrepository.com/artifact/org.apache.curator/curator-framework/4.3.0 Apache-2.0 - https://mvnrepository.com/artifact/org.apache.curator/curator-recipes/4.3.0 Apache-2.0 - https://mvnrepository.com/artifact/org.apache.curator/curator-x-discovery/4.3.0 Apache-2.0 + https://mvnrepository.com/artifact/org.apache.curator/curator-client/5.9.0 Apache-2.0 + https://mvnrepository.com/artifact/org.apache.curator/curator-framework/5.9.0 Apache-2.0 + https://mvnrepository.com/artifact/org.apache.curator/curator-recipes/5.9.0 Apache-2.0 + https://mvnrepository.com/artifact/org.apache.curator/curator-x-discovery/5.9.0 Apache-2.0 https://mvnrepository.com/artifact/org.apache.httpcomponents/httpasyncclient/4.1.5 Apache-2.0 https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient/4.5.13 Apache-2.0 https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore/4.4.16 Apache-2.0 @@ -335,9 +335,9 @@ The text of each license is the standard Apache 2.0 license. https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api/2.25.4 Apache-2.0 https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core/2.25.4 Apache-2.0 https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-slf4j-impl/2.25.4 Apache-2.0 - https://mvnrepository.com/artifact/org.apache.yetus/audience-annotations/0.5.0 Apache-2.0 - https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper/3.5.7 Apache-2.0 - https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper-jute/3.5.7 Apache-2.0 + https://mvnrepository.com/artifact/org.apache.yetus/audience-annotations/0.12.0 Apache-2.0 + https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper/3.9.5 Apache-2.0 + https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper-jute/3.9.5 Apache-2.0 https://mvnrepository.com/artifact/org.freemarker/freemarker/2.3.31 Apache-2.0 https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-reflect/1.7.10 Apache-2.0 https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-stdlib/1.7.10 Apache-2.0 diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index 0de690baf9c7..2cc1768421b6 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -301,6 +301,8 @@ * Add `@Stream(allowBootReshape = true)` opt-in for additive boot-time reshape of BanyanDB streams / measures. Code-defined stream classes (e.g. `AlarmRecord`) can now annotate their schema as eligible for in-place additive update at OAP boot — a new `@Column` is appended to the live tag-family / fields via `client.update` instead of being silently rejected with `SKIPPED_SHAPE_MISMATCH` (which previously forced operators to drop the measure / stream and lose historical rows). Additive includes both new tags / fields **and** relocating an existing tag between families when a `@Column`'s `storageOnly` flag flips (e.g. `id1` moving from `storage-only` → `searchable` when it becomes indexed). The opt-in is per-stream and gated by an `isPurelyAdditive` shape diff: tag type changes, tag drops, kind flips (tag↔field), entity / interval / sharding-key changes, and field re-typing still skip with `SKIPPED_SHAPE_MISMATCH`, so identity-breaking edits remain explicit operator actions. Only the init / standalone OAP performs the reshape; non-init peers continue through the existing poll-and-wait loop so a single node drives DDL. When a `check*` records `SKIPPED_SHAPE_MISMATCH` the dependent `IndexRule` / `IndexRuleBinding` reconciliation is also skipped — preventing the previous gap where the binding silently updated to a tag list that diverged from the live tag-family layout. `AlarmRecord` is opted in. Default remains `false` for all other models — boot-time reshape stays off unless the annotation is explicitly set. **Operator caveat:** BanyanDB does not physically migrate existing rows when a tag's family changes; pre-existing data stays in its original on-disk location while new writes go to the declared family — expect a backfill window for queries that route through new IndexRules on relocated tags. * Mask keywords `trustStorePass`, `keyStorePass` by default. * Bump up dependencies to clear CVE alerts on shipped OAP jars: log4j `2.25.3` → `2.25.4`, jackson `2.18.5` → `2.18.6`, kafka-clients `3.4.0` → `3.9.2`, postgresql `42.4.4` → `42.7.11`, commons-compress `1.21` → `1.26.2`. +* Bump Apache Curator `4.3.0` → `5.9.0` and Apache ZooKeeper `3.5.7` → `3.9.5` together to clear CVE-2023-44981 (the bundled ZooKeeper jar carried it; OAP is a ZooKeeper client only, so the server-side bug was never reachable, but the jar tripped Dependabot). The cluster-zookeeper and configuration-zookeeper plugins use only stable Curator APIs, so no source changes were required. Operator-facing change: the supported ZooKeeper server version is now 3.5+ — ZooKeeper 3.4.x is no longer supported. +* Bump test-scope assertj-core `3.20.2` → `3.27.7` to clear CVE-2026-24400 (XXE in `isXmlEqualTo`, not used by any test). * Fix: continuous profiling policy validation now rejects a threshold / count of `0` to match the error messages and rover's `value >= threshold` trigger semantics (a `0` threshold would always trigger). CPU percent and HTTP error rate are tightened from `[0-100]` to `(0-100]`. * Fix wrong BanyanDB resource options in record data. * Align the default BanyanDB stage `segmentInterval` values so each coarser stage is an integer multiple of the finer one (`records` cold `3` → `4`, `metricsMinute` cold `5` → `6`, `metricsHour` warm `7` → `10` and cold `15` → `20`), keeping hot → warm → cold lifecycle migration on the cheap whole-segment fast path. diff --git a/docs/en/setup/backend/backend-cluster.md b/docs/en/setup/backend/backend-cluster.md index ff1ea0e09f23..c7ed2f55fa72 100644 --- a/docs/en/setup/backend/backend-cluster.md +++ b/docs/en/setup/backend/backend-cluster.md @@ -82,7 +82,7 @@ You could have two options Zookeeper is a very common and widely used cluster coordinator. Set the **cluster/selector** to **zookeeper** in the yml to enable it. -Required Zookeeper version: 3.5+ +Supported Zookeeper server version: 3.5+. The bundled ZooKeeper client library is 3.9.x. ```yaml cluster: @@ -92,11 +92,11 @@ cluster: - `hostPort` is the list of zookeeper servers. Format is `IP1:PORT1,IP2:PORT2,...,IPn:PORTn` - `enableACL` - enable [Zookeeper ACL](https://zookeeper.apache.org/doc/r3.5.5/zookeeperProgrammers.html#sc_ZooKeeperAccessControl) to + enable [Zookeeper ACL](https://zookeeper.apache.org/doc/r3.9.3/zookeeperProgrammers.html#sc_ZooKeeperAccessControl) to control access to its znode. - `schema` is Zookeeper ACL schemas. - `expression` is a expression of ACL. The format of the expression is specific to - the [schema](https://zookeeper.apache.org/doc/r3.5.5/zookeeperProgrammers.html#sc_BuiltinACLSchemes). + the [schema](https://zookeeper.apache.org/doc/r3.9.3/zookeeperProgrammers.html#sc_BuiltinACLSchemes). - `hostPort`, `baseSleepTimeMs` and `maxRetries` are settings of Zookeeper curator client. Note: diff --git a/oap-server-bom/pom.xml b/oap-server-bom/pom.xml index 7a08c4593ba1..81768d79dbec 100644 --- a/oap-server-bom/pom.xml +++ b/oap-server-bom/pom.xml @@ -37,7 +37,7 @@ 3.14.9 4.5.13 2.10.5 - 3.5.7 + 3.9.5 32.0.1-jre 2.0 4.33.1 @@ -54,8 +54,8 @@ 0.6.0 1.8.0 2.3.2 - 4.3.0 - 2.12.0 + 5.9.0 + 5.9.0 2.18.0 2.3.31 3.25.0-GA diff --git a/oap-server/server-starter/src/main/resources/application.yml b/oap-server/server-starter/src/main/resources/application.yml index e02938cc8e60..089f6729bdd9 100644 --- a/oap-server/server-starter/src/main/resources/application.yml +++ b/oap-server/server-starter/src/main/resources/application.yml @@ -16,8 +16,7 @@ cluster: selector: ${SW_CLUSTER:standalone} standalone: - # Please check your ZooKeeper is 3.5+, However, it is also compatible with ZooKeeper 3.4.x. Replace the ZooKeeper 3.5+ - # library the oap-libs folder with your ZooKeeper 3.4.x library. + # Supported ZooKeeper server version: 3.5+. The bundled ZooKeeper client library is 3.9.x. zookeeper: namespace: ${SW_NAMESPACE:""} hostPort: ${SW_CLUSTER_ZK_HOST_PORT:localhost:2181} diff --git a/pom.xml b/pom.xml index 18490541ab88..75696d88a4a3 100755 --- a/pom.xml +++ b/pom.xml @@ -182,7 +182,7 @@ 3.1.0 1.37 true - 3.20.2 + 3.27.7 2.8.0 1.6.0 diff --git a/test/e2e-v2/cases/airflow/mock/requirements-replay.txt b/test/e2e-v2/cases/airflow/mock/requirements-replay.txt index 80c4250e3beb..2e302eda6988 100644 --- a/test/e2e-v2/cases/airflow/mock/requirements-replay.txt +++ b/test/e2e-v2/cases/airflow/mock/requirements-replay.txt @@ -1,4 +1,4 @@ -flask==3.0.3 +flask==3.1.3 grpcio==1.62.2 -protobuf==4.25.3 +protobuf==4.25.8 opentelemetry-proto==1.24.0 diff --git a/test/e2e-v2/java-test-service/e2e-mock-baseline-server/pom.xml b/test/e2e-v2/java-test-service/e2e-mock-baseline-server/pom.xml index e24dc67e3a3a..f1f91a022386 100644 --- a/test/e2e-v2/java-test-service/e2e-mock-baseline-server/pom.xml +++ b/test/e2e-v2/java-test-service/e2e-mock-baseline-server/pom.xml @@ -58,7 +58,7 @@ com.google.guava guava - 23.0 + 32.0.0-jre io.grpc diff --git a/test/e2e-v2/java-test-service/e2e-mock-sender/pom.xml b/test/e2e-v2/java-test-service/e2e-mock-sender/pom.xml index 922f7ed5e230..29dedb3b58ac 100644 --- a/test/e2e-v2/java-test-service/e2e-mock-sender/pom.xml +++ b/test/e2e-v2/java-test-service/e2e-mock-sender/pom.xml @@ -61,7 +61,7 @@ com.jayway.jsonpath json-path - 2.7.0 + 2.9.0 diff --git a/test/e2e-v2/java-test-service/e2e-service-provider/pom.xml b/test/e2e-v2/java-test-service/e2e-service-provider/pom.xml index 2de07cc331c9..74b73f137cc7 100644 --- a/test/e2e-v2/java-test-service/e2e-service-provider/pom.xml +++ b/test/e2e-v2/java-test-service/e2e-service-provider/pom.xml @@ -35,8 +35,8 @@ 1.2.17 - 2.17.1 - 1.2.3 + 2.25.4 + 1.2.13 @@ -109,7 +109,7 @@ com.google.guava guava - 23.0 + 32.0.0-jre diff --git a/test/e2e-v2/java-test-service/pom.xml b/test/e2e-v2/java-test-service/pom.xml index f7206b423a96..1a0e04524ecf 100644 --- a/test/e2e-v2/java-test-service/pom.xml +++ b/test/e2e-v2/java-test-service/pom.xml @@ -50,11 +50,11 @@ 2.1.2.RELEASE 5.6.0 2.9.7 - 30.1.1-jre + 32.0.0-jre 2.1.210 8.0.13 1.18.40 - 2.4.1 + 3.9.2 2.22.0 3.13.0 @@ -120,7 +120,7 @@ org.apache.kafka kafka-clients - 2.8.0 + ${kafka-clients.version} From 36dce5d8553f5df0116572981c06935f0590956f Mon Sep 17 00:00:00 2001 From: Wu Sheng Date: Wed, 17 Jun 2026 10:40:39 +0800 Subject: [PATCH 2/3] Add license-eye override for ZooKeeper 3.9.5 (jar license not auto-identifiable) The Dependency-licenses CI job failed because `license-eye dependency resolve` cannot auto-identify the license of the org.apache.zookeeper:zookeeper / zookeeper-jute 3.9.5 jars (reported as "Unknown" -> "failed to identify the licenses"), so the resolve step exited 1 before the LICENSE diff check. Add an explicit Apache-2.0 override in .licenserc.yaml for both artifacts, mirroring the existing overrides (okhttp, retrofit, postgresql, ...). Verified locally with the CI-pinned license-eye: resolve now succeeds and regenerates a LICENSE identical to the committed one (the LICENSE content was already correct). --- .licenserc.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.licenserc.yaml b/.licenserc.yaml index 7a85203fda51..b9399b20e604 100644 --- a/.licenserc.yaml +++ b/.licenserc.yaml @@ -144,4 +144,10 @@ dependency: - name: io.vertx:vertx-grpc version: 4.5.9 license: EPL-2.0 + - name: org.apache.zookeeper:zookeeper + version: 3.9.5 + license: Apache-2.0 + - name: org.apache.zookeeper:zookeeper-jute + version: 3.9.5 + license: Apache-2.0 From 7a73cb24ec5926740d57b447582390431bf259cc Mon Sep 17 00:00:00 2001 From: Wu Sheng Date: Wed, 17 Jun 2026 13:51:04 +0800 Subject: [PATCH 3/3] Fix ZooKeeper cluster e2e and correct the supported server version to 3.6+ The Cluster ZK e2e tests failed (oap1 unhealthy): the OAP's new Curator 5.9.0 / ZooKeeper 3.9.5 client cannot register its instance against the e2e's EOL zookeeper:3.5 server. Reproduced locally: Curator 5.x ServiceDiscovery (watchInstances=true) issues `addWatch` (persistent watch, opcode 106), which ZooKeeper added in server 3.6.0; a 3.5 server rejects it ("Received packet at server of unknown type 106") so the create request fails with ConnectionLoss -> ModuleStartException -> oap1 unhealthy. Verified the same path succeeds on zookeeper:3.6 and 3.9, fails on 3.5. - e2e: bump zookeeper:3.5 -> zookeeper:3.9 (matches the bundled 3.9.x client) in cases/cluster/zk/{es,banyandb}. - Correct the documented minimum from 3.5+ to 3.6+ (application.yml, backend-cluster.md, changes.md) -- persistent watches require server 3.6.0. --- docs/en/changes/changes.md | 2 +- docs/en/setup/backend/backend-cluster.md | 2 +- oap-server/server-starter/src/main/resources/application.yml | 2 +- test/e2e-v2/cases/cluster/zk/banyandb/docker-compose.yml | 2 +- test/e2e-v2/cases/cluster/zk/es/docker-compose.yml | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index 2cc1768421b6..17c291333f33 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -301,7 +301,7 @@ * Add `@Stream(allowBootReshape = true)` opt-in for additive boot-time reshape of BanyanDB streams / measures. Code-defined stream classes (e.g. `AlarmRecord`) can now annotate their schema as eligible for in-place additive update at OAP boot — a new `@Column` is appended to the live tag-family / fields via `client.update` instead of being silently rejected with `SKIPPED_SHAPE_MISMATCH` (which previously forced operators to drop the measure / stream and lose historical rows). Additive includes both new tags / fields **and** relocating an existing tag between families when a `@Column`'s `storageOnly` flag flips (e.g. `id1` moving from `storage-only` → `searchable` when it becomes indexed). The opt-in is per-stream and gated by an `isPurelyAdditive` shape diff: tag type changes, tag drops, kind flips (tag↔field), entity / interval / sharding-key changes, and field re-typing still skip with `SKIPPED_SHAPE_MISMATCH`, so identity-breaking edits remain explicit operator actions. Only the init / standalone OAP performs the reshape; non-init peers continue through the existing poll-and-wait loop so a single node drives DDL. When a `check*` records `SKIPPED_SHAPE_MISMATCH` the dependent `IndexRule` / `IndexRuleBinding` reconciliation is also skipped — preventing the previous gap where the binding silently updated to a tag list that diverged from the live tag-family layout. `AlarmRecord` is opted in. Default remains `false` for all other models — boot-time reshape stays off unless the annotation is explicitly set. **Operator caveat:** BanyanDB does not physically migrate existing rows when a tag's family changes; pre-existing data stays in its original on-disk location while new writes go to the declared family — expect a backfill window for queries that route through new IndexRules on relocated tags. * Mask keywords `trustStorePass`, `keyStorePass` by default. * Bump up dependencies to clear CVE alerts on shipped OAP jars: log4j `2.25.3` → `2.25.4`, jackson `2.18.5` → `2.18.6`, kafka-clients `3.4.0` → `3.9.2`, postgresql `42.4.4` → `42.7.11`, commons-compress `1.21` → `1.26.2`. -* Bump Apache Curator `4.3.0` → `5.9.0` and Apache ZooKeeper `3.5.7` → `3.9.5` together to clear CVE-2023-44981 (the bundled ZooKeeper jar carried it; OAP is a ZooKeeper client only, so the server-side bug was never reachable, but the jar tripped Dependabot). The cluster-zookeeper and configuration-zookeeper plugins use only stable Curator APIs, so no source changes were required. Operator-facing change: the supported ZooKeeper server version is now 3.5+ — ZooKeeper 3.4.x is no longer supported. +* Bump Apache Curator `4.3.0` → `5.9.0` and Apache ZooKeeper `3.5.7` → `3.9.5` together to clear CVE-2023-44981 (the bundled ZooKeeper jar carried it; OAP is a ZooKeeper client only, so the server-side bug was never reachable, but the jar tripped Dependabot). The cluster-zookeeper and configuration-zookeeper plugins use only stable Curator APIs, so no source changes were required. Operator-facing change: the supported ZooKeeper server version is now 3.6+ (Curator 5.x uses ZooKeeper persistent watches, added in server 3.6.0); older servers (3.5.x, 3.4.x) are no longer supported. * Bump test-scope assertj-core `3.20.2` → `3.27.7` to clear CVE-2026-24400 (XXE in `isXmlEqualTo`, not used by any test). * Fix: continuous profiling policy validation now rejects a threshold / count of `0` to match the error messages and rover's `value >= threshold` trigger semantics (a `0` threshold would always trigger). CPU percent and HTTP error rate are tightened from `[0-100]` to `(0-100]`. * Fix wrong BanyanDB resource options in record data. diff --git a/docs/en/setup/backend/backend-cluster.md b/docs/en/setup/backend/backend-cluster.md index c7ed2f55fa72..b5a352802f94 100644 --- a/docs/en/setup/backend/backend-cluster.md +++ b/docs/en/setup/backend/backend-cluster.md @@ -82,7 +82,7 @@ You could have two options Zookeeper is a very common and widely used cluster coordinator. Set the **cluster/selector** to **zookeeper** in the yml to enable it. -Supported Zookeeper server version: 3.5+. The bundled ZooKeeper client library is 3.9.x. +Supported Zookeeper server version: 3.6+. The bundled ZooKeeper client library is 3.9.x. ```yaml cluster: diff --git a/oap-server/server-starter/src/main/resources/application.yml b/oap-server/server-starter/src/main/resources/application.yml index 089f6729bdd9..d38269cfd303 100644 --- a/oap-server/server-starter/src/main/resources/application.yml +++ b/oap-server/server-starter/src/main/resources/application.yml @@ -16,7 +16,7 @@ cluster: selector: ${SW_CLUSTER:standalone} standalone: - # Supported ZooKeeper server version: 3.5+. The bundled ZooKeeper client library is 3.9.x. + # Supported ZooKeeper server version: 3.6+. The bundled ZooKeeper client library is 3.9.x. zookeeper: namespace: ${SW_NAMESPACE:""} hostPort: ${SW_CLUSTER_ZK_HOST_PORT:localhost:2181} diff --git a/test/e2e-v2/cases/cluster/zk/banyandb/docker-compose.yml b/test/e2e-v2/cases/cluster/zk/banyandb/docker-compose.yml index 88d9a491199b..98ae67deb93b 100644 --- a/test/e2e-v2/cases/cluster/zk/banyandb/docker-compose.yml +++ b/test/e2e-v2/cases/cluster/zk/banyandb/docker-compose.yml @@ -15,7 +15,7 @@ services: zk: - image: zookeeper:3.5 + image: zookeeper:3.9 expose: - 2181 networks: diff --git a/test/e2e-v2/cases/cluster/zk/es/docker-compose.yml b/test/e2e-v2/cases/cluster/zk/es/docker-compose.yml index 319227a124ff..79c019f8e7b0 100644 --- a/test/e2e-v2/cases/cluster/zk/es/docker-compose.yml +++ b/test/e2e-v2/cases/cluster/zk/es/docker-compose.yml @@ -15,7 +15,7 @@ services: zk: - image: zookeeper:3.5 + image: zookeeper:3.9 expose: - 2181 networks: