Skip to content

Remove legacy client benchmark and noop API plugin modules#21839

Open
gingeekrishna wants to merge 7 commits into
opensearch-project:mainfrom
gingeekrishna:fix/21708-remove-legacy-client-benchmark
Open

Remove legacy client benchmark and noop API plugin modules#21839
gingeekrishna wants to merge 7 commits into
opensearch-project:mainfrom
gingeekrishna:fix/21708-remove-legacy-client-benchmark

Conversation

@gingeekrishna
Copy link
Copy Markdown

Description

Removes two legacy modules that have no active usage and are not referenced by the maintained opensearch-benchmark project:

  • client/benchmark — a REST client benchmark harness (RestClientBenchmark, bulk/search tasks, metrics)
  • client/client-benchmark-noop-api-plugin — a noop OpenSearch plugin providing synthetic /_noop_bulk and /_noop_search endpoints solely to support the above harness

Investigation findings:

  • No references to _noop_bulk, _noop_search, RestClientBenchmark, or client-benchmark-noop-api-plugin exist in opensearch-project/opensearch-benchmark
  • Neither module is executed by any CI workflow in this repository
  • The only external references were in settings.gradle, gradle/missing-javadoc.gradle, and .github/dependabot.yml — all removed in this PR

Files removed: 26 source/config files (2001 lines)

Build file changes:

  • settings.gradle — removed two module includes
  • gradle/missing-javadoc.gradle — removed two project references
  • .github/dependabot.yml — removed two dependabot entries

Related Issues

Resolves #21708

Check List

  • Functionality includes testing. Not applicable — pure deletion, no logic changed.
  • API changes companion pull request created, if applicable. Not applicable — the noop endpoints (/_noop_bulk, /_noop_search) are not part of the supported OpenSearch API surface.
  • Public documentation issue/PR created, if applicable. Not applicable — no user-facing documentation references these modules.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

The client/benchmark harness and client/client-benchmark-noop-api-plugin
have no references from the maintained opensearch-benchmark project and
are not executed by any CI workflow. Keeping them incurs ongoing build
and dependency maintenance cost with no active usage.

Remove both modules and clean up references in:
- settings.gradle
- gradle/missing-javadoc.gradle
- .github/dependabot.yml

Resolves opensearch-project#21708

Signed-off-by: Radhakrishnan Pachyappan <gingeekrishna@gmail.com>
Copilot AI review requested due to automatic review settings May 26, 2026 20:45
@gingeekrishna gingeekrishna requested review from a team, jed326 and peternied as code owners May 26, 2026 20:45
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Removes the legacy client benchmarking tooling (the client/benchmark app and the client-benchmark-noop-api-plugin) from the repository’s Gradle build and dependency automation configuration.

Changes:

  • Removes client:benchmark and client:client-benchmark-noop-api-plugin from the Gradle multi-project build.
  • Deletes the benchmark application sources/resources and the noop benchmark plugin sources/build files.
  • Updates Dependabot configuration to stop tracking the removed Gradle subprojects.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated no comments.

Show a summary per file
File Description
settings.gradle Removes the benchmark and noop benchmark plugin projects from the included Gradle projects list.
gradle/missing-javadoc.gradle Stops applying missing-javadoc configuration to the removed benchmark-related projects.
.github/dependabot.yml Removes Dependabot update entries for the deleted Gradle subprojects.
client/client-benchmark-noop-api-plugin/build.gradle Deletes the Gradle build definition for the noop benchmark plugin subproject.
client/client-benchmark-noop-api-plugin/README.md Removes documentation for the deleted noop benchmark plugin.
client/client-benchmark-noop-api-plugin/src/main/java/org/opensearch/plugin/noop/NoopPlugin.java Deletes the noop plugin entrypoint (plugin removed).
client/client-benchmark-noop-api-plugin/src/main/java/org/opensearch/plugin/noop/action/bulk/NoopBulkAction.java Deletes noop bulk action type (plugin removed).
client/client-benchmark-noop-api-plugin/src/main/java/org/opensearch/plugin/noop/action/bulk/RestNoopBulkAction.java Deletes noop bulk REST handler (plugin removed).
client/client-benchmark-noop-api-plugin/src/main/java/org/opensearch/plugin/noop/action/bulk/TransportNoopBulkAction.java Deletes noop bulk transport action (plugin removed).
client/client-benchmark-noop-api-plugin/src/main/java/org/opensearch/plugin/noop/action/search/NoopSearchAction.java Deletes noop search action type (plugin removed).
client/client-benchmark-noop-api-plugin/src/main/java/org/opensearch/plugin/noop/action/search/RestNoopSearchAction.java Deletes noop search REST handler (plugin removed).
client/client-benchmark-noop-api-plugin/src/main/java/org/opensearch/plugin/noop/action/search/TransportNoopSearchAction.java Deletes noop search transport action (plugin removed).
client/benchmark/build.gradle Deletes the Gradle build definition for the benchmark application subproject.
client/benchmark/README.md Removes documentation for the deleted benchmark application.
client/benchmark/src/main/resources/log4j2.properties Deletes benchmark logging configuration (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/AbstractBenchmark.java Deletes benchmark harness base class (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/BenchmarkMain.java Deletes benchmark CLI entrypoint (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/BenchmarkRunner.java Deletes benchmark runner/formatter (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/BenchmarkTask.java Deletes benchmark task interface (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/metrics/Metrics.java Deletes benchmark metrics DTO (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/metrics/MetricsCalculator.java Deletes benchmark metrics calculations (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/metrics/Sample.java Deletes benchmark sample model (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/metrics/SampleRecorder.java Deletes benchmark sample recorder (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/ops/bulk/BulkBenchmarkTask.java Deletes bulk benchmark task implementation (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/ops/bulk/BulkRequestExecutor.java Deletes bulk request executor interface (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/ops/search/SearchBenchmarkTask.java Deletes search benchmark task implementation (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/ops/search/SearchRequestExecutor.java Deletes search request executor interface (benchmark app removed).
client/benchmark/src/main/java/org/opensearch/client/benchmark/rest/RestClientBenchmark.java Deletes REST client benchmark implementation (benchmark app removed).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 26, 2026

PR Reviewer Guide 🔍

(Review updated until commit 4442941)

Here are some key observations to aid the review process:

🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@gingeekrishna gingeekrishna requested a review from Copilot May 26, 2026 20:49
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated no new comments.

@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 46a7c35

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 46a7c35: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Radhakrishnan Pachyappan <gingeekrishna@gmail.com>
@gingeekrishna
Copy link
Copy Markdown
Author

Gradle check build 78356 failed. After investigation, there are no remaining references to the deleted modules (client/benchmark or client/client-benchmark-noop-api-plugin) anywhere in the codebase — all build file references (settings.gradle, gradle/missing-javadoc.gradle, .github/dependabot.yml) have been removed, and no other module has a compile or runtime dependency on these modules.

This failure appears to be a flaky test unrelated to the PR changes. Retriggering the gradle-check now.

@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit c0a430d

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for c0a430d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…changes)

Build 78356 and 78421 failed. Verified no remaining references to
deleted modules exist anywhere in the codebase. PR opensearch-project#21838 (unrelated)
shows the same failure/pass/failure pattern in builds 78353/78354/78380,
confirming this is systemic CI flakiness.

Signed-off-by: Radhakrishnan Pachyappan <gingeekrishna@gmail.com>
@gingeekrishna
Copy link
Copy Markdown
Author

Build 78421 also failed. After further investigation, this is not caused by the PR changes.

Evidence of systemic CI flakiness:

  • PR #21838 (a completely unrelated rollover fix) shows the same intermittent pattern: build 78353 FAILURE → build 78354 SUCCESS → build 78380 FAILURE, all within the same time window as our failures.
  • Our PR is a pure file deletion — I have exhaustively verified there are no remaining references to the removed modules in any .gradle, .yml, .java, .groovy, .txt, .properties, or documentation file.

Retriggering gradle-check again.

@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 7f6e74b

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 7f6e74b: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@gingeekrishna
Copy link
Copy Markdown
Author

The latest CI run (26497576780) is failing with a different error — GitHub Actions receives 403 Forbidden from Jenkins when trying to trigger the gradle-check, meaning the build never starts. The run completes in under 1 minute with no test output.

This is a CI infrastructure issue (GitHub Actions ↔ Jenkins authentication), not caused by the PR changes. Could a maintainer please manually trigger the gradle-check, or confirm when the CI infrastructure is back up?

@cwperks @peternied

Signed-off-by: Radhakrishnan Pachyappan <gingeekrishna@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit a1a03ed

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for a1a03ed: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@gingeekrishna
Copy link
Copy Markdown
Author

CI Infrastructure Issue — Jenkins trigger token expired

After inspecting the GitHub Actions logs for runs 26497576780 and 26512183477, the root cause is clear:

The gradle-check.sh script's POST to /generic-webhook-trigger/invoke is returning 403 Forbidden from Jenkins. jq then fails to parse the HTML response as JSON and loops every 60 seconds until the 2-hour timeout is reached.

This is not caused by the PR changes. The same failure is occurring on at least PR #21845 as well (failed at 10:49 UTC today in under 6 minutes), indicating the Jenkins TRIGGER_TOKEN secret has expired or been invalidated as of May 27.

Stopping retrigger attempts until the CI infrastructure is restored. @cwperks @peternied — could you help rotate the Jenkins trigger token or confirm when CI will be back up?

@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 4442941

@github-actions
Copy link
Copy Markdown
Contributor

✅ Gradle check result for 4442941: SUCCESS

@codecov
Copy link
Copy Markdown

codecov Bot commented May 31, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.49%. Comparing base (dad63c0) to head (4442941).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #21839      +/-   ##
============================================
- Coverage     73.51%   73.49%   -0.03%     
+ Complexity    75582    75545      -37     
============================================
  Files          6034     6017      -17     
  Lines        342661   342245     -416     
  Branches      49294    49268      -26     
============================================
- Hits         251918   251533     -385     
- Misses        70712    70766      +54     
+ Partials      20031    19946      -85     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluate removing the core REST client benchmark and noop benchmark plugin

2 participants