Skip to content

To main#274

Merged
leerho merged 15 commits intomainfrom
ToMain
Mar 3, 2026
Merged

To main#274
leerho merged 15 commits intomainfrom
ToMain

Conversation

@leerho
Copy link
Member

@leerho leerho commented Feb 24, 2026

This /main/ branch is basically a Java 25 extension of the DS-Memory 6.0.0 API, which required Java 21.

You must use Java 25 to compile and run this, but the API is very similar to the API used in DS-Memory 6.0.0.

These updates to /main/ do the following:

  • Added a @BeforeSuite "test" so that when running tests, this function will print out the Java version and the location of the JVM being used for the compile and test.
  • Added a testng.xml file which allows TestNG to be in control of the overall test suite.
  • Made considerable updates to the pom.xml to modernize and clean-up the pom. This pom now is virtually identical to the DS-Java pom, with the normal differences in identities, and a few unique capabilities of the DS-Java environment that are not required here.

I am moving to a new CI Architecture to reduce duplication of the GHA workflows across main and the release branches. It is taking advantage of the "Reusable Workflows" feature of GHA. Right now it is only implemented for the "auto-jdk-os-matrix" workflow. If successful, we can extend it to other workflows and also to other repositories.

Technical Advantages of the New CI Architecture

Single Source of Truth: Centralizing logic in the workflows branch eliminates the need to "cherry-pick" CI fixes across multiple legacy release branches (Java 8, 11, 17, 21, 25, etc.).

Multi-Branch Versioning: The reusable-jdk-os-matrix.yml uses a dual-checkout strategy. It pulls the specific Java source code from the target branch while simultaneously pulling the latest CI logic from the workflows branch.

Architecture-Aware Testing: The new matrix logic explicitly passes java_arch, ensuring compatibility as the project transitions toward ARM-based runners (AArch64) alongside standard x64.

Fail-Safe Observability: Debugging steps (like printing the current workflow) use continue-on-error: true. This prevents non-critical "print" failures from blocking the primary Maven test results.

Reduced CI Technical Debt: By decoupling the parameters (JDK version, OS) in the caller from the execution (Maven lifecycle) in the reusable engine, we ensure that infrastructure updates (like migrating to newer GitHub Actions versions) only need to be performed once.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR modernizes the project’s test execution and Maven build configuration by moving TestNG execution under an explicit suite definition and updating build/plugin settings in pom.xml.

Changes:

  • Add a testng.xml suite file and configure Surefire to run tests via that suite.
  • Ensure a @BeforeSuite hook always runs to print the test JVM/version details.
  • Modernize/align pom.xml plugin versions and build configuration; remove the VS Code workspace file from the repo.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/test/resources/testng.xml Introduces a TestNG suite definition used to drive test discovery/execution.
src/test/java/org/apache/datasketches/memory/internal/A_BeforeSuite.java Forces the environment-printing @BeforeSuite hook to always run.
pom.xml Updates Maven/TestNG/plugin versions, refactors JVM arg properties, and wires Surefire to testng.xml.
DS-Java-Memory-24.code-workspace Removes an editor workspace file from version control.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Member

@tisonkun tisonkun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. One comment to reduce unnecessary indirection if it is.

arch: aarch64

# Reference the reusable workflow on the "workflows" branch
uses: apache/datasketches-memory/.github/workflows/reusable-jdk-os-matrix.yml@workflows
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can simply inline the reuseable workflows at the main branch. Either under .github/workflows or under a new directory.

Copy link
Member Author

@leerho leerho Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to GitHub Docs, the reusable workflows (RWFs) can have the location syntax:
{owner}/{repo}/.github/workflows/{filename}@{ref} #in a different repo
OR
./.github/workflows/{filename}@{ref} #in the same repo
where {ref} can be a branch, tag, or gitHash.

I would prefer having the RWFs in a separate branch ... or a common repo ... hmm...
I have it in a separate branch here, but these RWFs would be identical to what we would need for DS-Java.
Perhaps a better idea would be to put these in a "workflows" branch on DS-Java. It would be a natural common place for all the various java repos. Since trying to share RWFs across different languages may be too cumbersome.

Actually, I'd prefer if you would approve it here so I can test it. And then if it works, I'll move the "workflows" directory to DS-Java, where it makes more sense and can be leveraged by other DS-"java" repos, like hive, pig, characterization, vector, server, etc (that is, if we want to upgrade them :) )

@leerho leerho requested a review from tisonkun March 2, 2026 23:40
Copy link
Member

@tisonkun tisonkun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go ahead for testing.

I remember that the reusable workflow can be written as:

uses: ./.github/workflows/test_behavior_binding_java.yml

... and thus use the yaml definition in the same checkout (of the same commit). Note that leading ./.

See https://github.com/apache/opendal/blob/01ce4ac16e6ec6689f4883f7294633051f9b2c3f/.github/workflows/test_behavior.yml#L118 as a reference. In this way we can maintain all the workflows under the same branch and keep it updated.

@leerho leerho merged commit 84a546c into main Mar 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants