Skip to content

Add Singularity definitions for DataStreamCLI and dependencies #35

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hariteja-jajula wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add Singularity definitions for DataStreamCLI and dependencies #35

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2871264
Add Singularity definitions for DataStreamCLI and dependencies
hariteja-jajula Jan 9, 2026
1d32a75
Sync fork with upstream CIROH-UA/datastreamcli main
hariteja-jajula Apr 8, 2026
debc8ed
Fix pip3 -> python3 -m pip, version-pin image tags from versions.yml,…
hariteja-jajula Apr 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions singularity/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.sif
Copy link

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new singularity/ directory lacks a README.md file to document its purpose, usage, and requirements. The docker/ directory has a README.md (docker/README.md) that documents the Docker setup. Following this pattern, a singularity/README.md should be added to explain how to build and use the Singularity images, what each image is for, system requirements (e.g., Singularity/Apptainer version), and how to run DataStreamCLI using the built containers. This is especially important since the PR description mentions this is targeting HPC environments where Singularity is commonly used.

Copilot uses AI. Check for mistakes.
23 changes: 23 additions & 0 deletions singularity/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash
set -e

SCRIPT_DIR=$(dirname "$(realpath "$0")")
VERSIONS_FILE="${SCRIPT_DIR}/../versions.yml"

DEPS_VERSION=$(grep 'datastream-deps' "$VERSIONS_FILE" | sed 's/.*: *"\(.*\)"/\1/')
DS_VERSION=$(grep '^datastream:' "$VERSIONS_FILE" | sed 's/.*: *"\(.*\)"/\1/')

echo "Building DataStreamCLI Singularity images..."
echo " datastream-deps version: ${DEPS_VERSION}"
echo " datastream version: ${DS_VERSION}"

sed "s|^From: awiciroh/datastream:.*|From: awiciroh/datastream:${DS_VERSION}|" \
datastream.def > datastream_pinned.def

singularity build --fakeroot datastream.sif datastream_pinned.def
rm datastream_pinned.def

singularity build merkdir.sif docker://zwills/merkdir:latest
singularity build ngiabinabox.sif "docker://awiciroh/ciroh-ngen-image:v${DS_VERSION}"

echo "All Singularity images built successfully."
13 changes: 13 additions & 0 deletions singularity/datastream.def
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Bootstrap: docker
From: awiciroh/datastream:latest

%files
../ /datastreamcli
Copy link

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line copies the entire parent directory (including .git/, .github/, tests/, docs/, docker/, etc.) after already copying specific subdirectories in lines 5-8. This creates redundancy and significantly increases image size. Additionally, the scripts/ directory is not explicitly copied in lines 5-8, but it's required based on the PR description's testing showing '/datastreamcli/scripts/datastream'. Consider either removing this line and adding 'scripts/' to the explicit file list, or removing lines 5-8 and only using this line if all files are needed.

Suggested change
../ /datastreamcli
../scripts/ /datastreamcli/scripts/

Copilot uses AI. Check for mistakes.

%post
python3 -m pip install -e /datastreamcli && \
python3 -m pip install pytest


%runscript
exec /datastreamcli/scripts/datastream "$@"