Skip to content

Preview images by default, require --upload#1251

Closed
jklare wants to merge 2 commits into
mainfrom
preview-by-default-require-upload
Closed

Preview images by default, require --upload#1251
jklare wants to merge 2 commits into
mainfrom
preview-by-default-require-upload

Conversation

@jklare

@jklare jklare commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What

Running openstack-image-manager (and osism manage images, which wraps it) previously imported images immediately when invoked without arguments. This meant the default action changed the cloud (uploading images and running outdated-image management) and required a working OpenStack connection just to see what the tool would do.

This change makes the default a safe, local-only preview. Without --upload the tool now:

  • lists the images that would be uploaded,
  • shows the total download size and a rough time estimate,
  • prints the exact command to actually perform the upload, and
  • links to the documentation.

It does not connect to OpenStack and makes no changes. Passing --upload restores the previous import and cleanup behaviour; --dry-run continues to work within that path.

Details

  • The preview is derived purely from the local image definitions, honouring the same enable/force/filter rules as a real run as well as --latest for images of type multi, and prefers mirror_url over url.
  • Download sizes are determined via HTTP HEAD requests (or the local filesystem for file: URLs); the time estimate assumes a rough 10-30 MB/s throughput range and is intentionally vague.

Example:

The following 2 image(s) would be uploaded:

  Ubuntu 24.04 (20260108)
  Ubuntu 24.04 Minimal (20260105)

Total download size: ~1.8 GiB
Estimated upload time: ~1-3 min (rough estimate, actual time depends on network and Glance backend)

No changes have been made. To actually upload these images, run:

  openstack-image-manager --upload --images etc/images/ubuntu.yml --latest

For more information and options, see:
  https://osism.tech/docs/guides/operations-guide/openstack/tools/image-manager/

Breaking change

This is a clean break: automation relying on the implicit upload must now pass --upload. The Zuul integration test is updated accordingly.

Important

The osism manage images wrapper lives in the osism/osism repository and must be updated there separately to pass --upload.

Testing

  • flake8 clean
  • Full unit suite passes (33 tests), including new tests for the preview helpers and an assertion that the preview never connects to OpenStack.

Running openstack-image-manager (and "osism manage images", which
wraps it) previously imported images immediately when invoked without
arguments. This meant the default action changed the cloud (uploading
images and running outdated-image management) and required a working
OpenStack connection just to see what the tool would do.

Make the default a safe, local-only preview instead. Without --upload
the tool now lists the images that would be uploaded, shows the total
download size and a rough time estimate, prints the exact command to
perform the upload, and links to the documentation. It does not
connect to OpenStack and makes no changes. Passing --upload restores
the previous import and cleanup behaviour; --dry-run continues to work
within that path.

The preview is derived purely from the local image definitions,
honouring the same enable/force/filter rules as a real run as well as
--latest for images of type multi, and prefers mirror_url over url.
Download sizes are determined via HTTP HEAD requests (or the local
filesystem for file: URLs); the time estimate assumes a rough
10-30 MB/s throughput range and is intentionally vague.

This is a clean break: automation relying on the implicit upload must
now pass --upload. The Zuul integration test is updated accordingly.
The osism manage images wrapper lives in the osism/osism repository
and must be updated there separately to pass --upload.

DocImpact
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jan Klare <klare@osism.tech>
@jklare

jklare commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Only unit tests were run so far, so this needs to be verified against a testbed before it can be merged.

mypy could not prove that the size-formatting loop always returns,
because the guaranteed exit relied on the final list element rather
than on a statement it could see. Iterate over the smaller units only
and return the TiB case unconditionally after the loop so the function
provably returns a str on every path.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jan Klare <klare@osism.tech>

@ideaship ideaship left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge-coordination note (not a code defect)

The > [!IMPORTANT] note in the PR body says the osism manage images wrapper "lives in the osism/osism repository and must be updated there separately to pass --upload." Two things worth tightening before merge:

  • The wrapper actually lives in osism/python-osism (the image_manager task in osism/tasks/openstack.py); I couldn't find an osism/osism repo. Until a python-osism change passes --upload, once python-osism bumps its pin to include this PR the osism manage images path (and the clusterapi/gardenlinux/octavia paths) will silently upload nothing.

  • This PR links the docs change but not the code change that has to land with it. The docs PR is referenced:

    ...but there's no corresponding python-osism PR linked. Since the wrapper change and this breaking change must merge together, please open and cross-link the python-osism --upload PR here so the two are tracked as a pair.

size /= 1024
return f"{size:.1f} TiB"

def build_upload_command(self) -> str:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — build_upload_command() drops behavior-affecting options.

It only echoes --cloud/--images/--filter/--latest, but a real --upload run is affected by many more options that silently vanish here. Two concrete failure modes:

  1. The suggested command uploads a different set than the preview showed. --force gates disabled images in read_image_files, and --tag X changes both the import tag and which cloud images count as managed. So a preview run with --force/--tag custom lists/plans images that the emitted command then processes differently — the tool prints "To actually upload these images, run: X" while X disagrees with the preview.
  2. The command performs different cleanup than the user configured. Because --upload also runs manage_outdated_images(), dropping --delete/--hide/--deactivate/--keep/--check-age/--max-age (and --no-check, --stuck-retry) means the suggested command won't do the cleanup the user selected while previewing.

A hand-maintained allowlist is inherently fragile — it already omits ~8 options, and every future CLI flag has to be remembered here or it drops out. Prefer reconstructing the actual invocation (echo the user's argv with --upload toggled on) so no behavior-affecting flag can silently disappear.

typer.echo(f"\nFor more information and options, see:\n {DOCS_URL}")
return

typer.echo(f"The following {len(planned)} image(s) would be uploaded:\n")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — preview claims every configured version "would be uploaded" without consulting Glance.

collect_planned_uploads reads only local YAML and never connects, but a real --upload run only imports versions missing from the cloud (process_image skips versions already present). For the documented day-2 use case — catalog already uploaded, re-running to catch new versions — a real run uploads nothing, yet this prints the full catalog with a large size/time estimate. The count, size, and time are all wrong for the most common invocation.

Since the no-connect design is intentional, the fix is wording, not connecting: frame it as configured candidates / an upper bound, e.g. "N image versions are defined and enabled; versions already present in the cloud will be skipped (up to ~X GiB if none exist yet)."

return None

try:
response = requests.head(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — serial 30 s HEAD probes on the new default path, and no HTTP status check.

Two issues:

  1. show_upload_preview calls this once per planned image, serially, each with REQUESTS_TIMEOUT = 30. This is now the default for every bare invocation (which previously did real cloud work rather than reaching out to N external mirrors). A few slow or blocked mirrors make the default command appear to hang for minutes. Consider a much shorter timeout for size estimation and/or concurrent probes, and maybe a "fetching sizes…" notice.
  2. There's no response.raise_for_status() before reading Content-Length, so a 404/403/proxy error page that carries a body length gets folded into the size total as if it were the image. Low severity (estimate-only, never affects the upload) but a genuine accuracy gap — add a status check before trusting the header.

@github-project-automation github-project-automation Bot moved this from New to In review in Human Board Jul 11, 2026
@jklare

jklare commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

Closing as not needed.

@jklare jklare closed this Jul 13, 2026
@github-project-automation github-project-automation Bot moved this from In review to Done in Human Board Jul 13, 2026
@jklare jklare deleted the preview-by-default-require-upload branch July 13, 2026 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants