Skip to content

DM-56116: Support lsst.images format for AP metric inputs #494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
isullivan wants to merge 1 commit into
base: main
Choose a base branch
from
+33 −4
Open

DM-56116: Support lsst.images format for AP metric inputs #494

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion pipelines/apDetectorVisitQualityCore.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ parameters:
# Note that these parameter names are shared with ApPipe, and are overwritten
# by changes in the values there.
coaddName: goodSeeing
# Whether the images this pipeline reads from are legacy afw Exposures or
# lsst.images types.
image_type: legacy

tasks:
analyzeAssociatedDiaSourceTable:
Expand Down Expand Up @@ -272,8 +275,9 @@ tasks:
analyzePreliminarySummaryStats:
class: lsst.analysis.tools.tasks.CalexpSummaryAnalysisTask
config:
connections.data: preliminary_visit_image.summaryStats
connections.data: preliminary_visit_image
connections.outputName: initialPviSummary
input_image_type: parameters.image_type
atools.initialPviSummaryMetrics: CalexpSummaryMetrics
python: from lsst.analysis.tools.atools import *
analyzeVisitSampledImageSubtractionMetrics:
Expand Down
31 changes: 28 additions & 3 deletions python/lsst/analysis/tools/tasks/calexpSummaryAnalysis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@
"CalexpSummaryAnalysisTask",
)

import dataclasses

import lsst.pex.config as pexConfig
from lsst.pipe.base import (
InputQuantizedConnection,
OutputQuantizedConnection,
Expand All @@ -42,16 +45,38 @@ class CalexpSummaryAnalysisConnections(
defaultTemplates={"inputName": "calexp.summaryStats", "outputName": "calexpSummary"},
):
data = cT.Input(
doc="Calibrated exposure summary statistics to load from the butler",
doc="Summary statistics of a visit image, which is a legacy Exposure or an "
"lsst.images.VisitImage depending on input_image_type. Only the parent dataset type is "
"taken from connections.data; input_image_type sets the component that is read from it.",
name="calexp.summaryStats",
storageClass="ExposureSummaryStats",
dimensions=("visit", "band", "detector"),
deferLoad=False,
)

def __init__(self, *, config=None):
super().__init__(config=config)

parent = self.data.name.split(".")[0]
if config.input_image_type == "future":
name = f"{parent}.summary_stats"
storage_class = "ObservationSummaryStats"
else:
name = f"{parent}.summaryStats"
storage_class = "ExposureSummaryStats"
self.data = dataclasses.replace(self.data, name=name, storageClass=storage_class)

@TallJimbo TallJimbo Oct 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing a connection dataset type rename after the config has been frozen is (AFAIK) new territory, and while I don't see a specific reason why it's unsound, that worries me. But given that you do need to do the storage class name override here (and that is an established pattern), let's give it a try. It might be that this a perfectly acceptable pattern we should use in other places, too.

@isullivan isullivan Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've expanded the docstring of the data input connection to clarify the behavior.



class CalexpSummaryAnalysisConfig(AnalysisBaseConfig, pipelineConnections=CalexpSummaryAnalysisConnections):
pass
input_image_type = pexConfig.ChoiceField[str](
"Which image type to read the summary statistics from.",
allowed={
"legacy": "Read ``summaryStats`` from an `lsst.afw.image.Exposure`.",
"future": "Read ``summary_stats`` from an `lsst.images.VisitImage`.",
},
optional=False,
default="legacy",
)


class CalexpSummaryAnalysisTask(AnalysisPipelineTask):
Expand All @@ -70,6 +95,6 @@ def runQuantum(

summary = inputs["data"]
if summary is None:
raise UpstreamFailureNoWorkFound("No summary stats attached to calexp.")
raise UpstreamFailureNoWorkFound("No summary stats attached to the input image.")
outputs = self.run(data=summary.__dict__)
butlerQC.put(outputs, outputRefs)
Loading