Skip to content

Add source parameter to CONUS hydrology + arctic hydrology endpoints#720

Open
Joshdpaul wants to merge 12 commits intomainfrom
hydroviz_diff
Open

Add source parameter to CONUS hydrology + arctic hydrology endpoints#720
Joshdpaul wants to merge 12 commits intomainfrom
hydroviz_diff

Conversation

@Joshdpaul
Copy link
Copy Markdown
Contributor

@Joshdpaul Joshdpaul commented Apr 29, 2026

Summary

Both the CONUS hydrology and arctic hydrology stats coverages in Rasdaman have been updated (conus_hydro_segments_stats_combined and ak_hydro_segments_stats_combined) to include a source dimension encoding three variants of the data. This branch wires up that new dimension as a ?source= query parameter and simplifies the conus_hydrology/stats/{stream_id} and arctic_hydrology/stats/{stream_id} endpoints by moving the historical baseline adjustment math out of inline application code and into the coverage itself.

Changes:

  • New ?source= parameter on /conus_hydrology/stats/{stream_id}, arctic_hydrology/stats/{stream_id}, /conus_hydrology/modeled_climatology/{stream_id}, and /arctic_hydrology/modeled_climatology/{stream_id}. Accepted values are:
    • gcm_diff_applied_to_maurer (default for conus hydrology): GCM-projected changes applied to the Maurer historical baseline
    • gcm_diff_applied_to_cheng (default for arctic hydrology): GCM-projected changes applied to the Cheng historical baseline
    • original_gcm: raw GCM output
    • gcm_diff: ratio/absolute difference between projected and historical GCM (for stats endpoint only; not valid for climatology)
  • Hidden hydroviz app /conus_hydrology/hydroviz/{stream_id} and /arctic_hydrology/hydroviz/{stream_id} endpoints are simplified: the historical baseline adjustment calculation previously done inline is now handled by calculate_and_apply_gcm_diffs_to_maurer_climatology() or calculate_and_apply_gcm_diffs_to_cheng_climatology(), and is called from the modeled climatology endpoint when source is "gcm_diff_applied_to_maurer" (the conus default) or "gcm_diff_applied_to_cheng" (the arctic default). Note that this math is now optional depending on the choice of source: the user can get an un-adjusted hydrograph too by using source="original_gcm", if that's what they want.
  • ma99 removed from client-side calculation: annual mean flow is now stored in the coverage, so calculate_and_populate_annual_mean_flow() is deleted.
  • Source notes in metadata: variable descriptions in JSON and CSV output include a brief note describing what the values represent for the selected source. For the JSON, notes were included in each variable description to be very explicit about what data the user is getting. I did this because I don't think it's safe to assume anyone is reading a JSON completely top to bottom - they might be accessing only certain parts and so the redundancy might be useful. For the CSV, the note is included near the top of the metadata as I expect users will read the notes at the top before accessing values.
  • Application-level validation: ?source= values are validated via marshmallow in application.py.
  • Documentation: ?source= options and parameter chaining documented for both endpoints.

Note that the source notes (in variable metadata descriptions and CSV headers) and their documentation wording are first drafts. The language describing what each source mode means should be revised if there is a better way to clearly communicate the data provenance. Any ideas here are welcome, or we could workshop it with the rest of the team later.

XREF: ua-snap/hydroviz#184
XREF: ua-snap/rasdaman-ingest#157

XREF: ua-snap/arctic_rivers#4
XREF: ua-snap/rasdaman-ingest#158

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the CONUS hydrology API to support a new Rasdaman “source” dimension by exposing it as a ?source= query parameter, while simplifying hydroviz logic by moving Maurer-adjustment math out of inline endpoint code and into shared processing.

Changes:

  • Add ?source= support for /conus_hydrology/stats/{stream_id} and /conus_hydrology/modeled_climatology/{stream_id} (with gcm_diff_applied_to_maurer as default).
  • Update Rasdaman stats coverage wiring to conus_hydro_segments_stats_combined and add WCS subsetting for the source axis.
  • Add “source notes” into JSON variable descriptions and CSV metadata output; remove client-side ma99 derivation.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
templates/documentation/conus_hydrology.html Documents ?source= options for stats and modeled climatology endpoints.
routes/conus_hydrology.py Implements source handling, updates stats coverage, adds Maurer adjustment helper, and simplifies hydroviz aggregation.
generate_requests.py Adds optional SUBSET=source(...) to CONUS hydrology WCS request generation.
csv_functions.py Adds source-specific notes to CSV metadata and adds None-skipping in CSV row construction.
application.py Adds global marshmallow validation for source query parameter values.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread routes/conus_hydrology.py
Comment thread routes/conus_hydrology.py Outdated
Comment thread csv_functions.py
Comment thread csv_functions.py Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Agent-Logs-Url: https://github.com/ua-snap/data-api/sessions/3db7f7c2-1e57-4578-a221-88bb2156d8ec

Co-authored-by: Joshdpaul <99696041+Joshdpaul@users.noreply.github.com>
Joshdpaul added 2 commits May 4, 2026 16:35
…t mirroring conus hydrology endpoint structure; add source dimension

support; Cheng-baseline
delta adjustment for GCM models (PGW models ignored, use original_gcm
values); add source notes to CSV output.
@Joshdpaul Joshdpaul changed the title Add source parameter to CONUS hydrology endpoints Add source parameter to CONUS hydrology + arctic hydrology endpoints May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants