Skip to content

Scenario data read-path fixes and dataset_id index - #311

Merged
knoxsp merged 3 commits into
masterfrom
perf/tier1-scenario-data
Oct 23, 2026
Merged

knoxsp merged 3 commits into
masterfrom
perf/tier1-scenario-data

Conversation

@knoxsp

@knoxsp knoxsp commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Index tResourceScenario.dataset_id. The migration only creates the index when none already leads with that column, since InnoDB indexes the FK automatically.
  • get_scenario: new include_values flag (default True) to skip the large tDataset.value column when only structure is needed.
  • Scenario.get_data: keep include_results, data-type and value filters when recursing to the parent scenario. They were previously dropped.
  • compare_scenarios: pass get_parent_data (include_parent_data was silently ignored by get_scenario) and skip value comparison when both sides reference the same dataset row.
  • update_resource_attribute: set the mapped attr_is_var column. The previous is_var assignment was a no-op.

Behaviour changes

  • compare_scenarios now includes inherited parent data, as its docstring describes, so diffs for child scenarios may differ from before.
  • update_resource_attribute now actually changes the flag.

Testing

tests/test_scenario.py (37) and tests/attributes (63) pass on the SQLite test backend. No MySQL run, so the migration is untested against MySQL.

Not included

Batching update_resourcedata, the dedup hash change and dataset garbage collection are left for follow-up changes.

- Index tResourceScenario.dataset_id (migration only creates it if no
  existing index leads with that column; InnoDB already indexes the FK).
- get_scenario: add include_values flag so callers can skip the large
  tDataset.value column.
- Scenario.get_data: keep include_results/type/value filters when
  recursing to the parent scenario instead of silently dropping them.
- compare_scenarios: pass get_parent_data (include_parent_data was
  ignored by get_scenario) and skip value comparison when both sides
  reference the same dataset row.
- update_resource_attribute: set the mapped attr_is_var column; the
  previous is_var assignment was a no-op.
Scenario.get_data, get_all_resourcescenarios and lib.scenario.get_scenario
accept an optional attr_ids list, applied as ResourceAttr.attr_id IN (...)
in the query, so callers needing a few attributes no longer load and
build every resource scenario.
Keep this branch's scenario data read-path parameters, which include the
attr_ids filter that master already has from #313, and re-parent the
dataset_id index migration onto master's head so there is a single head.
@knoxsp
knoxsp merged commit 082e317 into master Oct 23, 2026
3 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants