feat: allow setting a consumption/production sensor in the flex-model of an asset#2190
Conversation
… patching Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… DBStorageFlexModelSchema Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Context: - StorageScheduler could already write state-of-charge schedules to a secondary sensor - Users wanted the same for consumption and production power Change: - Add StorageScheduler._build_consumption_production_schedules() static method - Call it in compute(), with resampling and rounding matching the soc_schedule pattern - If only consumption sensor defined: full power profile (consumption positive, production negative) - If only production sensor defined: full power profile inverted (production positive, consumption negative) - If both defined: split — non-negative part to consumption sensor, sign-flipped non-positive part to production sensor - Include results in return_multiple output as consumption_schedule / production_schedule entries - Sign convention is encoded in the key name so no consumption_is_positive attribute is needed
…ut sensors Context: - StorageScheduler now supports writing schedules to consumption and production sensors Change: - test_battery_solver_multi_commitment: add consumption and production output sensors to the battery, include them in the flex-model, and verify unit conversion (MW → kW) and the split logic (all-positive schedule → consumption all positive, production all zero) - test_trigger_schedule_uses_state_of_charge_sensor_for_soc_at_start: add production output sensor and verify 96 beliefs are stored after scheduling - test_add_storage_schedule_uses_state_of_charge_sensor_for_soc_at_start: add consumption output sensor and verify 48 beliefs are stored after scheduling
… changelog entry Context: - StorageScheduler now writes schedules to consumption/production sensors Change: - Expand CONSUMPTION and PRODUCTION metadata descriptions with the split logic (only consumption, only production, or both defined) and clarify that the sign convention is encoded in the key name (no consumption_is_positive attribute needed) - Add changelog entry in v0.33.0 New features section (PR number TBD)
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…sor-in-db-flex-model
Fixes #2084 Context: - StorageScheduler._prepare crashes with AttributeError when a device in the asset tree has no sensor in its flex-model (only power-capacity) - Lines 672 and 740 already guard sensor_d against None, but line 902 was missed Change: - Add 'sensor_d is not None' check before accessing event_resolution - Matches the existing pattern used elsewhere in the same method Signed-off-by: F.N. Claessen <claessen@seita.nl>
…x-model # Conflicts: # documentation/changelog.rst
…o the scheduling resolution Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ution Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…roduction output schedules The StorageScheduler already applies correct sign conventions when returning consumption/production schedules (e.g., production values are inverted to be positive). However, the persistence layer was then applying the default power sensor sign logic again, negating this carefully applied inversion. This resulted in production sensors receiving negative values when they should receive positive values, and vice versa for consumption sensors. Solution: Skip the default sign inversion logic for consumption and production output schedules (identified by result["name"] being "consumption_schedule" or "production_schedule"), as their sign convention is already correctly encoded by the scheduler.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Context:
- The endpoint always returned consumption-positive values, giving callers
no way to request production-positive or raw database values.
Change:
- Add ScheduleSignConvention constants class to scheduling schemas.
- Extend GetScheduleSchema with a sign-convention field (default:
consumption-positive) validated against the three allowed modes.
- Update get_schedule to apply the chosen convention:
* consumption-positive: invert DB values when consumption_is_positive=False
* production-positive: invert DB values when consumption_is_positive=True
* wysiwyg: return raw database values unchanged
- Regenerate openapi-specs.json with the new parameter.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
feat: add sign-convention query parameter to get_schedule endpoint
Context:
- The endpoint always returned consumption-positive values, giving callers
no way to request production-positive or raw database values.
Change:
- Add ScheduleSignConvention constants class to scheduling schemas.
- Extend GetScheduleSchema with a sign-convention field (default:
consumption-positive) validated against the three allowed modes.
- Update get_schedule to apply the chosen convention:
* consumption-positive: invert DB values when consumption_is_positive=False
* production-positive: invert DB values when consumption_is_positive=True
* wysiwyg: return raw database values unchanged
- Regenerate openapi-specs.json with the new parameter.
Context: - The new sign-convention query parameter needs user-facing documentation. Change: - notation.rst: replace single-sentence note with a bulleted list describing all three modes (consumption-positive, production-positive, wysiwyg). - data-model.rst: expand the signs_of_power_beliefs section similarly. - api/change_log.rst: add entry for the new parameter in v3.0-31.
Context:
- Both sign-convention test functions only covered the default
consumption-positive convention (no sign-convention parameter was passed).
Change:
- Add sign_convention as a parametrize dimension (3 values) to both
test_get_schedule_sign_convention_json_flex_model and
test_get_schedule_sign_convention_db_flex_model (2 × 2 × 3 = 12 cases each).
- Extract _assert_schedule_sign_convention() helper with a documented
decision table for expected sign vs the consumption-positive reference:
consumption + consumption-positive → same sign as main
consumption + production-positive → opposite sign
consumption + wysiwyg → same sign (DB is consumption positive)
production + consumption-positive → same sign
production + production-positive → opposite sign
production + wysiwyg → opposite sign (DB is production positive)
- Add force-new-job-creation to the JSON flex-model trigger to avoid Redis
job-cache collisions across parametrized runs.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… database values, rather than 'raw database values', and stop using the term 'reflect', which itself suggest a sign flip Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…e_schedule for all output sensor cases Context: - Previously _build_consumption_production_schedules flipped the sign for the production-only case (-power_series) and for the both-sensors case ((-power_series).clip(lower=0)), forcing _resolve_schedule_output_sign to bypass its inversion logic for dedicated output sensors. Change: - Production-only case: pass power_series unchanged (consumption positive); make_schedule inverts via consumption_is_positive=False on the sensor. - Both-sensors case: clip production to power_series.clip(upper=0) (the non-positive part, still in consumption-positive convention); make_schedule inverts via the same attribute. - Documentation in the docstring updated to reflect the new flow.
… and simplify sign resolution Context: - The _set_output_sensor_consumption_is_positive safety-net ran only inside make_schedule (after scheduling), too late to surface attribute conflicts. - _resolve_schedule_output_sign special-cased dedicated output sensors with an early return of 1, which was only correct because _build_consumption_ production_schedules was flipping signs itself. Change: - Add _set_flex_model_output_sensors_consumption_is_positive() that iterates the deserialized flex model and assigns consumption_is_positive to each output sensor (True for consumption, False for production), raising ValueError immediately on conflict. - Call it in create_scheduling_job() right after deserialize_config(), so attribute conflicts surface as 422 responses before any job is enqueued. - Simplify _resolve_schedule_output_sign(): remove the _is_consumption_ production_output() early-return and let it fall through to the consumption_is_positive attribute check that already handles all cases. - Retain the _set_output_sensor_consumption_is_positive() call inside make_schedule as a safety net for direct invocations (moved before save_to_db for fail-fast behaviour).
Context: - Now that consumption_is_positive conflicts are detected at job-creation time, the trigger endpoint returns 422 immediately instead of creating a job that later fails. Change: - Rename test to test_conflicting_consumption_is_positive_attribute_prevents_job_creation. - Remove RQ job execution and job.is_failed check. - Assert trigger_response.status_code == 422 and that the error message contains 'consumption_is_positive'. - Fix assertion to use str() around response.json['message'] because the unprocessable_entity helper nests the error under a 'json' key.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
|
@saerts-gp care to take a look at the documentation changes? Specifically, the first 5 files of this PR's diff. I'd be interested to know if this clarifies the sign conventions in FlexMeasures, and whether you'd feel more empowered after merging this PR. |
There was a problem hiding this comment.
Thank you for adjusting the documentation, this certainly clarifies the current functionality of the code a lot more!
Reading this clarification, I'm wondering what the power sign of a storage device which is both positive and negative would be. It's also neither the consumption nor production of that device. Strictly speaking, I could map the consumption of a storage device to the over-time losses or conversion (heat-dissipation) .
Using FlexMeasures, we will on our end always translate to one single power sensor, where the sign indicates the meaning thereof.
I understand your point. Are there different terms you would consider? Perhaps feed-in and feed-out could be mentioned in the field description, or some other clarifying term. We've tried to consistently use consumption and production throughout FlexMeasures to denote a directional quantity of energy flowing at a given point in the system. Without presuming to know what goes on under it, we abstract away from what happens behind that connection point. For instance, an AC Charge Point puts most of the power it draws from the grid into an EV, with only a small percentage lost within the Charge Point components. We consider the Charge Point consumption not to be those heat losses, but to be the overall power drawn from the grid. Likewise for a bi-directional Charge Point. We claim it produces electricity, while in fact the Charge Point itself only has heat losses.
Good. So I think then with this PR you'll have the right knobs to tune in order to get what you need. For instance:
|
nhoening
left a comment
nhoening
left a comment
There was a problem hiding this comment.
Sending in some questions, not sure I grok it 100% yet, especially the automatic setting of the attribute.
If I set it manually, it will not be overwritten?
Exactly. The API caller will immediately get back an error when:
|
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
nhoening
left a comment
nhoening
left a comment
There was a problem hiding this comment.
My only comments are about clarity of concepts.
A question: users can now separate power readings/forecasts and schedules on different sensors. In which situations is this advisable, versus using one for everything?
…nd what the scheduler did Signed-off-by: F.N. Claessen <claessen@seita.nl>
Good question. I guess sharing a single sensor for all forces the data to have the same unit, resolution and sign convention. It represents different sources forming beliefs about the same events. However, one could also prefer to store these things on distinct sensors. Then you don't have to deal with filtering by source. |
Is there a good spot to add this in the docs? We might also follow up with a new section on what sensors a schedulable asset might have, i.e. an example somewhere near getting-started |
3 participants
Description
consumptionandproductionflex-model fields for saving power schedules on a sensor with an explicit sign convention.consumption-positive( the default, even when fetching a schedule from a production sensor)production-positivewysiwyg(what-you-see-is-what-you-get): return the values with the same sign as database values and what is seen in UI chartsstorage-efficiencyfield references a sensor, resampling happens from the resolution of the given sensor to the scheduling resolution. This used to be resampled from the resolution of the power sensor.storage-efficiencyfield is a fixed quantity, resampling happens from the resolution of the power sensor to the scheduling resolution. If the power sensor is missing, the consumption sensor is used. If that is missing, too, the production sensor is used. If that is missing, too, no resampling happens, so the storage-efficiency is interpreted as having a resolution equal to the scheduling resolution.consumption_is_positiveattribute is already defined on the sensor and points the wrong way for the given field.documentation/changelog.rstLook & Feel
How to test
pytest -k test_trigger_schedule_uses_state_of_charge_sensor_for_soc_at_startextended to cover production-sensor.pytest -k test_add_storage_schedule_uses_state_of_charge_sensor_for_soc_at_startextended to cover consumption-sensor.pytest -k test_battery_solver_multi_commitmentextended to cover both fields.Further Improvements
The scheduling service currently sets the
consumption_is_positiveattribute on the consumption sensor toTrueand on the production sensor toFalse. This way, theget_schedulesensor endpoint knows whether to flip the sign when creating a response (this endpoint always returned consumption as positive and production as negative values, but now lets the user switch to a different sign convention). I investigated whether storing the flex-config in the data source attributes is a better way to persist the sign definition (#2205). I think it may be better kept as a sensor attribute for now than moving it to a data source attribute. Arguments:consumption_is_positiveattribute on the data source, so we only get 2 extra sources.consumption_is_positiveattribute set on the sensor, the sign convention is already taken into account by theStorageScheduler. The alternative of encoding the sign convention in the data source would require additional logic when the data is used as a scheduling input. Likewise for using the sensor data in the profit or loss reporter.These are all issues that can be overcome, but I prefer not to increase the scope of this PR any further.
Related Items