Add initial version of flexmeasures-openadr3 plugin#1
Conversation
|
Migrate to using ingestion redis queue |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
|
||
| OPENADR_EVENT_SOURCE_NAME = "OpenADR 3 VTN" | ||
| OPENADR_EVENT_SOURCE_TYPE = "gateway" | ||
| FETCH_EVENTS_QUEUE_NAME = "forecasting" # "ingestion" |
There was a problem hiding this comment.
update to ingestion redis queue name
| @@ -0,0 +1,10 @@ | |||
| FROM lfenergy/flexmeasures:v0.33.0 | |||
There was a problem hiding this comment.
Wait for flexmeasures v0.33.0 release before merging
| runs-on: ubuntu-latest | ||
| name: "Check (on Python3.11)" | ||
| steps: | ||
| - uses: actions/setup-python@v4 |
There was a problem hiding this comment.
Use hashes for action versions.
| on: push | ||
|
|
||
|
|
||
| jobs: |
There was a problem hiding this comment.
Optional: add zizmor for GH actions checking. See https://github.com/ElaadNL/openadr3-client/blob/main/.github/workflows/ci.yml
| python-version: 3.11 | ||
| - name: Check out src from Git | ||
| uses: actions/checkout@v3 | ||
| - run: make test |
There was a problem hiding this comment.
Does this work?
| version = "0.1" | ||
| dependencies = [ | ||
| "flexmeasures>=0.32.0", | ||
| "openadr3-client>=2.0.4", |
There was a problem hiding this comment.
Update openadr3-client version before release
| ] | ||
|
|
||
| # Existing tool configurations (keep these as-is) | ||
| [tool.pytest.ini_options] |
There was a problem hiding this comment.
Add
addopts = ["--import-mode=importlib"]As per https://github.com/zupo/awesome-pytest-speedup?tab=readme-ov-file
|
|
||
| @field_validator("utc_trigger_time") | ||
| @classmethod | ||
| def _validate_utc_trigger_time(cls, value: str, info: ValidationInfo) -> str: |
There was a problem hiding this comment.
Better to use a Pydantic validator for this
| field_name = str(issue["loc"][-1]) | ||
| errors.add(field_name, issue["msg"]) | ||
| else: | ||
| assert config is not None |
There was a problem hiding this comment.
Are inline asserts good here?
| ) | ||
| return cls(import_sensor=import_sensor, export_sensor=export_sensor) | ||
|
|
||
| def sensor_for(self, payload_type: EventPayloadType) -> Sensor | None: |
There was a problem hiding this comment.
get_sensor_for
|
|
||
|
|
||
| def _fetch_events_queue() -> Queue: | ||
| return cast(Queue, current_app.queues[FETCH_EVENTS_QUEUE_NAME]) |
There was a problem hiding this comment.
Shouldn't this be typed out of the box?
| """ | ||
|
|
||
|
|
||
| import warnings # NOQA: E402 |
There was a problem hiding this comment.
Why are these not at the top? Please add a comment
There was a problem hiding this comment.
I'd like some clarification about dependencies. I'm not sure what the uv.lock file is used for?
Maybe it is useful for running only this plugin, and it installs FlexMeasures + dependencies?
How compatible is that with running multiple plugins?
In our practical experience, we have a FlexMeasures docker image, and we might create a dedicated one with some custom plugin with some extra requirements installed. And we might install extra plugins (like this one) even at the deployment step (e.g. with ansible).
| ) | ||
| db.session.commit() | ||
| finally: | ||
| self.schedule(ven_client, sensor_config, replace_existing=False) |
There was a problem hiding this comment.
The chain of scheduling jobs would break if one job fails before this point. The finally safeguards against many sources of failure, but if a worker is restarted during job handling before reaching this point, that is problematic.
I'm flagging this as a potential follow-up to move to rq.cron.CronScheduler, which will require a dedicated rq-cron-scheduler service in docker-compose.yml running something like:
from rq.cron import CronScheduler
cron = CronScheduler(connection=app.redis_connection)
cron.start()
And then registering recurring jobs with something like:
cron.register(
self._execute,
queue_name=app.queues[FETCH_EVENTS_QUEUE_NAME],
cron_string="0 0 * * *",
kwargs={"ven_id": ven_client.id, "config_name": sensor_config.name},
)
| ] | ||
| version = "0.1" | ||
| dependencies = [ | ||
| "flexmeasures>=0.32.0", |
There was a problem hiding this comment.
| "flexmeasures>=0.32.0", | |
| "flexmeasures>=0.33.0", # introduces the ingestion queue |
4 participants
No description provided.