Skip to content

migrid-ux based peers ui and all supporting logic#453

Open
albu-diku wants to merge 2 commits into
nextfrom
feature/admin-ui
Open

migrid-ux based peers ui and all supporting logic#453
albu-diku wants to merge 2 commits into
nextfrom
feature/admin-ui

Conversation

@albu-diku
Copy link
Copy Markdown
Contributor

@albu-diku albu-diku commented Feb 11, 2026 •
edited
Loading

@albu-diku albu-diku force-pushed the feature/admin-ui branch 3 times, most recently from 910ecac to 3c361ed Compare February 24, 2026 19:29
@albu-diku albu-diku force-pushed the feature/admin-ui branch 4 times, most recently from 1b0ef8a to d94b039 Compare March 30, 2026 14:53
@albu-diku albu-diku changed the title WIP: admin UI migrid-ux based peers ui and all supporting logic Mar 31, 2026
@albu-diku albu-diku marked this pull request as ready for review April 13, 2026 13:33
Martin-Rehr
Martin-Rehr reviewed Apr 17, 2026
View reviewed changes
Comment thread mig/shared/configuration.py
Martin-Rehr
Martin-Rehr reviewed Apr 17, 2026
View reviewed changes
Comment thread mig/lib/templates/__init__.py
@Martin-Rehr
Copy link
Copy Markdown
Contributor

Martin-Rehr commented Apr 17, 2026

I think this PR should be split into three PRs:

  1. Core configuration changes (introduction of RuntimeConfiguration)
  2. Core templating functionality (introduction of ninja templates)
  3. Peers specific functionality

@albu-diku
Copy link
Copy Markdown
Contributor Author

albu-diku commented Apr 17, 2026 •
edited
Loading

I think this PR should be split into three PRs:

1. Core configuration changes (introduction of RuntimeConfiguration)

2. Core templating functionality (introduction of ninja templates)

3. Peers specific functionality

Will get right too that! :)

Let me share a little context: the separation you noted is definitely there. Specifically, RuntimeConfiguration and ninja templates were separate PRs originally that were folded into this as necessary to support and undertake the rest of the work.

So with that said, it should not actually be too difficult.

  • RuntimeConfiguration: it is here and was to a large extent reviewed by @rasmunk already.
  • Same goes with templates, will dust off the old PR (reopening if needed)

The only effort that may be needed, particularly in the latter case, is bring the separated versions in-sync with what's in this branch. There was some iteration and evolution on those bits in the course of rest of the work. I am much much happier with what's here, so it's basically copy the files as they are from this branch into separate branches and push those up.

Thanks for the first look.

@albu-diku
Copy link
Copy Markdown
Contributor Author

albu-diku commented Apr 19, 2026

I think this PR should be split into three PRs:

1. Core configuration changes (introduction of RuntimeConfiguration)

2. Core templating functionality (introduction of ninja templates)

3. Peers specific functionality

Have now done this - PR description is updated with links to those this work dependent upon.

This was referenced Apr 24, 2026
Open
Closed
albu-diku added 2 commits May 19, 2026 14:51
@albu-diku
Introduce support for jinja2-based templates.
3549c0e 
This adds the necessary logic both to maintain a store of templates
and making them available for serving within any page via a new
template output object type.

The template type is wired into the output path. The existing path for
object validation check the arguments passed to the template only, so
so a specific addition is made to the validation code to ensure validity
of the template object itself. In the case of a requested template being
missing a specific exception is thrown.

Make sure there is a separation of the test templates from the mainline
stuff by wiring a TEMPLATES section into Configuration and reading the
necessary paths via the loaded configuration. Doing so allows both
source location and the cache directory to be specified via the main
ini file and the values are made runtime accessible without hard-coding.
Use this facility within the tests to adjust the paths. This config
section is exposed to the command line similarly to other values.

In the process of doing this work a test was added to checckonf due to
some changes being nearby. In doing so it was found checkconf would not
successfully complete against an empty file, so this is fixed here also
to allow attaining coverage of all code affecting the use of templates.
@albu-diku
Implement support infrastructure for peers ui.
9333d0d 
Original commit messages:

Cover peersactionwith a basic test.

Heavily extend (any|every)thing peer related within accountreq.

Add direct listing functions for pending and accepted peers.

Test the creation of the pending_peers file via picklesupp. Do so
using a raw comparison of the pending peers file

Introduce a pending peers fixture and assert that manage_pending_peers
produces what is in it then consume it
when testing the pending peers listing function.

make common the identical parts of ONWRITE and ONREAD hints selection

Cnvert template support code to package loading.

* expand template packages up-front in TemplateStore.populated()
* template packages is UNPOPULATED by default
* back template listing onto back_packages and rejig some

The following from original commit was addressed by squashes:

template loading and rendering works, but listing templates needs to be
restored as does priming ...am most of the way convinced that per-package
cache dirs want to be done as part of that preparation process and not a
call to os.makedirs() when we do an average "get a template store" call

do all template cache directory ahead of time during priming

explicitly create configuration for testing of tempate defaults

do this to avoid any changes spilling over - this seemed to be happening
with the previous phrasing in the tests though it is not entirely clean
to me how it can have happened given how the test support code should be

repair support for html fragments in the output path and test the results

Expose template rendering via a handler.

fixup: ensure tmplinterface works when called via the cgi-bin

serve html from tmplinterface

make tmplinterface accessible via cgi

Wire the ability to specify a template cache dir through shared/install.

Allow a value of AUTO for the option which, when read, will construct
the necessary path when the configuration is loaded.

avoid config loading reaching out for log file in the templates cli

template cli: repair prime and restore a "cache" command. add tests

rework the tests added for the CLI

repair the first peersaction test

Place the priming code into a method attached to TemplateStore.

rework tmplinterface

Get tmplinterface returning accepted peers and cover with real data.

Get requested peers being returned via tmplinterface.

switch tmpl interface tests to snapshots

add datainterface

get to a working, JSON **responding**, datainterface

datainterface - implement /peer/new and alwats return JSON with a status

Implement the deletion of accepted peers.

restore test suite to full function (also meaning peersaction itself)

Handle peer deletion as a bulk POST to a .../delete endpoint.

Unify the lack of a need to prefix routes between (data|tmpl)interface.

Define peers UI backend routes as though overrides for the migux package.

update datainerface documentation

Pull out much of the finer grained user test support code into a usersupp.

Place the error key at the top level and nest a data key in JSON responses.

Decide a location for peers app assets and reference its bootstrap.

implement the peers summary endpoint

Set only the necessary content-type header across (data|tmpl)interface

Enforce a header check when expecting JSON responses via content_format.

Remove unused import.

Remove repetetive boilerplate from all (data|tmpl)interface tests.

Switch provisioning of pending peers over to taking a DN argument.

WIP peer acceptance

fix logger local in accountreq

fix condtional attr access in shared/configuration

fix condtional attr access in datainterface

fixup - shared/configuration

Implement deletion of requested peers.

Provide well contained user peers and user pending peers assertions.

We already tarted to arrange this for pending peers, but take a moment to
do this for accepted peers as well. As "User" as an infix in the assertions
to avoid any potential confusion and contain the argless hint application
to the loaded pickle files within the two assertion functions.

Remove the pending user entry when we reject the peer.

rename test support pending user retrieval

Cover peer creation and removal of the pending user file on accept.

Get migux package installed automatically across all local invocations.

replace calling out to wget with download via urllib3

replace calling out to unip with the use of zipfile

urllib3 in local reqs

Axe unused tests and improve comments in datainterface.

restore wget fetch but tie its use to an argument

this is done to allow operation in an environment without urllib3

Move to a feature specific requirement file for migux.

support writing the indexurls to a file

fixup

Always use fresh packages and allow for postinstall (add one for migux).

Be more straightforward when setting templates values from a config.

Make config file arugment required in templates CLI.

Arrange covering empty templates base packages from config and fix it.

enable migux by default

Add a comment for a new commit id.

Nudge to latest version.

Nudge to latest version.

Switch local package installation to a single local file url.

Nudge to latest version.

Nudge to latest version.

Attempt to persuade dumb-pypi to execute within GitHub actions.

Restore fixturesupp chunk lost to rebase.

Remove duplication due to rebase.

Axe wrapper properties for config vars within accountreq tests.

Axe remnants of earlier iterations from tmplinterface.

Fix some fallout from splitting peersaction.

try to account for image differences in CI

maybe?

Are we hitting differences between the CI targets?

Ensure the base now date will always allow month replacement.

(cherry picked from commit c628b6c)

Avoid a variable that can be two types to appease linter.

Avoid a variable that can be two types to appease linter.

Align some request handling basics across data and tmpl interfaces.

alignment 2/x

repair invoke in local dev environments

Remove arg fiddling in tmplinterface and use explicit tmpl route variable.

Fix tmplinterface test case name.

Rework template handlers for direct return of render info.

Use "staging" as the name for the dir serving that purpose within envhelp.

Axe the need for urllib3.

Repair "Rework template handlers for direct return of render info."

A fat fingered convert key broke processing fields.

Restore filtering of peer listings.

Repair make unittest after 'Use "staging" as the name for the dir...'.

Further refinement to plugin interface and a bunch of polish.

Relocate RuntimeConfiguration to align with further iterations.

strip out the _send_email function pointer

Align to RuntimeConfiguration as a proxy object.

Rework email send as a reference in context and provide its fake.

Use the runtime configuration and its persistent context as the means
to expose the email sending logic to codepaths making use of it. This
allow centralised control over the active email sending implementation
without having to thread extra arguments through internal codepaths.

As a result, we are able to centrally intercept email sending. Do this,
and as a consequence sprinkle a bunch of additional assertions into calls
that happen to send email but this side-effect was completely uncovered.

Add a means for tests to check the the number of emails sent and whether
an email was sent to a particular recipient. For cases where the email
sent is not relevant to what is being tested, there is an escape hatch.

Repair email sending in peersaction thus activating it for datainterface.

Use a flag to make peer processing work as before by default.

Overhaul peer validation thereby restoring checking lost during rebase.

Return 400 status codes when a failure occurs during a new peer creation.

appease checks on CI by reworking some code for pylint's benefit alone

Add documentation to datainterface and module header to tmplinterface.

Unify the common portions of request coalescing into a reqinfo lib.

pull out the one-off peer_user_dict given there is a better alternative

use a better on disk fixture format for the single peer fixture

in the process, find out that expire needed to be saved as an mktime
value but ended up being saved in iso format traversing datainterface

Add support for filtering by expire.

contain the AND and OR supporting search within tmplinterface

support requested peers filtering by kind

Nudge to latest version.

patch booleanify so its resilient against in-practice data

Nudge to latest version.

cover peer import support in peersaction

restore the peers import route
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Martin-Rehr Martin-Rehr Martin-Rehr left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@albu-diku @Martin-Rehr