Add intersectional-test skill

skills/intersectional-test/SKILL.md

@@ -0,0 +1,99 @@
+---
+name: intersectional-test
+description: Run a guided MSF DHIS2 intersectional testing for some MSF Operational Directorate (OD)
+version: 0.0.1
+---
+
+# Description
+
+This skill guides a user (= focal point) of a particular OD (Operational Directorate) of MSF in the intersectional testing required whenever a new DHIS2 version needs to be validated. The skill expects a workbook file (ie an Excel file) as source of truth to verify the testing points, using either API calls or a browser when UI interaction is required. Finally, write as an output the updated copy of the workbook and some custom user-friendly report. 
+
+# Requirements
+
+Assumes a coding agent with shell and HTTP access to the testing instances. Uses curl, jq, openpyxl, playwright.
+
+# Folder structure of the execution
+
+## `config.yaml`
+
+Configuration file describing the full testing environment, copied into the execution folder from `config.template.yaml` (which ships with this skill, beside `SKILL.md`) and filled in with the user.
+
+## Source workbook
+
+The workbook to use as source of truth (ie `DHIS2_VERSION - DHIS2 Intersectional testing.xlsx`). It contains the testing conditions, the ODs/focal points, and the tests to perform for each subdomain `Main domain - Inner domain` (`General - Menus`, `Users`, `Maintenance - Constants`). Known sheets to ignore: `Aux`, `Main New Features`, `All Bugs And Features`.
+
+Domain sheets carry roughly these columns: `Scenario`, `Action`, `Steps to Follow`, `Expected Result`, `Comments/conditions`, `Passed?`, `Test done by`, `Fail details`. Treat that as indicative, not a schema — the sheets were written by hand over years and none of it is guaranteed. Notes:
+
+- Resolve columns by header name, never by position: the order varies between sheets (some swap `Passed?`/`Test done by`) and so does the spelling (`Test Done by`, `Section` instead of `Scenario`). Read only those columns; some sheets report thousands of spurious empty ones.
+- Sanity-check what you read rather than trusting the layout: if headers and values disagree (a status column full of OD names), believe the values; skip rows that merely repeat the header; expect sheet names to differ from their Summary label by wording or stray whitespace. Match on meaning, not on an exact structure.
+- A fully blank row is a separator, not a step. Count real steps before reporting progress, and if a row range the user gave does not match the number of steps in it, say so (rows 2-6 of Capture (Tracker programs) is 4 steps, not 5).
+- A blank `Scenario`/`Action` means "same as the row above": to build a step, walk up to the nearest non-empty cell. Steps within such a block are ordered and often depend on the previous ones (ie. "repeat the above for...").
+- `Passed?` is written by humans, so expect variants (`Passed`, `Failed`, `Fail`, `NA`, `N/A`, blank). Read case-insensitively; write only `Passed`/`Failed`/`Not Executed`/`N/A` (with the appropriate background color).
+- `N/A` and `Not Executed` are not the same: `N/A` means the step can never apply to this version/OD (feature removed, or the OD does not use that module) and is final, so later runs skip it; `Not Executed` means it applies but has not run yet, so it stays pending.
+
+# Interactions
+
+- Perform only browser UI interactions/assertions: `dhis2-browser-interaction.md` has the technical details on how to interact with the browser. Do not open/close the browser for each action step. Keep it open and reuse the tab. Do not perform manual requests to the API (unless specifically stated by the action step), always use the UI to perform assertions.
+
+## `output/artifacts/`
+
+This output folder will contain all the technical data and scripts generated by the agent in the session. This data will be used to: 
+
+1. generate the final report.
+2. serve as a technical report in itself for reproducibility.
+3. store the necessary data for resumability, in case we continue the session at a later time.
+
+- `output/artifacts/Subdomain/`
+- `output/artifacts/Subdomain/NNN`: Folder for a specific subdomain step `NNN` (ie. `002` means row 2 in the workbook, for easy matching). Store here any script or data that was used/generated while testing the step: `test.py`, whatever API payloads it asserted against, `trace.zip`, final screenshots `success.png` if ok, `failure.png` if it failed.
+- `output/artifacts/Subdomain/NNN/result.json`: the step's verdict, written as soon as the step ends - status, timestamp, the note that feeds `Comments/conditions`, and the evidence paths. This is what makes a run resumable and what Step 3 assembles the reports from, so a step without one has not been run.
+- `output/artifacts/Subdomain/NNN/transcript.txt`: the slice of the agent's own session transcript (prompts, reasoning, tool calls, output) spent on this step, cut in Step 3 from `started_at`/`timestamp` in `result.json`. It is the "why" behind the verdict - the script shows what was asserted, the transcript shows how the agent got there and what it tried first.
+- `output/artifacts/Subdomain/results.json`: the roll-up for the subdomain, plus the run's context (OD, focal point, server, DHIS2 version, start/finish).
+
+# Execution workflow
+
+The focal point user will invoke the skill (`/intersectional-test`) and the process will begin:
+
+## Step 1 - Environment setup
+
+- Read the workbook to get the ODs list, their focal point and the DHIS2 target version (`Testing conditions` -> `DHIS2 Core Version`). Show a very concise report to the user of that analysis. If some important information is missing or unclear, ask the user before continuing the process.
+- With that information in hand, guide the user to fill `config.yaml` (if not existing, copy from the template). That includes all the keys (except the field server, which is optional)
+- Check that the testing DHIS2 central server is up and its version matches the target DHIS2.
+- Read this OD's assignment from the Summary sheet and show a report of what their domain are, which are completed and which are still pending, so have an idea of the status of what's pending to do.
+- The Summary's progress colours are hand-maintained and can be wrong, so treat them as a hint and derive real status by counting `Passed?` in the domain sheet itself. Row 36 was coloured "Test completed" while every `Passed?` cell on that sheet was blank. Report any disagreement to the user rather than silently picking one.
+
+## Step 2 - Execution of a subdomain and its steps
+
+- The user has at this point a clear view of the task status, so ask them how to proceed. Typically, they will instruct the agent to run tests for pending subdomains/steps or just the some tests for some specific subdomain. By default, actions in the domain tab already marked as Passed/NA will be skipped (inform the user of that), unless that step is necessary for the next to succeed, or the user instructs otherwise.
+- Before starting the execution of a subdomain, the agent will check if the version of the app (or apps) to be used in that subdomain match the requirements in the Testing Conditions. If not, guide the user on how to install them.
+- Now, for each step intended to be executed, follow the steps specified in the corresponding sheet (the summary label and the sheets may not match 100%, use your judgement) and its comments/description/expected-results and so on, and interact with the DHIS2 instance (either using CLI commands or a real browser). 
+- If a user with some specific permissions (other than admin) should be used, create it on-the-fly (inform the user about that).
+- An Expected Result that the app has outgrown is not a failing step. The workbook was written over years and describes UI that has since changed (row 4 expects a "List button" Capture 105.7 no longer has). Assert the closest real behaviour, pass the step if that behaviour is correct, and record what changed in `Comments/conditions` - a `Failed` here would report a defect that does not exist. Flag these to the user; they are the rows the next round should rewrite.
+- Prefer a contrast to a presence check. "Program is selectable in the dropdown" is nearly unfalsifiable on its own; comparing an org unit that has the program (14 offered) against one that does not (1 offered), both matching the API, is a real test.
+- Derive expected values from the API at runtime, never hardcode them. Names, uids and counts differ per OD, and a hardcoded expectation is a step that only ever worked on one instance.
+- The browser runs headless, so a run looks like nothing is happening. The first time a step needs the browser, tell the user how to watch it live and how to replay a step afterwards; both commands are printed by the run itself (see `dhis2-browser-interaction.md`).
+- The process should be mostly automatic, so show to user clearly which action you are executing at each moment (name of the subdomain, name of the action, the overall progress, and so on)
+- The moment you begin a step (before reading its row in detail or writing any script), record the wall-clock time (`date -Is`) and store it as `started_at` in that step's `result.json`; `timestamp` is when the verdict was written. Step 3 uses that window to cut the step's transcript slice, and `elapsed_seconds` only covers the script run - the agent's own time on the step (reading, writing, debugging) is usually far longer and is what the slice has to cover.
+- If something fundamental is going wrong (ie. you cannot open the browser or the apps), stop and fix that with the help of the user. When things go normal, a step can pass, fail, or flagged as `Not Executed` if something was missing. Write the artifacts (data/scripts) needed for debugging/reporting/resuming and continue to the next step. 
+- After the requested execution has finished, show in the screen a summary of what you have done and what output you have generated (see next step)
+
+## Step 3 — Report
+
+The previous step will have created some artifacts data. From this data, create these reports files (assuming `DHIS2_VERSION - DHIS2 Intersectional testing.xlsx` was the input file):
+
+- `output/skill/`: Clone of the skill that guided the session.
+
+- `output/DHIS2_VERSION - DHIS2 Intersectional testing (original).xlsx`
+
+- `output/DHIS2_VERSION - DHIS2 Intersectional testing (updated).xlsx`: a cloned version of the input workbook with fields filled from every accumulated `result.json`, not only this session's. Typical columns to fill: `Passed?`, `Test done by`, `Fail details`.
+    - openpyxl keeps formulas but drops their cached results, so cells like the Summary totals read back as `None` and show empty until Excel or LibreOffice recalculates them on open. Expected, not corruption - do not try to "fix" it by writing literals over the formulas.
+    - Verify the output by diffing every cell against the input and confirming the only changes are the cells this run meant to write.
+
+- `output/report.html` A report that converts the current session. Contents:
+    - A brief summary of what was tested (OD, focal point, subdomains and the success/failed steps), with links to the input/output xlsx files. 
+    - A detailed table with filters showing the result (columns: `Row`, `Subdomain`, `Scenario`, `Action`, `Passed?`, `Elapsed (secs)`). To keep it readable, strive for single-line rows. For that, 1) cells containing long text should be ellipsed (...), and a hover would show the rest of the text, and 2) The row has an icon (position: at the left of the row) to open more details, showing extra info (non-table form) like: `Steps to follow`, `Expected result`, `Passed?`, `What was asserted`, `Resources` (links to the most relevant artifact files of that step), `Playback`(the command a user should run to retrace the step, ie `.venv/bin/playwright show-trace "output/artifacts/Capture (Tracker programs)/003/trace.zip"` - adjust for the current OS)
+    - Coding agent (with version) used, model, input/output tokens, and so on. Read the agent/token figures from the coding agent's own session transcript rather than estimating them.
+    - Agent time of the testing session. If the transcript carries no duration field, derive both from entry timestamps.
+    - This skill's `version` from its frontmatter. The ODs run independently over weeks, so a returned result has to say which revision produced it - and `output/skill/` is the copy it refers to.
+    - Full agent conversation, internal reasoning and output, with a button that downloads it as a `.txt`. Build that file client-side from the text already in the page (a Blob and `<a download>`), not by linking a sidecar file - the report gets shared on its own, and a link to a neighbouring file breaks the moment it does.
+    - Per-step transcript: cut one slice per step into `output/artifacts/Subdomain/NNN/transcript.txt` - the transcript entries between the step's `started_at` and `timestamp` (widen to the next step's `started_at` when the verdict was written late), with the same header/format as the full file. Link it from the step's `Resources` as `Agent transcript`, next to `trace.zip` and the screenshots: it is an artifact like them (so the sidecar rule above does not apply), and a reviewer questioning one verdict should not have to scroll a 100-entry session to find its 8 entries.
+    How to display: Keep the summary always visible and put the other sections in accordions, so the contents of the report are visible at a glance. Add a button for dark/light theme (dark by default).

skills/intersectional-test/config.template.yaml

@@ -0,0 +1,6 @@
+od: "Operational Directorate" # OCG, OCB, OCA, OCBA, OCP, WACA
+focal_point: "Name Surname" # Focal point responsible for the tests in the OD
+workbook: "SOMEFILE.xlsx"   # ie "VERSION - DHIS2 Intersectional testing.xlsx"
+servers: # DHIS2 servers (central: required, field: required only when running the Sync domain)
+    central: { url: "http://localhost:8080", username: admin, password: district }
+    field:   { url: "http://localhost:8081", username: admin, password: district }