test(enterprise): cover the model availability round trip (#1669) - #1670
Merged
Conversation
The models screen's WRITE half, excluded from #1660 on the strength of #1659 — which reported `Edit models` as an enabled control that opened nothing. That report was three measurement errors and is closed invalid. `Edit models` is a TOGGLE: `aria-pressed` goes false -> true, and each of the 44 rows swaps its state text for a `role="switch"` named after its model, alongside `Enable all models`, `Hide all models`, `Cancel` and `Save models`. The three errors, recorded in the doc because the same three would catch anyone probing this screen: looking for a dialog; reading a shrinking panel (1645 -> 1295 chars) as a collapse when the item count is 44 in both states; and counting the wrong role twice — `switch` BEFORE entering edit mode, then `button` where the control is a `switch`. Four tests: - Entering edit mode offers one switch per model and writes NOTHING yet. A screen that wrote on entering an edit mode would be surprising in the worst way and nothing else here would notice. - Enabling one and saving lands exactly that model's key — `PUT /api/v1/model-availability-policy` -> 200, `enabled_model_keys: ["openai::llm::<model>"]` — keyed on the model the switch NAMED rather than on a position. - Enabling a model does NOT make it visible while the provider has no credentials: the row stays `Pending` / `Hidden`. - `Cancel` discards, asserted on the request rather than on a state read taken immediately after. The third test replaced the one I was about to write. The natural assertion — "the row stops reading Hidden" — is FALSE, and asserting it would have manufactured a defect. Builder visibility follows the provider's CREDENTIALS, not the availability policy: one enabled model on a provider that is still `Pending` is not something a builder can pick. The two are different axes and the screen does not conflate them, which is the same lie #1660's read tests catch, arriving from the other direction. Four mutations: never entering edit mode, asserting a model the test did not pick, asserting the row reads `Visible`, and replacing `Cancel` with `Save models` all go red. 29 passed across `admin-console/` twice; the instance is left with `enabled_model_keys: []` and no governance records. One advisory noted and not filed: `503 /api/v1/session` "Authorization service unavailable" appeared once across the two runs. Not reproducible — 76 polls of that endpoint during four policy writes all answered 200 — and the same shape has been seen once before on this image without reproducing. The fixture logs it as advisory and does not fail, which is the right handling for a single unreproducible transient. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1669. The write half of the models screen, which #1660 excluded on a premise that turned
out to be my own measurement error.
1.
Edit modelsworks — #1659 was three errors, not a defectIt is a toggle:
aria-pressedgoesfalse→true, and each of the 44 rows swaps its statetext for a control.
with
Enable all models,Hide all models,CancelandSave modelsappearing alongside.The three errors, recorded in the doc because the same three would catch anyone probing this
screen:
role=dialog,role=menu, a URL change, all correctly zero, becausenone of them is how the control works.
item count is 44 in both states. It is 44 rows each losing the word
Disabledto a switch.role=switchbefore entering edit mode (0, correctly, andthe probe's own guard then meant it never clicked at all), then
role=buttonwith the toggle'sname where the control is a
role=switch.That is the third time in this area a partial read of a screen produced a confident wrong
conclusion — after the delete button's
aria-labeland the provider card that "vanished" whilesitting in the next region of the same screen. #1659 is closed invalid with the same write-up.
2. Covered tests
entering edit mode offers one control per model and writes nothing yetenabled_model_keysuntouched. A screen that wrote on entering an edit mode would be surprising in the worst way.enabling a model and saving lands exactly that model in the policySave modelsissuesPUT /api/v1/model-availability-policy→200; the policy holds the key of the model the switch named, not a position.enabling a model does not make it visible while the provider has no credentialsPending/Hidden.cancelling an edit discards it3. Test 3 replaced the test I was about to write
The natural assertion — "the row stops reading
Hidden" — is false. Measured: after saving oneenabled model the row still reads
Pending/Hidden, and that is correct. Builder visibilityfollows the provider's credentials, not the availability policy: one enabled model on a provider
that is still
Pendingis not something a builder can pick.Asserting the natural version would have manufactured a defect. The two are different axes and the
screen does not conflate them — the same lie #1660's read tests catch, arriving from the other
direction.
4. Force-fail
Visibleafter enablingCancelreplaced bySave models5. Validation
langflow-enterprise:latest(2026-08-27), RBAC variant.admin-console/: 29 passed, twice, ~1.3 min.enabled_model_keys: [], no governance records.One advisory, noted and deliberately not filed.
503 /api/v1/session— "Authorization serviceunavailable" — appeared once across the two directory runs. Not reproducible: 76 polls of that
endpoint during four consecutive policy writes all answered
200, and the same shape has beenobserved once before on this image without reproducing. The fixture logs it as advisory and does not
fail, which is the right handling for a single unreproducible transient — filing it would send
someone after a phantom, and hiding it would be worse.
6. Notes for the reviewer
DELETE /api/v1/model-provider-governance/{id}withexpected_revision, which removes the approval and the availability it created. Each test revokesbefore it approves rather than assuming a clean instance.
@stable(project-management: the destructive 'deleting every folder' test wipes sibling tests' folders and flows under fullyParallel #1010). Tags unchanged.QA-CHECKLIST.md§ 22.7: the[!]bullet pointing at enterprise models screen: Edit Models is enabled, opens nothing, and collapses the row instead #1659 becomes four[-].🤖 Generated with Claude Code