Skip to content

test: make the smoke run's narrowing check able to fail - #35

Merged
kanushka merged 2 commits into
test/smoke-env-and-identity-serverfrom
test/smoke-narrowed-acquisition
Aug 6, 2026
Merged

test: make the smoke run's narrowing check able to fail#35
kanushka merged 2 commits into
test/smoke-env-and-identity-serverfrom
test/smoke-narrowed-acquisition

Conversation

@kanushka

@kanushka kanushka commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #34, stacked on it. Closes the last open definition-of-done box in docs/plans/login-first-slice.md.

The problem

The live smoke run's narrowing check could not fail.

login_smoke_test.go acquired once, asking for config.Scopes — every permission the session already carries. internal/auth/narrowing.go verifies by comparing the issued scopes against the requested ones, so when the request is the whole session, that comparison is a set against itself. It holds however the deployment behaved. Point the run at a backend that disregards a narrowing request entirely and it would still print granted.

The audience half of that verification was real and did catch things — an Asgardeo run refused on it. The scope half was theatre. Which matters, because the definition of done reads "against a backend proven to satisfy the scope/audience policy", and half of that proof was vacuous.

config.NarrowTarget() had existed for exactly this since the smoke package was written, and nothing called it.

What this does

Acquires twice, because the two acquisitions fail for different reasons:

LOGIN SMOKE: granted  — asked for everything the session carries, received access of
                        1261 characters bound to "reference-status" carrying
                        [reference:status:read reference:status:write]
LOGIN SMOKE: narrowed — asked for one permission out of the 2 the session holds,
                        received access of 1230 characters bound to "reference-status"
                        carrying [reference:status:read]

The first proves the broker can derive access at all, which is what a first run against a new deployment needs to know. The second is a strict subset — the request a module actually makes — and is the only form the verification can disagree with.

A refusal on the broad request stops the run rather than repeating one finding: a deployment that will not issue against this audience at all says nothing about narrowing. A refusal on the narrowed request is a real statement about the deployment, and still passes, because the shell declining to hand a module more authority than it asked for is the designed outcome.

One thing the change surfaced

The first attempt reused a single auth.Broker and failed:

auth.already_granted: the "reference" module asked for access twice in one command

The shell allows a module one acquisition per command. So the two acquisitions run as two invocations — which is not a way around the rule but the accurate model of it: two commands, in turn, against one session, the way the shell is actually used. Against a rotating deployment the second acquisition now also proves the first persisted its replacement refresh token, since it can only reach the token endpoint with what the first one stored.

Not changed, and why

The same definition-of-done clause says the reference module receives the token. That half needs nothing further. TestLoginThenTheModuleReceivesIssuerMintedNarrowedAccess already launches the real module subprocess, has the issuer introspect what it presented, and pins refresh rotation across three runs. A module never learns which issuer minted its token — that is what the source seam is for — so running it against a live deployment would add a subprocess and a status service to a test whose value is about the deployment, and prove nothing the deterministic chain does not.

Verification

make smoke-login    granted 1261 chars [read write], then narrowed 1230 chars [read]
make test           exit 0
./scripts/acceptance.sh   exit 0
make lint           0 issues, tagged and untagged
make vet            clean
gofmt -l            no output

Live run against wso2/wso2is:7.3.0, the deployment #34 registered. The two tokens differ in length and in scope set, which is the evidence the previous single-acquisition form could not produce.

docs/plans/login-first-slice.md is now fully ticked, definition of done included.

Relates to #17.

https://claude.ai/code/session_01RojiAgW9hi3b9f9G6ZXBVp

@kanushka
kanushka requested a review from hevayo as a code owner August 6, 2026 04:36
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7bcf299b-86cc-495a-ae8c-56843e8d317e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the live login smoke run so its scope-narrowing verification can actually disagree with a misbehaving deployment, by performing two brokered acquisitions: one “broad” (prove access derivation works at all) and one “narrowed” (strict subset, so narrowing can meaningfully fail). It also updates the smoke-run documentation and the implementation plan to reflect the new evidence and definition-of-done completion.

Changes:

  • Update TestLoginSmoke to acquire twice (broad + narrowed) using separate auth.Broker instances, and add an acquire helper to standardize smoke output.
  • Expand test/smoke/RUNNING.md to explain why the second acquisition is the meaningful narrowing measurement and how to interpret refusals.
  • Mark the definition-of-done item as completed in docs/plans/login-first-slice.md, with a short summary of the new empirical evidence.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
test/smoke/RUNNING.md Documents the new two-acquisition smoke run and clarifies how to interpret narrowing-related outcomes.
test/smoke/login_smoke_test.go Implements the two-step acquisition (broad + narrowed) so the narrowing check can fail when a backend ignores narrowing.
docs/plans/login-first-slice.md Updates plan status/definition-of-done text to reflect the now-non-vacuous live narrowing proof.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/smoke/RUNNING.md
The live run asked the broker for config.Scopes — every permission the session
already carries — so internal/auth/narrowing.go compared the issued scopes
against an identical request. That comparison holds however the deployment
behaved. A backend that disregarded the narrowing request entirely would still
have been reported as granted, and the definition of done asks for a backend
*proven* to satisfy the scope policy.

The audience half of the same verification was real and had caught a live
misconfiguration. The scope half could not fail. config.NarrowTarget() had
existed for this since the package was written and nothing called it.

So the run now acquires twice. The first request is the broad one and proves the
broker can derive access at all, which is what a first run against a new
deployment needs to know. The second asks for one permission out of the several
the session holds — the request a module actually makes, and the only form the
verification can disagree with. A refusal on the broad request stops the run
rather than repeating one finding, because a deployment that will not issue
against this audience at all says nothing about narrowing.

Reusing one broker for both refused with auth.already_granted: a module gets one
acquisition per command. Two invocations is not a way around that rule but the
accurate model of it — two commands, in turn, against one session. Against a
rotating deployment the second acquisition now also proves the first persisted
its replacement, since it can only reach the token endpoint with what the first
one stored.

Measured against wso2/wso2is:7.3.0: 1261 characters carrying both permissions,
then 1230 carrying one. Two tokens the previous single-acquisition form could
not have told apart.

The module half of the same definition-of-done clause needs nothing further. A
module never learns which issuer minted its token, so running it against a live
deployment proves nothing TestLoginThenTheModuleReceivesIssuerMintedNarrowedAccess
does not already prove against the fake issuer, where the token is introspected
and rotation is pinned across three runs.

Claude-Session: https://claude.ai/code/session_01RojiAgW9hi3b9f9G6ZXBVp
@kanushka
kanushka force-pushed the test/smoke-narrowed-acquisition branch from 0959a52 to e8aa8e4 Compare August 6, 2026 04:57
Review found the smoke run's refusal example truncated. The run prints the
shell's own message on an indented second line, and the example stopped before
it — while the paragraph beside it said that message is what distinguishes one
cause from another. A reader following that advice found nothing to follow it
with.

The example now carries the indented line, and the four other messages are
tabulated beside it. The summary sentence above them is deliberately identical
for all five, because auth.narrowing_unavailable covers five distinct causes and
a summary naming one would be wrong four times out of five. Only the indented
line is specific, which is precisely why it had to be in the example.

Two corrections in section 8 of the walkthrough, which the new cross-reference
sends readers to and which contradicted what this branch's parent measured.

The audience row still said the value to configure is the API resource
identifier. That is right on Identity Server and wrong on Asgardeo, where the
only value ever placed in `aud` is the client ID — so a reader hitting an
audience refusal on Asgardeo was being told to set the one value that cannot
work. It now names both products and points at the sections that measured them.

The paragraph under the table still said whether Asgardeo narrows would be
recorded "once measured against a live tenant". It was measured on 2026-08-06,
on both products, and both narrow.

Claude-Session: https://claude.ai/code/session_01RojiAgW9hi3b9f9G6ZXBVp
@kanushka
kanushka merged commit 6ea7fd5 into feature/login Aug 6, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants