feat: Microsoft connector; allow pagination when more than 1000 AD groups

[XDanny322]

Overview

Using the Microsoft connector (EntraID / Azure AD), if a user has more than 1000
transitive group memberships, Dex fails to authenticate them. The Microsoft Graph API
endpoint POST /v1.0/directoryObjects/getByIds rejects requests with more than 1000
identifiers:

Request_BadRequest: Number of included identifiers cannot exceed '1000'.

What this PR does / why we need it

getGroupNames resolves Azure AD group UUIDs to display names by POSTing all IDs at once
to /v1.0/directoryObjects/getByIds. When a user has more than 1000 transitive group
memberships (common in large enterprises with nested security groups), Graph rejects the
request and authentication fails entirely.

This PR wraps the existing inner loop in an outer chunking loop that splits the input IDs
into batches of ≤999 before each getByIds call:

const maxBatchSize = 999

for i := 0; i < len(ids); i += maxBatchSize {
    batch := ids[i:min(i+maxBatchSize, len(ids))]
    // existing inner loop handles response pagination (nextLink) for each batch
}

Why the existing loop doesn't already handle this: The inner for loop in
getGroupNames follows nextLink to page through response results (~100 per page).
It does not batch the input IDs — the full slice is sent in the first request body.
This PR adds a separate outer loop for input batching; the inner response-pagination
loop is unchanged.

A new unit test TestGetGroupNamesBatchLimit is added in connector/microsoft/microsoft_test.go:

• Builds a slice of 1500 fake group IDs
• Mock server returns HTTP 400 if any batch exceeds 999 IDs (mirroring real Graph behavior)
• Asserts exactly 2 batch calls are made (ceil(1500 / 999) = 2)
• Asserts all 1500 names are returned

This was also validated end-to-end against a real Azure AD tenant with a user in 1068
transitive group memberships. Before the fix: authentication fails with the
Request_BadRequest error. After the fix: all 1068 groups are resolved and
authentication succeeds.

Special notes for your reviewer

1. Tested end-to-end: built a custom image, deployed it, and confirmed a user with 1068
   transitive AD groups can authenticate successfully. All groups are returned.

2. Once this fix is applied and all group display names are resolved, a secondary issue
   can surface for users with very large group counts: Dex embeds all group names in its
   JWT access token (~61 KB for 1068 groups). If an nginx ingress sits in front of Dex,
   the default large_client_header_buffers of 4 8k will reject the oversized
   Authorization: Bearer header when the token is used to call /userinfo. The
   workaround is to raise the buffer to 4 64k. This is a separate issue from the one
   fixed here — noting it so anyone hitting the same setup isn't surprised. See here

[matheuscscp]

Looks good, the only thing that looks odd is using 999 when the documented limit is 1000. Is this a "safety buffer" or something like that? If so, I'd write one or two lines about it in the comment

[brianvandenbersselaar]

This would fix the issue we're experiencing in our enterprise setup, nice MR.

[XDanny322]

Looks good, the only thing that looks odd is using 999 when the documented limit is 1000. Is this a "safety buffer" or something like that? If so, I'd write one or two lines about it in the comment

Ya, it was a safety buffer, but seems unnecessary. Likely causes more confusion then good. I've updated it back to 1000, code and the test , tested e2e, and all good.

Thanks for taking a look - feel free to trigger actions / merge etc.

[matheuscscp]

I'm considering if we should make this behavior opt-in in the config of the connector...

• Today, a user with 1k+ groups toss a single request to Microsoft and get a login failure
• After this feature, a single user with with 1k+ groups will cause Dex to make multiple requests and fetch a ton of groups

Not sure how big this impact can be on the compute resources needed by Dex or if Microsoft can start rate-limiting...

Would every user of the Microsoft connector be happy with this change? 🤔

[XDanny322]

Well, i guess i can see the concern.

Today users with 1k+ group just doesn't work. Given the lack of other issues / users reporting this =), i am guessing its not a popular usecase (but in enterprise, its pretty common). Thats what makes me sayd, i think its safe to push this as is. But i do agree that having a feature flag to gate this would help / future prove this, especially re compute concern.

Claude and i took a quick look, and it doesn't seem that bad.

I install dex via helm, and quick looks shows in the config key is a passtho from the gobinary to helm, so the control would look something like this, what do you think?

    - type: microsoft
      id: microsoft
      name: Microsoft
      config:
        clientID: $AZURE_AD_DEX_OAUTH_CLIENTID
        clientSecret: $AZURE_AD_DEX_OAUTH_CLIENTSECRET
        redirectURI: https://dex...../callback
        tenant: foo.onmicrosoft.com
        onlySecurityGroups: true
        groupNameFormat: name
        batchGroupLookups: true   # new: opt in to fetch group names for >1000 group members

Finally, should i send a PR to website as well? Happy to, just checking

[matheuscscp]

Yeah config proposal looks good to me. Yes, website PR pls!

[brianvandenbersselaar]

LGTM 👍

[XDanny322]

Hi @matheuscscp -

all done. Updates code to add feature flag, default to false. If true, then the new paging logic kicks on. Updated tests to test both if batchGroupLookups is true or false. All passing. Tested this all e2e (created a test image) in my setup and worked accordingly, per batchGroupLookups setting.

Doc PR here: dexidp/website#216

Thanks for the review!