[PRD]: Entra ID / Microsoft Graph integration & unified Principal Picker

[larsgeorge-db]

Summary

Add an optional Entra ID (Azure AD) integration via a Unity Catalog HTTP Connection, plus a unified PrincipalPicker component that replaces ~14 inconsistent principal-entry UIs across the app with a Gmail-style badge input.

• Configured mode: type-ahead + popup search dialog backed by Microsoft Graph (Users + Groups). Each result row shows displayName on top and email/UPN (users) or GUID (groups) underneath, so homonyms are always disambiguated.
• Unconfigured mode: typed values become removable badges; clicking a badge reverts it to editable text.
• Auth: UC HTTP Connection only — UC handles OAuth2 client credentials and token refresh via ws.serving_endpoints.http_request(conn=name, ...). No app-side token cache, no client secret stored in the app database.
• Storage: keep existing string / List[str] fields untouched. No Alembic migration for v1.

Full PRD: docs/prds/prd-entra-id-graph-integration.md

Why

Today, ~14 different forms each implement principal entry differently (plain Input, comma-separated string, hard-coded Select, even unwired wizard placeholders). There is no validation, no display-name resolution, and no defense against typos. Operators with an Entra tenant cannot pick from a directory; consumers are guessing whether a group is data-eng or data_engineering.

The existing UC HTTP Connection plumbing (used by the workflow webhook step) already gives us a clean, secret-free path to Microsoft Graph. The pieces are in place; what is missing is one shared component, a small Graph search backend, and migration of every call site.

Decisions locked in (from PRD Q&A)

• Replacement scope: all ~14 call sites in v1.
• Principal types: Users + Groups (defer service principals).
• Storage: keep strings (List[str] of emails / group names).
• Search UI: both type-ahead AND popup dialog in the same component.
• Pre-existing data: no re-validation; renders as plain badges.
• Manual-mode badges: click to edit (true Gmail-like).
• Disambiguation: two-line rows (name + email/GUID) plus tooltip on selected badges.

Sub-issues (to be filed)

Backend

• Extract shared list_http_connections(ws) helper from workflows_routes.py and expose GET /api/settings/uc-http-connections
• New EntraIdManager (src/backend/src/controller/entra_id_manager.py) with search_users, search_groups, get_user, get_group, test, OData escaping, in-memory TTL cache
• New routes GET /api/entra/status, GET /api/entra/search, POST /api/entra/test
• Persist new setting ENTRA_UC_HTTP_CONNECTION_NAME in SettingsManager (`_load_persisted_settings` + `update_settings` + `get_settings`)
• Pydantic Principal model + tests for normalisation

Frontend — shared component

• PrincipalPicker component at src/frontend/src/components/common/principal-picker.tsx with both modes, two-line result rows, tooltip on badges, Zustand entra-store for status
• Settings UI: new "Entra ID" tab under Integrations (entra-settings.tsx + view) with UC connection select + test button + clear button
• i18n keys in all 8 locales (entra namespace + common.principalPicker.*)

Frontend — call-site migration (15 fields across 13 files)

• Group 1 (single user): `assign-owner-dialog.tsx`, `create-review-request-dialog.tsx`, `mdm/create-review-dialog.tsx`, `data-products/team-member-form-dialog.tsx`, `data-contracts/team-member-form-dialog.tsx`
• Group 2 (multi user/group): `role-form-dialog.tsx` (groups + users), `entitlements.tsx` persona groups, `comments/comment-sidebar.tsx` audience
• Group 3 (group only): `tags-settings.tsx` permissionForm.group_id, `teams/team-form-dialog.tsx` member_identifier
• Group 4 (workflow designer): notification `recipients` + approval `approvers` via "Custom principals…" toggle
• Group 5 (wizard placeholders): `data-contract-wizard-dialog.tsx` dc-owner / stakeholders / groups / primary support email

Testing

• Backend unit tests for `EntraIdManager` against mocked `serving_endpoints.http_request`
• Backend route tests for `/api/entra/*`
• Frontend tests for `PrincipalPicker` (both modes, accepts filter, badge tooltip)
• Frontend tests for `entra-settings.tsx` test button toast behaviour

Out of scope (v1)

• Service principals
• Storage migration to structured Principal records
• Bulk re-validation of pre-existing values
• OBO / delegated Graph queries
• Server-side display-name caching beyond the 5-min query cache
• Replacing existing role/team Selects in the workflow designer (additive toggle only)

Acceptance criteria

1. Admin can configure a UC HTTP Connection in Settings → Integrations → Entra ID and the test button reports success/failure of a real Graph probe.
2. With Entra configured, every principal-entry field across the app uses the new picker; type-ahead returns results after 2 chars, debounced 250ms.
3. Every search result row shows the friendly name plus the email/UPN/GUID underneath; selected badges expose the email/UPN/GUID via tooltip + `title` attribute.
4. With Entra not configured, every field falls back to the same picker in manual mode (Enter/Tab/comma → badge, click badge → editable text).
5. Existing API payloads continue to use the same string identifiers; existing DB rows render as plain badges with no error decoration.
6. No client secret is stored in the app database; UC handles OAuth.

[larsgeorge-db]

Example:

class AzureAdService {
  private accessToken: string | null = null;
  private tokenExpiry: number = 0;


  /**
   * Check if Azure AD service is configured
   */
  isConfigured(): boolean {
    return azureAdConfig.isConfigured;
  }

  /**
   * Get an access token using client credentials flow
   */
  private async getAccessToken(): Promise<string> {
    // Return cached token if still valid (with 5 minute buffer)
    if (this.accessToken && Date.now() < this.tokenExpiry - 300000) {
      return this.accessToken;
    }

    if (!azureAdConfig.tokenUrl || !azureAdConfig.clientId || !azureAdConfig.clientSecret) {
      throw new Error('Azure AD is not configured');
    }
    logger.debug('Requesting new Azure AD access token');

    const params = new URLSearchParams({
      client_id: azureAdConfig.clientId,
      client_secret: azureAdConfig.clientSecret,
      scope: 'https://graph.microsoft.com/.default',
      grant_type: 'client_credentials',
    });

    const response = await fetch(azureAdConfig.tokenUrl, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/x-www-form-urlencoded',
      },
      body: params.toString(),
    });

    if (!response.ok) {
      const errorText = await response.text();
      logger.error('Failed to get Azure AD access token', {
        status: response.status,
        error: errorText,
      });
      throw new Error(`Failed to get Azure AD access token: ${response.status}`);
    }

    const tokenResponse = (await response.json()) as TokenResponse;
    this.accessToken = tokenResponse.access_token;
    this.tokenExpiry = Date.now() + tokenResponse.expires_in * 1000;

    logger.debug('Azure AD access token obtained successfully');
    return this.accessToken;
  }
}
 

Usage example:

    /**
     * Get user's onPremisesSamAccountName from Microsoft Graph API
     * @param email The user's email address (UPN)
     * @returns The onPremisesSamAccountName or undefined if not found
     */
    async getOnPremisesSamAccountName(email: string): Promise<UsernameResolutionResult> {
      if (!this.isConfigured()) {
        logger.warn('Azure AD service not configured, skipping username resolution');
        return {
          email,
          error: 'Azure AD not configured',
        };
      }
      try {
        const accessToken = await this.getAccessToken();
        // Encode the email for URL use
        const encodedEmail = encodeURIComponent(email);
        const graphUrl = `${azureAdConfig.graphApiUrl}/users/${encodedEmail}?$select=id,displayName,userPrincipalName,onPremisesSamAccountName`;
        logger.debug('Fetching user from Microsoft Graph', { email, url: graphUrl });
        const response = await fetch(graphUrl, {
          method: 'GET',
          headers: {
            Authorization: `Bearer ${accessToken}`,
            'Content-Type': 'application/json',
          },
        });

        if (!response.ok) {
          if (response.status === 404) {
            logger.warn('User not found in Azure AD', { email });
            return {
              email,
              error: 'User not found in Azure AD',
            };
          }

          const errorText = await response.text();
          logger.error('Microsoft Graph API error', {
            status: response.status,
            statusText: response.statusText,
            error: errorText,
          });
          throw new Error(`Microsoft Graph API error: ${response.status} ${response.statusText}`);
        }
        const user = (await response.json()) as AzureAdUser;
        if (user.onPremisesSamAccountName) {
          logger.debug('onPremisesSamAccountName resolved', {
            email,
            username: user.onPremisesSamAccountName,
          });
          return {
            email,
            username: user.onPremisesSamAccountName,
          };
        }
        logger.warn('User found but onPremisesSamAccountName is not set', { email });
        return {
          email,
          error: 'onPremisesSamAccountName not set for user',
        };
      } catch (error) {
        logger.error('Error fetching user from Azure AD', { email, error });
        return {
          email,
          error: error instanceof Error ? error.message : 'Unknown error',
        };
      }
    }