fix(confluence): name the response when a Confluence URL answers with something other than JSON

[tabossert]

Summary

The Confluence connection check could fail with only this:

UserError: Failed to connect to find any Confluence space: AttributeError

Cause: atlassian-python-api 4.0.7 (AtlassianRestAPI.get()) does not raise when a 2xx response is not JSON. It returns the body as a string (response.text), or None when the body is empty. Only 4xx/5xx responses raise HTTPError. ConfluenceIndexer._paginate_v2_results then called response.get("results", []) on that string or None, or on a JSON list, which raised AttributeError. precheck wraps every unexpected exception with safe_error_summary, which keeps only the exception type name, so the cause was lost.

This happens when the base URL leads to a login or SSO page, to the web UI, or to anything else that answers 200 with a body that is not JSON.

Fix: _paginate_v2_results now raises a UserError whenever client.get() returns something other than a JSON object. The v2 listing is shared by the space check, the configured-space lookup and the page listing. The error message:

• names the URL it called, with credentials and query removed via DataSanitizer.sanitize_url
• names the API path it requested
• says what came back: an HTML page, an empty body, a body that is not JSON, or a JSON value of another type
• tells the user to check the base URL (for Confluence Cloud, https://<site>.atlassian.net/wiki) and the auth type
• says that spaces and pages are listed through the Confluence Cloud REST API v2

The response body is never echoed. precheck already re-raises UnstructuredIngestError as is, so the message reaches the connection check intact.

Example:

Unexpected response from Confluence at https://confluence.example.com (api/v2/spaces): expected a JSON object but got an HTML page (often a login or SSO page, or the web UI). Check that the URL is the Confluence base URL (for Confluence Cloud, https://<site>.atlassian.net/wiki) and that the auth type matches the instance. Spaces and pages are listed through the Confluence Cloud REST API v2.

Version bump: 1.11.20 to 1.11.21 (patch), with a CHANGELOG entry.

Production evidence

Production check-executioner logs show five connection-check attempts on a self-hosted (not *.atlassian.net) Confluence URL. All five failed with Failed to connect to find any Confluence space: AttributeError. The traceback ends at the raise UserError( in precheck, and the code path is the same as in the 1.11.12 release the index plugin locks. A sixth attempt got HTTPError(status_code=404). The original exception is suppressed with from None, so the logs cannot tell an HTML body from an empty one. The new message states which one it was.

Proof

Repro. The new tests fail on origin/main. I restored only confluence.py and __version__.py from origin/main:

$ pytest test/unit/connectors/test_confluence.py -k "not_a_json_object or omits_url_credentials" -q
E       AssertionError: assert 'https://***@dummy/wiki (api/v2/spaces)' in 'Failed to connect to find any Confluence space: AttributeError'
ERROR    unstructured_ingest:confluence.py:583 Failed to connect to find any Confluence space: AttributeError
5 failed, 62 deselected

With the fix:

$ pytest test/unit/connectors/test_confluence.py -k "not_a_json_object or omits_url_credentials" -q
5 passed, 62 deselected
$ pytest test/unit/connectors/test_confluence.py -q
67 passed
$ pytest -n auto test/unit --ignore test/unit/unstructured -q
1791 passed, 69 skipped
$ make check
All checks passed!

• The 69 skips are all outside this change: connectors whose extras are not installed locally (astradb, databricks, the fsspec connectors, openai embedders, weaviate). None are in test_confluence.py.
• make check-version was not run locally, because scripts/version-sync.sh needs GNU sed 4.3 or later. I checked by hand that the CHANGELOG heading [1.11.21] matches __version__.

Tests (mocked at HTTPAdapter.send, so they run the real atlassian client parsing):

• A 200 response with an HTML page, an empty body, a non-JSON body, or a JSON list. Each produces a UserError that names the URL, the path and what came back, and never mentions AttributeError.
• A URL with credentials and a query string produces a message without either.

SND test plan

Nothing reaches users until the index plugin moves off its unstructured-ingest 1.11.12 lock. To test on an SND:

1. Build the index plugin with this branch's wheel vendored and deploy it. Confirm the pod reports unstructured_ingest 1.11.21.
2. Stand up an in-cluster endpoint that answers 200 text/html on every path, and another that answers 200 with an empty body.
3. Create a Confluence source pointing at the HTML endpoint and run Test connection. The failure should start with Unexpected response from Confluence at http://<svc> (api/v2/spaces): expected a JSON object but got an HTML page and should not contain AttributeError.
4. Repeat with the empty-body endpoint. The message should say but got an empty body.
5. Regression check: a real Confluence Cloud test site still passes Test connection.

Follow-ups (not in this PR)

• Server/Data Center support: the connector lists spaces and pages only through the v2 API. Atlassian documents v2 only for Cloud, so a Confluence Server or Data Center instance cannot pass the check with any base URL. This PR only makes that visible in the message.
• 404 message: a 404 on the v2 path still shows only HTTPError(status_code=404).
• Plugin lock: the platform-plugins index plugin needs its unstructured-ingest lock moved to 1.11.21 or later to ship this.

SND verification

Not run. There is no Confluence instance on the trevor-biogen2 SND, and this pass did not build an index plugin with this branch's wheel. The SND test plan above is still open.

🤖 Generated with Claude Code

[cubic-dev-ai]

No issues found across 4 files

<sub>Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.

Re-trigger cubic</sub>