feat: data connector source metadata for all providers (spec v0.7.9)

[kdr]

Summary

Syncs the SDK to spec v0.7.9 (cloudglue/cloudglue-api-spec#103) and bumps the package to 0.7.19.

Generated

• New ZoomSourceMetadata, RecallSourceMetadata, GoogleDriveSourceMetadata, DropboxSourceMetadata, and GongSourceMetadata schemas; SourceMetadata is now a discriminated union on source_type across all six connectors, with required fields loosened to source_type + the provider's primary id (everything else nullable)
• Updated data-connector endpoint descriptions: per-connector from/to + title_search support, filtered-pagination contract, 501 now only for s3/gcs, 502 on unvalidatable upstream responses

Wrapper (src/)

• types.ts exports the SourceMetadata family (all six variants) and SourceMetadataResponse, so consumers can narrow file.source_metadata / getSourceMetadata() results on source_type
• EnhancedDataConnectorsApi docs refreshed: accurate per-connector filter support matrix on ListDataConnectorFilesParams, per-file provider metadata on listFiles, filtered-pagination guidance (keep paging until next_page_token is null), source-metadata support for all six connectors

Test plan

• npm run build clean, npm test 106/106
• Live API battery (read-only, all six connectors connected): listFiles returns per-file provider metadata with matching source_type; getSourceMetadata 200s and narrows to the right variant for grain/zoom/recall/google-drive/dropbox/gong; Dropbox folder drill via path; title_search verified on google-drive (native, folders exempt) and gong (matched while listing) — 20/20 checks passed

---

Note

Medium Risk
Grain SourceMetadata field optionality and broader union typing can break consumers that assumed required fields; the generator pairing change affects all regenerated types but reduces prior nullability bugs.

Overview
Syncs the client to OpenAPI spec v0.7.9 and releases 0.7.19, centered on provider source metadata for data connectors and safer post-generation TypeScript fixes.

Source metadata & connectors: SourceMetadata is now a source_type discriminated union across Grain, Zoom, Recall, Google Drive, Dropbox, and Gong, with new Zod/TS shapes for each provider. Grain metadata is relaxed so only source_type and grain_recording_id stay required; other fields are optional/nullable. Generated data-connector docs describe per-file metadata on listFiles, sync/source_metadata for all six providers, expanded getSourceMetadata support (501 only for S3/GCS), and filtered-pagination behavior.

generate.js: Nullish/nullable TS widening is scoped to matching const Zod schema / type alias pairs (via the TypeScript AST) instead of file-wide field names, fixing incorrect | null on unrelated types (e.g. File.created_at vs RecallSourceMetadata.created_at).

Public API surface: types.ts re-exports the full SourceMetadata family and SourceMetadataResponse. EnhancedDataConnectorsApi and ListDataConnectorFilesParams document per-connector from/to/title_search support and pagination. Deep search collection filter is SearchFilter only (no null); Response instructions input typing drops explicit null where the schema is optional-only.

^{Reviewed by Cursor Bugbot for commit d663cee. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by CodeRabbit

• New Features
  • Added public types for provider-specific source metadata returned with synced files.
  • Improved generated schema typings so nullable and optional fields are represented accurately.
• Bug Fixes
  • Corrected deep-search collection filters to reject unsupported null values.
• Documentation
  • Clarified pagination, filtering, file URI, metadata, synchronization, and connector error behavior.
• Chores
  • Released version 0.7.19 with updated specification references.

[coderabbitai]

📝 Walkthrough

Walkthrough

generate.js now scopes nullability widening to matching generated schema/type pairs. Public data-connector metadata exports and documentation are expanded, the collections filter type is tightened, and package/spec revisions are updated.

Changes

Schema and API contract updates

Layer / File(s)	Summary
Scoped generated-type nullability generate.js	Compiler-based helpers detect nullable fields per top-level schema and rewrite only the matching type aliases, including nullable object handling.
Public connector and search contracts src/types.ts, src/api/data-connectors.api.ts, src/api/deep-search.api.ts	Source metadata aliases are exported, connector API documentation is expanded, and the collections filter no longer accepts null.
Release and source synchronization package.json, spec	The package version changes to 0.7.19, and the referenced specification commit is updated.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• cloudglue/cloudglue-js#98: Updates related generated nullability handling and deep-search/type contracts.
• cloudglue/cloudglue-js#124: Refines .nullish() and .nullable() field detection in generate.js.
• cloudglue/cloudglue-js#128: Relates generated nullable source_metadata types to generate.js handling.

Suggested reviewers: amyxst, chillenberger

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: source metadata support for all data connectors.
Description check	✅ Passed	The description matches the template with a Summary section and a Test plan checklist, and it provides the key implementation details.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch kdr-moresyncs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 653500e. Configure here.}

[kdr]

Bugbot's finding was real but the root cause was in generate.js, not the schemas: the nullish-type transform collected nullable field names per file and widened every same-named field, so the new nullable RecallSourceMetadata.created_at / DropboxSourceMetadata.media_info (spec v0.7.9) leaked | null onto File.created_at, File.media_info, and Describe.created_at — whose Zod schemas (correctly, per the spec) never accepted null. So the runtime validator was right and the TS types were wrong, the reverse of the flagged failure mode; no API response would actually have been rejected.

Fixed in 77db625: detection/widening are now scoped to matching const <Name>/type <Name> declaration pairs via the TypeScript AST. The regen also removed older leaks from the same bug (File.thumbnail_url, Describe.duration_seconds/thumbnail_url, CreateResponseRequest.instructions, DeepSearchKBCollections.filter, NewCollection.face_detection_config, nested thumbnail_url in Segmentation/FrameExtraction) — every removal cross-checked against the spec as non-nullable. Zod schemas byte-identical before/after.

Verified: npm run build clean, 106/106 unit tests, and the live six-connector battery re-run at 20/20.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@generate.js`:
- Around line 210-211: Increase both schema backward-search limits in
generate.js: update the rawStart calculation near lines 210-211 to look back
10000 characters, and update the corresponding lookback near line 386 to search
200 lines. Preserve the existing search and widening behavior otherwise.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3adf4254-443d-4bbe-bd53-1dedf5e97636

📥 Commits

Reviewing files that changed from the base of the PR and between 7775f80 and 77db625.

⛔ Files ignored due to path filters (7)

• generated/Collections.ts is excluded by !**/generated/**
• generated/Data_Connectors.ts is excluded by !**/generated/**
• generated/Deep_Search.ts is excluded by !**/generated/**
• generated/Files.ts is excluded by !**/generated/**
• generated/Response.ts is excluded by !**/generated/**
• generated/common.ts is excluded by !**/generated/**
• package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (6)

• generate.js
• package.json
• spec
• src/api/data-connectors.api.ts
• src/api/deep-search.api.ts
• src/types.ts

[kdr]

Addressed the generate.js lookback finding in d663cee — went one step further than the suggested larger limits: since detection is now scoped to a single schema declaration slice, the arbitrary windows are dropped entirely and the search runs to the start of the slice, which is bounded by construction. Regenerated output is byte-identical; build clean, 106/106 unit tests, live six-connector battery re-run 20/20.