Skip to content

gdrive-log connector - #1414

Open
eschultink wants to merge 4 commits into
rc-v0.7.1from
s233-proxy-gdrive
Open

gdrive-log connector#1414
eschultink wants to merge 4 commits into
rc-v0.7.1from
s233-proxy-gdrive

Conversation

@eschultink

Copy link
Copy Markdown
Member

Features

  • Add beta gdrive-log connector for Drive events from Workspace audit logs.

Change implications

  • dependencies added/changed? no
  • something important to note in future release notes? no
  • breaking changes? no

eschultink and others added 4 commits September 1, 2026 15:16
…ogs.

Restricts the proxy to the Drive Reports API activity endpoint, covering views and comments that the files API misses, with Google's ~6-month retention limit.

Co-authored-by: Cursor <cursoragent@cursor.com>
…l test tooling.

Check AWS/Google/Azure auth and the Terraform backend before apply, and install psoxy-test in this checkout when node_modules is missing so worktrees do not inherit a false "already installed" from remote state.

Co-authored-by: Cursor <cursoragent@cursor.com>
…n, gdrive drive.readonly, and release tooling.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ient IPs.

Drive audit events leak titles, emails, Gaia IDs, and residential IPs outside the event-parameter allowlist; hash those and expand the example fixtures to cover the extra event types.

Co-authored-by: Cursor <cursoragent@cursor.com>
@eschultink eschultink self-assigned this Sep 4, 2026
Copilot AI lite review requested due to automatic review settings September 4, 2026 21:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new redact allowlist regex is incorrectly anchored (missing grouping) and the updated test-tool installer now fails hard when npm/tool source is missing, which can break Terraform/examples-dev flows.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a new beta gdrive-log connector for ingesting Google Drive events from Google Workspace audit logs (Admin Reports API), with corresponding sanitizer rules, tests, Terraform connector specs, and end-user documentation. The PR also refactors examples-dev apply flows to run a shared preflight + ensure the worktree-local psoxy-test tool deps are installed.

Changes:

  • Add gdrive-log sanitizer rules (Java + YAML) and a dedicated JUnit test fixture.
  • Register the new connector in Terraform specs and update docs/navigation to expose the connector and its setup/scopes.
  • Introduce shared examples-dev ./apply preflight/apply tooling and make test-tool installation worktree-aware.
File summaries
File Description
tools/install-test-tool.sh Makes psoxy-test dependency install idempotent/worktree-aware (but now exits non-zero on missing prereqs).
tools/examples-dev-apply.sh Shared apply wrapper that runs preflight + installs test tool + prompts for bundle rebuild.
tools/examples-dev-apply-preflight.sh New preflight validations for backend/tfvars and cloud CLI auth for examples-dev overlays.
java/core/src/test/java/co/worklytics/psoxy/rules/google/GDriveLogTests.java New sanitizer test coverage for the gdrive-log rules.
java/core/src/main/java/co/worklytics/psoxy/rules/google/PrebuiltSanitizerRules.java Adds and registers GDRIVE_LOG prebuilt rules, including parameter allowlist + redactions.
infra/modules/worklytics-connector-specs/google-workspace.tf Adds gdrive-log connector metadata/spec to the Workspace connector catalog.
infra/modules/psoxy-test-tool/main.tf Documents worktree vs remote-state behavior for local-exec install of psoxy-test.
infra/examples-dev/README.md Documents using ./apply and explains the new preflight + test-tool install behavior.
infra/examples-dev/gcp/google-workspace-variables.tf Treats gdrive-log as a Workspace connector for validation gating.
infra/examples-dev/gcp/apply Switches to shared tools/examples-dev-apply.sh.
infra/examples-dev/aws/google-workspace-variables.tf Treats gdrive-log as a Workspace connector for validation gating.
infra/examples-dev/aws/apply Switches to shared tools/examples-dev-apply.sh.
infra/examples-dev/aws/.gitignore Removes ignore entry for a helper script no longer present/used.
docs/SUMMARY.md Adds the new connector docs page to the sidebar.
docs/sources/README.md Adds gdrive-log to the connectors index table.
docs/sources/google-workspace/README.md Adds gdrive-log to Workspace connector list/scope table and admin privilege notes.
docs/sources/google-workspace/gdrive/README.md Cross-links to gdrive-log for audit-log-based Drive events.
docs/sources/google-workspace/gdrive-log/README.md New connector documentation page (ID, caveats, scopes, examples).
docs/sources/google-workspace/gdrive-log/gdrive-log.yaml New example rules YAML for the connector.
docs/sources/google-workspace/gdrive-log/example-api-responses/sanitized/drive-activities.json Sanitized sample payload demonstrating transforms.
docs/sources/google-workspace/gdrive-log/example-api-responses/original/drive-activities.json Original sample payload for reference/testing.
docs/README.md Adds the new connector to the top-level connector summary table.
Review details

Suppressed comments (1)

tools/install-test-tool.sh:38

  • Exiting with status 1 when npm is missing makes tools/install-test-tool.sh a hard dependency, which can cause terraform apply (via the psoxy-test-tool module) to fail on machines without Node.js, contrary to the module’s “no effect if no NPM” intent. Suggest warning and exiting 0 so Terraform/dev apply can proceed (with tests unavailable).
if ! command -v npm >/dev/null 2>&1; then
  printf "${ERR}NPM / Node.JS not available; could not install test tool. Install Node.js (https://nodejs.org/ LTS version preferred), then re-run.${NC}\n"
  exit 1
fi
  • Files reviewed: 24/24 changed files
  • Comments generated: 4
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

- "$.items[*].events[*].parameters[?(@.name == 'ip_address')].multiValue[*]"
- !<redact>
jsonPaths:
- "$.items[*].events[*].parameters[?(!(@.name =~ /^owner|target_user|new_owner|delegating_principal|actor_is_collaborator_account|billable|primary_event|doc_id|doc_type|is_encrypted|originating_app_id|owner_is_shared_drive|owner_shared_drive_id|shared_drive_id|visibility|visibility_change|old_visibility|added_role|removed_role|requested_role|destination_folder_id|source_folder_id|copy_type|new_owner_is_shared_drive|new_owner_shared_drive_id|target_domain|membership_change_type|ip_address$/i))]"
Comment on lines +391 to +393
.jsonPath("$.items[*].events[*].parameters[?(!(@.name =~ /^" +
String.join("|", GDRIVE_LOG_EVENT_PARAMETERS_ALLOWED) +
"$/i))]")
Comment on lines +21 to 24
if [ ! -d "${TEST_TOOL_ROOT}" ]; then
printf "${ERR}No test tool source found at ${TEST_TOOL_ROOT}. Failed to install test tool.${NC}\n"
exit
exit 1
fi
Comment on lines +7 to +8
Please review the [Google Workspace&trade; README](../README.md) for general information applicable to
all Google Workspace connectors.

---
Google Workspace&trade; and related marks are trademarks of Google LLC.
Worklytics&trade; is a trademark of Worklytics, Corp.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worklytics, Co.

Comment thread docs/sources/README.md
@@ -23,6 +23,7 @@ To add a source, add its Connector ID to the `enabled_connectors` list in your `
| `gcal` | [Google Calendar](google-workspace/calendar/README.md) | API | GA |
| `gdirectory` | [Google Directory](google-workspace/directory/README.md) | API | GA |
| `gdrive` | [Google Drive](google-workspace/gdrive/README.md) | API | GA |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DEPRECATED

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like out of scope change?


Each example includes a `reset-example` symlink to [`tools/reset-example.sh`](../../tools/reset-example.sh) for resetting local IaC state during development (back up / recover `terraform.tfvars`, etc.).

Use `./apply` (not a bare `terraform apply`) so auth and backend checks run first. That helper calls [`tools/examples-dev-apply-preflight.sh`](../../tools/examples-dev-apply-preflight.sh), which requires `terraform.tfvars` and `backend.tf`, confirms the initialized Terraform backend matches `backend.tf` (prefer a remote backend so worktrees share state), and checks AWS, Google, and Azure CLI auth only when that example actually needs them. A GCS backend needs Google ADC even on the AWS example. Microsoft 365 (`msft_tenant_id`) needs Azure CLI, preferably sandboxed via `./az-auth` / `./auth` into `.azure`. It also installs `tools/psoxy-test` npm deps in this worktree when they are missing; Terraform state from another worktree is not enough, because `node_modules` is local and gitignored.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like out of scope change?

{
"userIdentity": {
"id": "0neiluseridxxxxx",
"userEmail": "neil@worklytics.co"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"userEmail": "neil@worklytics.co"
"userEmail": "bob@worklytics.co"

"ownerIdentity": [
{
"userIdentity": {
"id": "0neiluseridxxxxx",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"id": "0neiluseridxxxxx",
"id": "0bobuseridxxxxx",

.add("ip_address")
.build();

static final RESTRules GDRIVE_LOG = Rules2.builder()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have YAML generated rules for Gemini; we should do the same for gdrive instead of keep adding java rules here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants