Skip to content

fix(lambda): add an opt-in cleanup sweep - #986

Merged
cristim merged 3 commits into
mainfrom
fix/712-lambda-cleanup-sweep
Oct 17, 2026
Merged

cristim merged 3 commits into
mainfrom
fix/712-lambda-cleanup-sweep

Conversation

@cristim

@cristim cristim commented Oct 16, 2026

Copy link
Copy Markdown
Member

Part of #712. This adds an opt-in cleanup sweep on AWS Lambda ("A1b2" in the #712 plan) and corrects the retention docs.

Changes

  • terraform/modules/compute/aws/lambda
    • New variables: enable_cleanup_schedule (default false) and cleanup_schedule (rate(1 day)).
    • A cleanup entry in sweep_tasks, which creates <stack>-sweep-cleanup. That name does not collide with the unconsumed aws/cleanup-lambda module's <stack>-cleanup rule.
  • terraform/environments/aws: the same variable pair, passed to module "compute_lambda" only.
  • tests/sweeps.tftest.hcl: cleanup is absent in the defaults run and present in the opt-in run, with the same name, schedule and input assertions as the other sweeps.
  • scheduled_task_coverage_test.go: the aws-lambda knownGaps entry, which held only cleanup, is removed. A missing cleanup sweep or pass-through now fails the test.
  • scheduled_task_env_test.go: tfvarsOffRe now matches exactly false or "false", optionally followed by a # or // comment. It no longer matches an unterminated "false or falsey. New fixtures cover both.
  • docs/DEPLOYMENT.md, "Database Cleanup Jobs"
    • The retention sentence now matches CleanupOldExecutions(30) exactly.
    • It names the AWS Lambda opt-in variables and warns that the first run deletes the whole backlog older than 30 days.
    • It says Azure Container Apps schedules cleanup by default.
  • known_issues/33_executions_retention_cleanup.md
    • Status is now "Partially resolved", because retention is unscheduled by default on AWS, GCP and CloudFormation.
    • Adds the AWS Lambda opt-in bullet.
    • Fixes the stale handler line reference.

Why opt-in

The owner decided this after the retention finding. cleanup deletes expired sessions and, through CleanupOldExecutions(30):

  • completed executions whose scheduled_date is more than 30 days old;
  • canceled executions whose updated_at is more than 30 days old;
  • executions with any other non-failed status whose expires_at is more than 30 days past.

Failed executions are kept. Nothing has ever run cleanup on AWS, so the first enabled run deletes the whole backlog. Operators enable it per environment after deciding whether to keep that history.

Deploy note

Merging triggers deploy-aws-lambda.yml to dev, because it touches terraform/environments/aws/** and the lambda module. The default is off, so the dev apply creates no new rule and changes no behaviour.

Verification (macOS)

  • Terraform 1.10.5 (CI's pin) and 1.14.7, on the lambda module: init -backend=false -lockfile=readonly, fmt -check -recursive, validate and test all pass (2 runs). terraform/environments/aws validates.

  • Go:

    • go test ./terraform/modules/compute ./internal/server -count=3 -shuffle=on passes.
    • go vet ./... is clean.
    • make lint (go1.26.9) reports 0 issues.
    • Re-run after rebasing onto main.
  • Mutation probes (each reverted, each makes a test fail):

    • cleanup default set to true: the tftest defaults run fails;
    • cleanup pass-through dropped from compute.tf: "undocumented gap: cleanup";
    • cleanup entry removed from sweep_tasks;
    • old tfvarsOffRe restored: the new fixture fails;
    • #-only comment handling restored: the // note fixture fails.

    A scratch tfvars containing enable_finalize_revocations_schedule = false // x is now caught.

  • The retention text was checked against internal/config/store_postgres.go (CleanupOldExecutions), types.go (HealthScoredExecutionStatuses) and internal/server/handler.go:272.

  • Not verified: a real AWS run of cleanup. It stays off until an operator enables it.

Review

A non-author adversarial reviewer ran two passes. Pass 1 found three low-severity issues, all fixed: stale known_issues wording, the // comment regression, and the Azure wording. Pass 2 had no actionable findings.

The rebase onto main after #949 merged is clean: range-diff = on all three commits. A fresh independent gate review at the head SHA is still needed.

🤖 Generated with Claude Code

cristim and others added 3 commits October 16, 2026 15:42
Add enable_cleanup_schedule (default false) and cleanup_schedule to the
Lambda module and the AWS environment, a cleanup entry in sweep_tasks
(rule name <stack>-sweep-cleanup, clear of the unconsumed cleanup-lambda
module's <stack>-cleanup), tftest assertions for the default and opt-in
runs, and drop cleanup from the aws-lambda knownGaps.

The server cleanup task deletes expired sessions and, via
CleanupOldExecutions(30), completed executions (scheduled_date), canceled
executions (updated_at) and executions outside failed/canceled
(expires_at) older than 30 days, so it stays opt-in: the first run
deletes the whole backlog.

Part of #712

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
State the exact CleanupOldExecutions(30) retention in DEPLOYMENT.md and
add the AWS Lambda opt-in variables. In known_issues/33 correct the
Status (retention is unscheduled by default on AWS, GCP and CFN), fix the
handler line reference and note the Lambda opt-in.

Part of #712

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The old pattern also matched an unterminated "false and values such as
falsey. Add a negative fixture for both.

Part of #712

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cristim cristim added urgency/this-sprint Within the current sprint priority/p1 Next up; this sprint severity/high Significant harm impact/many Affects most users effort/m Days type/bug Defect triaged Item has been triaged labels Oct 16, 2026
@coderabbitai

coderabbitai Bot commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

This review includes 9 billable files and costs up to $2.25.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 46 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 65 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: LeanerCloud/cloud-commitments-platform/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 28f60b4e-476d-41f4-bf33-4727a6dc69e9
📥 Commits

Reviewing files that changed from the base of the PR and between f44330b and d58ec6e.

📒 Files selected for processing: 9
  • docs/DEPLOYMENT.md
  • known_issues/33_executions_retention_cleanup.md
  • terraform/environments/aws/compute.tf
  • terraform/environments/aws/variables.tf
  • terraform/modules/compute/aws/lambda/sweeps.tf
  • terraform/modules/compute/aws/lambda/tests/sweeps.tftest.hcl
  • terraform/modules/compute/aws/lambda/variables.tf
  • terraform/modules/compute/scheduled_task_coverage_test.go
  • terraform/modules/compute/scheduled_task_env_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@cristim

cristim commented Oct 16, 2026

Copy link
Copy Markdown
Member Author

Independent review verdict: CLEAN at d58ec6e

Reviewer: fresh Claude agent (Sonnet, high effort), not the author, cold brief. The PR body was treated as a claim. No cloud credentials were used, and no plan or apply was run against a real backend. Coordinator: claude-cc-reviewer.

Blocking findings: none.

Default dev apply should be a no-op:

  • The new cleanup entry joins sweep_tasks (terraform/modules/compute/aws/lambda/sweeps.tf:13).
  • active_sweeps filters on enabled, and enable_cleanup_schedule defaults to false (lambda/variables.tf:270; environments/aws/variables.tf:375).
  • Existing sweep addresses stay keyed by task name, so nothing is renamed or recreated. With defaults, aws_cloudwatch_event_rule.sweep holds only finalize_revocations, as before.
  • No tfvars in terraform/ sets the variable.
  • The evidence for "no-op" is the mocked terraform test defaults run plus a read of the for_each gating, not a real plan.

When enabled:

  • The rule is <stack>-sweep-cleanup with input {action="cleanup"}. It does not collide with aws/cleanup-lambda's <stack>-cleanup (main.tf:100).
  • The task exists: internal/server/handler.go:82 maps cleanup to TaskCleanupExpiredRecords (:37), which is dispatched at :192 to handleCleanupExpiredRecords (:272). That calls CleanupOldExecutions(ctx, DefaultExecutionTTLDays=30) (constants.go:15).

Docs match the SQL (store_postgres.go:2150-2163):

  • completed rows are deleted by scheduled_date;
  • canceled rows are deleted by updated_at;
  • other statuses are deleted by expires_at, with a 30-day age limit;
  • failed rows are kept.

The first-run backlog warning is in DEPLOYMENT.md, both variable descriptions and known_issues/33.

What was attacked and how:

  • go test -count=1 ./terraform/modules/compute/...: 34 tests pass, in 4 packages.
  • terraform init -backend=false && terraform test in the lambda module, with a mocked aws provider: 2/2 runs pass (defaults, opt_in).
  • terraform fmt -check -recursive on the touched directories is clean.
  • Mutation: deleting the cleanup sweep entry makes the coverage test fail ("undocumented gap: cleanup").
  • Regex probes for tfvarsOffRe:
    • These match: false, "false", false # c, false // c, false\r, and false with trailing spaces.
    • These don't: unterminated "false, falsey, true # false.
    • Non-blocking: false/* c */ doesn't match either, which is not a claimed case.

Gate:

@cristim

cristim commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

Merge driver status (claude-cc-pr-merge, 2026-10-17T15:25:27Z): code gate met, merge BLOCKED on @cristim's explicit OK for the dev deploy. Merging to main runs the existing deploy workflows (Terraform apply to dev), and merge-driver sessions are not permitted to trigger that, so a maintainer must merge or approve it (the review expects a no-op apply with defaults).

@cristim

cristim commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

Refreshing CI (claude-cc-pr-merge-2): main changed terraform/modules/compute/aws/lambda and terraform/environments/aws (#1042, #997) after this PR's CI ran. The merge-tree is clean, but CI must re-run on the current merge ref before merging. Reopening now.

@cristim cristim closed this Oct 17, 2026
@cristim cristim reopened this Oct 17, 2026
@cristim

cristim commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

Gate record for d58ec6e (claude-cc-pr-merge-2):

  • Independent adversarial review: CLEAN at this exact SHA (fix(lambda): add an opt-in cleanup sweep #986 (comment); non-author, cold brief, coordinated by claude-cc-reviewer). The local verification evidence and its limits (fixture or mock level, no real-account apply) are in that verdict.
  • Owner hold lifted 2026-10-17 (~/.claude/decisions/2026-10-17-platform-holds-lifted.md). Opt-in Lambda cleanup sweep (off by default); merging applies to dev.
  • CI: required checks green on this SHA, re-checked by merge-gated.sh at merge time, including the guard against main changing the same packages since CI ran.
  • Labels mirror the linked issue. No unresolved review threads. CodeRabbit: skipped (quota).
  • Merging with squash and --match-head-commit; no --admin.

@cristim
cristim merged commit cb875f4 into main Oct 17, 2026
66 of 68 checks passed
@cristim
cristim deleted the fix/712-lambda-cleanup-sweep branch October 17, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/m Days impact/many Affects most users priority/p1 Next up; this sprint severity/high Significant harm triaged Item has been triaged type/bug Defect urgency/this-sprint Within the current sprint

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant