Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .github/ACTIONS-REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ GitHub provides two mechanisms for storing configuration values:
| CDK_FILE_UPLOAD_CORS_ORIGINS | Variable | No | None | Platform | Additional CORS origins for the file upload S3 bucket only (appended to global CORS origins) |
| CDK_FILE_UPLOAD_MAX_SIZE_MB | Variable | No | `10` | Platform | Maximum file upload size in megabytes |
| CDK_FINE_TUNING_CORS_ORIGINS | Variable | No | None | SageMaker Fine-Tuning | Additional CORS origins for the fine-tuning S3 bucket only (appended to global CORS origins) |
| CDK_FINE_TUNING_ENABLED | Variable | No | `true` | App API | Mounts the `/fine-tuning` and `/admin/fine-tuning` routers (sets the container's `FINE_TUNING_ENABLED`). Default ON; set to `false` as a kill switch. Storage and IAM are provisioned either way, so switching off never orphans datasets or trained models. |
| CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS | Variable | No | `0` | App API | Default monthly GPU-hour quota for all authenticated users. `0` = whitelist-only (admin must grant each user). Positive value (e.g. `5`) = open access with that default budget. |
| CDK_FRONTEND_BUCKET_NAME | Variable | No | None | Frontend | S3 bucket name for frontend assets (defaults to generated name with account ID) |
| CDK_FRONTEND_CORS_ORIGINS | Variable | No | None | Frontend | Additional CORS origins for the frontend SSM export only (appended to global CORS origins) |
Expand Down
22 changes: 22 additions & 0 deletions .github/docs/deploy/step-03-github-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,28 @@ The per-origin cert vars below are **optional overrides** — set one only if yo
| `CDK_ARTIFACTS_EXTRA_FRAME_ANCESTORS` | — | Comma-separated extra origins (beyond `https://{CDK_DOMAIN_NAME}`) allowed to embed artifact iframes via CSP `frame-ancestors` — applied to both the CloudFront response-headers policy and the render Lambda. Set to `http://localhost:4200` to point a local SPA at this deployment. **Leave unset in production**: every listed origin can frame your users' artifacts (still render-token gated, but a real loosening on a shared environment). |
| `CDK_MCP_SANDBOX_EXTRA_FRAME_ANCESTORS` | — | Comma-separated extra origins (beyond `https://{CDK_DOMAIN_NAME}`) allowed to embed the MCP Apps sandbox proxy via CSP `frame-ancestors`. Set to `http://localhost:4200` to point a local SPA at this deployment. **Leave unset in production.** |
| `CDK_FINE_TUNING_CORS_ORIGINS` | — | Comma-separated extra CORS origins for the SageMaker fine-tuning data bucket, beyond `https://{CDK_DOMAIN_NAME}`. Optional — fine-tuning itself is always provisioned. |
| `CDK_FINE_TUNING_ENABLED` | `true` | Mounts the fine-tuning routers. Default ON — leave unset unless you want the kill switch. Note this is a *runtime* flag on the app-api container; the identically-named CDK stack gate was removed in the single-stack migration. |
| `CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS` | `0` | Monthly GPU-hour quota auto-granted to any authenticated user. `0` = whitelist-only (an admin grants each user). A positive value (e.g. `10`) = open access with that budget. |

### Managed Knowledge Bases

Every variable below is **optional**. Leave them all unset for the shipped state: the managed knowledge-base backend is deployed but **dormant** — no knowledge base is created managed, no migration runs, and the daily reconciler reports what it *would* delete without deleting anything.

The three flags are independent opt-ins that each default to **off**. An unset GitHub Variable arrives at the deploy as an empty string, which is read as off — so forgetting one never silently arms it.

| Variable Name | Default | Description |
|---------------|---------|-------------|
| `CDK_MANAGED_KB_NEW_DEFAULT` | `false` | Set to `true` so newly created knowledge bases are provisioned on the managed backend instead of the legacy one. Existing knowledge bases are untouched. |
| `CDK_MANAGED_KB_MIGRATION_ENABLED` | `false` | Set to `true` to let the background migration worker run at all. While unset, the worker performs no work and its schedule stays disabled. |
| `CDK_MANAGED_KB_RECONCILER_ARMED` | `false` | Set to `true` to let the daily reconciler **delete** orphaned knowledge bases. While unset the reconciler still runs and still logs every deletion it intends to make — review those logs before arming it. |
| `CDK_MANAGED_KB_PER_OWNER_BYTES` | `104857600` (100 MB) | Per-owner stored-bytes cap for the standard role tier, **in bytes**. Deliberately below the 1 GB user-files precedent: at 30,000 users a 1 GB cap permits 30 TB. |
| `CDK_MANAGED_KB_PER_OWNER_ELEVATED_BYTES` | `1073741824` (1 GB) | Per-owner cap for the elevated, admin-granted tier, **in bytes**. |
| `CDK_MANAGED_KB_PER_KB_CEILING_BYTES` | `524288000` (500 MB) | Ceiling for any single knowledge base, **in bytes**, bounding one runaway corpus inside an owner's allowance. |
| `CDK_MANAGED_KB_RETENTION_WINDOW_DAYS` | `30` | How long legacy vector data is kept after a knowledge base is promoted to the managed backend, **in days**, so a rollback stays possible. Do not set below `30`. |
| `CDK_MANAGED_KB_STORAGE_ALARM_GB` | `500` | CloudWatch alarm threshold for **fleet-wide** managed knowledge base storage, **in GB**. The per-owner caps above bound one user; this is the only thing that bounds the whole account. |
| `CDK_MANAGED_KB_DAILY_COST_ALARM_USD` | `100` | CloudWatch alarm threshold for the rolled-up daily Knowledge-Base cost, **in USD**. Set alongside the storage alarm — per-owner caps alone permit roughly two orders of magnitude more spend than expected usage. |

> Accepted values for the three flags are `true`, `false`, `1`, `0`, or empty (empty means off). Anything else fails fast at deploy time with a message naming the variable.

---

Expand Down
61 changes: 61 additions & 0 deletions .github/docs/deploy/step-05-verify.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,67 @@ In order of likelihood:

---

### 6. Subscribe to Platform Alarms (required — not automated)

The deploy creates one SNS topic that **every** CloudWatch alarm in the stack
publishes to. It has no subscribers until you add them, so until you do this step
the alarms change colour in the console and tell nobody.

This is deliberately not infrastructure-as-code. Several teams usually need to
hear about failures, and their membership changes far more often than the
infrastructure does — requiring a pull request, a review, and a CloudFormation
deploy to add one email address is how a notification list goes stale and stops
being trusted. Subscribing is a one-line command that touches no code.

Find the topic and subscribe:

```bash
PREFIX="your-project-prefix" # the same CDK_PROJECT_PREFIX you deployed with

TOPIC=$(aws ssm get-parameter \
--name "/${PREFIX}/observability/alarm-topic-arn" \
--query Parameter.Value --output text)

aws sns subscribe \
--topic-arn "$TOPIC" \
--protocol email \
--notification-endpoint platform-team@example.edu
```

AWS sends a confirmation email; the subscription is inactive until the recipient
clicks the link. Repeat for each address or distribution list.

Other useful protocols:

| Protocol | Use for |
|---|---|
| `email` | A team distribution list. Simplest, and enough for most forks. |
| `https` | PagerDuty, Opsgenie, ServiceNow, or any webhook receiver. |
| `sms` | Genuine paging. Costs per message. |
| `lambda` | Custom routing, e.g. severity-based fan-out or Slack formatting. |

Verify it took:

```bash
aws sns list-subscriptions-by-topic --topic-arn "$TOPIC" \
--query 'Subscriptions[].[Protocol,Endpoint,SubscriptionArn]' --output table
```

A `SubscriptionArn` of `PendingConfirmation` means the email has not been
confirmed yet.

Then open the health dashboard — `{PREFIX}-platform-health` in the CloudWatch
console — and confirm the alarm-status row is populated and green. Row 1 tells
you whether traffic is being served, row 2 tells you why, and row 3 lists every
alarm's current state.

> **Note on latency alarms:** the chat path uses server-sent events, so response
> times of tens of seconds are normal for a healthy agent turn. Latency alarms are
> deliberately set at 120 seconds. A *drop* in latency can actually mean turns are
> failing early.

---

## You're Done!

Your AgentCore Public Stack is deployed and running. Here's what you have:
Expand Down
83 changes: 83 additions & 0 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,89 @@ jobs:
- name: Deploy kb-sync worker image
run: bash scripts/build/deploy-image-lambda-one.sh kb-sync-worker

build-kb-migration:
name: Build kb-migration image
needs: test-backend
# Native ARM64 runner — all four kb-migration Lambdas are arm64 (see the
# managed-kb CDK construct), matching the kb-sync pattern.
runs-on: ubuntu-24.04-arm
environment: ${{ (github.ref == 'refs/heads/main' && 'production') || 'development' }}

permissions:
id-token: write
contents: read

env:
CDK_AWS_REGION: ${{ vars.AWS_REGION }}
CDK_AWS_ACCOUNT: ${{ vars.CDK_AWS_ACCOUNT }}
CDK_PROJECT_PREFIX: ${{ vars.CDK_PROJECT_PREFIX }}
AWS_REGION: ${{ vars.AWS_REGION }}
AWS_ACCOUNT_ID: ${{ vars.CDK_AWS_ACCOUNT }}
AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

outputs:
image_tag: ${{ steps.build.outputs.image_tag }}

steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: ./.github/actions/build-and-push-image
id: build
with:
image-name: kb-migration
aws-region: ${{ vars.AWS_REGION || 'us-west-2' }}
aws-role-arn: ${{ secrets.AWS_ROLE_ARN }}
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

deploy-kb-migration-code:
name: Deploy kb-migration Lambda images
# ONE image, FOUR functions: dispatcher, worker, reconciler and ingestion
# consumer share the kb-migration image and differ only in
# ImageConfig.Command (CDK-owned), so a single job points all four at the
# freshly-built tag.
#
# This is the job that replaces the bootstrap stub
# (infrastructure/bootstrap-assets/kb-migration/) with the real handlers.
# Until it has run once, an enrolled knowledge base sits in `shadow` while
# the dispatcher ticks into a no-op — safe, because the work keys are sparse
# and the first real tick picks up everything that accumulated.
needs: [build-kb-migration, test-backend]
runs-on: ubuntu-24.04
environment: ${{ (github.ref == 'refs/heads/main' && 'production') || 'development' }}

permissions:
id-token: write
contents: read

env:
CDK_AWS_REGION: ${{ vars.AWS_REGION }}
CDK_AWS_ACCOUNT: ${{ vars.CDK_AWS_ACCOUNT }}
CDK_PROJECT_PREFIX: ${{ vars.CDK_PROJECT_PREFIX }}
AWS_REGION: ${{ vars.AWS_REGION }}
AWS_ACCOUNT_ID: ${{ vars.CDK_AWS_ACCOUNT }}
AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: ./.github/actions/configure-aws-credentials
with:
aws-region: ${{ vars.AWS_REGION || 'us-west-2' }}
aws-role-arn: ${{ secrets.AWS_ROLE_ARN }}
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Deploy kb-migration dispatcher image
run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-dispatcher
- name: Deploy kb-migration worker image
run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-worker
- name: Deploy kb-migration reconciler image
run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-reconciler
- name: Deploy kb-migration ingestion consumer image
run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-ingestion-consumer

build-scheduled-runs:
name: Build scheduled-runs image
needs: test-backend
Expand Down
81 changes: 81 additions & 0 deletions .github/workflows/platform.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,11 @@ jobs:
CDK_PROJECT_PREFIX: ${{ vars.CDK_PROJECT_PREFIX }}
CDK_DOMAIN_NAME: ${{ vars.CDK_DOMAIN_NAME }}
CDK_CORS_ORIGINS: ${{ vars.CDK_CORS_ORIGINS }}
# The Environment tag value. Not cosmetic: it is the filter the managed-KB
# reconciler and scripts/teardown/managed-kb.sh match knowledge bases on, so
# an unset value here made a dev deploy tag its knowledge bases 'prod' while
# teardown looked for 'dev' — matching nothing and reporting success.
CDK_TAG_ENVIRONMENT: ${{ vars.CDK_TAG_ENVIRONMENT }}
CDK_VPC_CIDR: ${{ vars.CDK_VPC_CIDR }}
CDK_ALB_SUBDOMAIN: ${{ vars.CDK_ALB_SUBDOMAIN }}
CDK_CERTIFICATE_ARN: ${{ vars.CDK_CERTIFICATE_ARN }}
Expand Down Expand Up @@ -128,6 +133,25 @@ jobs:
# and see the ones their role grants. Which skills a cohort gets is a
# role's `grantedSkills`, managed in the admin roles UI.
CDK_SKILLS_ENABLED: ${{ vars.CDK_SKILLS_ENABLED }}
# SageMaker fine-tuning. The tables, bucket, SageMaker role and IAM
# grants deploy unconditionally, but these two decide whether the feature
# is reachable and how it is rationed, and neither was forwarded before —
# so every deployed environment served 404s from `/fine-tuning/*` while
# the repo variable `CDK_FINE_TUNING_ENABLED` read "true".
#
# ENABLED is default ON with a kill switch: unset resolves to empty
# string, which config.ts treats as the default (on). Set it to "false"
# to dark-stop the routes; storage is untouched either way, so no dataset
# or trained model is orphaned.
#
# DEFAULT_QUOTA_HOURS picks the access model: `0` (the default) is
# whitelist-only, where an admin grants each user explicitly; a positive
# value auto-grants that monthly GPU-hour budget to any signed-in user.
CDK_FINE_TUNING_ENABLED: ${{ vars.CDK_FINE_TUNING_ENABLED }}
CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS: ${{ vars.CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS }}
# Extra CORS origins for the fine-tuning data bucket, beyond the global
# CDK_CORS_ORIGINS above. Also never forwarded until now.
CDK_FINE_TUNING_CORS_ORIGINS: ${{ vars.CDK_FINE_TUNING_CORS_ORIGINS }}
# Agent Designer /agents surface. Default OFF (opt-in) until the Phase-4
# Designer UI ships — a headless API helps no forker. Set the
# `CDK_AGENTS_API_ENABLED` variable to "true" in an environment (e.g. dev)
Expand All @@ -146,6 +170,63 @@ jobs:
# cdk.context.json stay inert.
CDK_MCP_TOKEN_ENRICHMENT_ENABLED: ${{ vars.CDK_MCP_TOKEN_ENRICHMENT_ENABLED }}
CDK_MCP_TOKEN_ENRICHMENT_CLAIMS: ${{ vars.CDK_MCP_TOKEN_ENRICHMENT_CLAIMS }}
# Managed knowledge bases (.kiro/specs/managed-kb-migration). THREE
# INDEPENDENT OPT-IN flags, all defaulting to OFF — the inverse of the
# kill-switch flags above, and the difference matters here. An unset
# GitHub Actions variable renders as an EMPTY STRING, not as absent, so
# a `!== 'false'` reading of an unset variable would resolve to TRUE and
# arm the feature on every fork. config.ts reads these with
# parseBooleanEnv, which maps both unset and empty to undefined and falls
# through to `false` (Requirement 19.8). Leave all three unset to deploy
# the managed backend without starting a fleet migration.
#
# CDK_MANAGED_KB_NEW_DEFAULT new KBs are created managed
# CDK_MANAGED_KB_MIGRATION_ENABLED the background migrator runs at all
# CDK_MANAGED_KB_RECONCILER_ARMED the daily reconciler DELETES orphans
# rather than only reporting them
#
# reconcilerArmed is the inverted one: the Reconciler is deployed and
# running from day one but DISARMED, so its judgement can be reviewed
# against real data before it deletes anything (Requirements 14.7, 19.7).
CDK_MANAGED_KB_NEW_DEFAULT: ${{ vars.CDK_MANAGED_KB_NEW_DEFAULT }}
CDK_MANAGED_KB_MIGRATION_ENABLED: ${{ vars.CDK_MANAGED_KB_MIGRATION_ENABLED }}
CDK_MANAGED_KB_RECONCILER_ARMED: ${{ vars.CDK_MANAGED_KB_RECONCILER_ARMED }}
# Storage cost controls. Byte_Caps are in BYTES (Requirement 12.2),
# defaulting to 100 MB standard / 1 GB elevated / 500 MB per knowledge
# base; the retention window is in DAYS and must stay >= 30
# (Requirement 15.11). Leave unset to take those defaults — these exist
# so an environment can tune them without a code change.
CDK_MANAGED_KB_PER_OWNER_BYTES: ${{ vars.CDK_MANAGED_KB_PER_OWNER_BYTES }}
CDK_MANAGED_KB_PER_OWNER_ELEVATED_BYTES: ${{ vars.CDK_MANAGED_KB_PER_OWNER_ELEVATED_BYTES }}
CDK_MANAGED_KB_PER_KB_CEILING_BYTES: ${{ vars.CDK_MANAGED_KB_PER_KB_CEILING_BYTES }}
CDK_MANAGED_KB_RETENTION_WINDOW_DAYS: ${{ vars.CDK_MANAGED_KB_RETENTION_WINDOW_DAYS }}
# Fleet-level alarm thresholds (Requirement 12.13). The Byte_Caps above
# bound ONE owner; these two bound the whole account, which is the gap
# between ~$169/month expected and ~$15,000/month that per-owner caps
# alone permit. Storage is in GB (default 500), daily cost in USD
# (default 100). Leave unset to take those defaults.
CDK_MANAGED_KB_STORAGE_ALARM_GB: ${{ vars.CDK_MANAGED_KB_STORAGE_ALARM_GB }}
CDK_MANAGED_KB_DAILY_COST_ALARM_USD: ${{ vars.CDK_MANAGED_KB_DAILY_COST_ALARM_USD }}
# Observability. All optional — unset means the default in config.ts.
# XRAY_SAMPLING_RATE is a rate (0.0-1.0), not a percentage.
CDK_OBSERVABILITY_ALARM_TOPIC_ENABLED: ${{ vars.CDK_OBSERVABILITY_ALARM_TOPIC_ENABLED }}
CDK_OBSERVABILITY_LOG_RETENTION_DAYS: ${{ vars.CDK_OBSERVABILITY_LOG_RETENTION_DAYS }}
CDK_OBSERVABILITY_ALB_TARGET_5XX_THRESHOLD: ${{ vars.CDK_OBSERVABILITY_ALB_TARGET_5XX_THRESHOLD }}
CDK_OBSERVABILITY_ALB_P99_LATENCY_MS: ${{ vars.CDK_OBSERVABILITY_ALB_P99_LATENCY_MS }}
CDK_OBSERVABILITY_AGENTCORE_LATENCY_MS: ${{ vars.CDK_OBSERVABILITY_AGENTCORE_LATENCY_MS }}
CDK_OBSERVABILITY_AGENTCORE_ERROR_THRESHOLD: ${{ vars.CDK_OBSERVABILITY_AGENTCORE_ERROR_THRESHOLD }}
CDK_OBSERVABILITY_LAMBDA_ERROR_THRESHOLD: ${{ vars.CDK_OBSERVABILITY_LAMBDA_ERROR_THRESHOLD }}
CDK_OBSERVABILITY_LAMBDA_DURATION_PERCENT_OF_TIMEOUT: ${{ vars.CDK_OBSERVABILITY_LAMBDA_DURATION_PERCENT_OF_TIMEOUT }}
CDK_OBSERVABILITY_DYNAMO_THROTTLE_THRESHOLD: ${{ vars.CDK_OBSERVABILITY_DYNAMO_THROTTLE_THRESHOLD }}
CDK_OBSERVABILITY_ECS_CPU_PERCENT: ${{ vars.CDK_OBSERVABILITY_ECS_CPU_PERCENT }}
CDK_OBSERVABILITY_ECS_MEMORY_PERCENT: ${{ vars.CDK_OBSERVABILITY_ECS_MEMORY_PERCENT }}
CDK_OBSERVABILITY_XRAY_SAMPLING_RATE: ${{ vars.CDK_OBSERVABILITY_XRAY_SAMPLING_RATE }}
CDK_OBSERVABILITY_XRAY_SAMPLING_RESERVOIR: ${{ vars.CDK_OBSERVABILITY_XRAY_SAMPLING_RESERVOIR }}
CDK_OBSERVABILITY_XRAY_INSIGHTS_NOTIFICATIONS: ${{ vars.CDK_OBSERVABILITY_XRAY_INSIGHTS_NOTIFICATIONS }}
CDK_OBSERVABILITY_AGENTCORE_APPLICATION_LOGS_ENABLED: ${{ vars.CDK_OBSERVABILITY_AGENTCORE_APPLICATION_LOGS_ENABLED }}
CDK_OBSERVABILITY_PROMPT_CACHE_AVOIDABLE_MISS_THRESHOLD: ${{ vars.CDK_OBSERVABILITY_PROMPT_CACHE_AVOIDABLE_MISS_THRESHOLD }}
CDK_OBSERVABILITY_PROMPT_CACHE_WASTED_USD_THRESHOLD: ${{ vars.CDK_OBSERVABILITY_PROMPT_CACHE_WASTED_USD_THRESHOLD }}
CDK_OBSERVABILITY_PROMPT_CACHE_SESSION_WASTED_USD_THRESHOLD: ${{ vars.CDK_OBSERVABILITY_PROMPT_CACHE_SESSION_WASTED_USD_THRESHOLD }}
# Secrets
AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
Expand Down
1 change: 1 addition & 0 deletions .kiro/specs/managed-kb-migration/.config.kiro
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"specId": "612a1431-367c-427e-8fe0-08872d03a9b9", "workflowType": "design-first", "specType": "feature"}
Loading
Loading