Add redshift-support-specialist skill and custom agent#30
Open
andrehass wants to merge 23 commits into
Open
Conversation
Adds the redshift-support-specialist skill for query optimization, operational reviews, disaster recovery guidance, incident detection guidance, and cost optimization on Amazon Redshift provisioned clusters and Serverless workgroups. The skill talks to Redshift exclusively through the six tools exposed by the awslabs.redshift- mcp-server MCP server (list_clusters, list_databases, list_schemas, list_tables, list_columns, execute_query) and includes a ready-to-use serverless (Lambda + API Gateway) deployment of that MCP server. Also adds a companion redshift-support-specialist custom agent that orchestrates the skill and forces active-chat-only execution (never background), and documents both the deployment and the skill/agent usage in their respective READMEs. Updates llms.txt with the new skill entry.
Removes the Lambda Function URL endpoint from the Redshift MCP server deployment, leaving a single API Gateway REST API (execute-api, AWS_IAM/SigV4) as the only way to reach the MCP server. - template.yaml: drops FunctionUrlConfig, CallerInvokePermission, and the Function URL invoke policy/output. - deploy.sh: replaces Function URL creation with API Gateway provisioning (REST API, /mcp resource, method, integration with caller-identity passthrough, Prod stage deployment); caller grants now use put-role-policy for execute-api:Invoke + lambda:InvokeFunction. - deployer-permissions-policy.json: swaps Function URL Lambda actions for apigateway:* management and iam:PutRolePolicy. - scripts/*.py: default MCP_SIGV4_SERVICE to execute-api and update usage examples to the API Gateway URL shape. - README.md / sam-app/README.md: updated architecture, deploy, auth, invoke-grant, testing, and teardown sections to match the single-endpoint setup.
Adds a "Quick deploy (no prompts)" command as the primary deploy instruction in both the main skill README and deployment/sam-app/README.md: a single sam deploy call using --capabilities CAPABILITY_NAMED_IAM, --resolve-s3, --no-confirm-changeset, and --no-fail-on-empty-changeset, with all template defaults and --parameter-overrides for customization. Always using CAPABILITY_NAMED_IAM (a superset of CAPABILITY_IAM) removes the need to branch on whether CreateDevOpsAgentRole is set, so the same command works either way and avoids the "Requires capabilities: [CAPABILITY_NAMED_IAM]" changeset failure users hit with sam deploy --guided's default capability. The previous sam deploy --guided flow is kept as a secondary "Guided deploy" option for anyone who wants interactive prompts, with the capability gotcha still called out there. Verified with sam validate --lint against the current template.
Restructures the main skill README's Prerequisites section into a strict, numbered dependency chain (Step 1: Agent Space, Step 2: deploy the MCP server + test it, Step 3: register/connect the MCP server to the Agent Space) instead of listing the capability-provider registration steps before the deployment steps they depend on. Adds a "Setup Overview" block right after the architecture diagram with anchor links to each step, so a first-time reader sees the full flow before the details. Adds a real "Optional: Create a Custom Agent" section linking to custom-agents/redshift-support-specialist/, which was previously only mentioned once in a buried Skill Contents note. Custom agent README and deployment/README.md cross-references updated to the new step anchors.
Adds the two-step GitHub connection setup (register at Capability Providers, then attach via the Agent Space's Capabilities tab -> Pipeline section) to the skill's "Option A: Import from GitHub" instructions, based on the actual console navigation path (Capability Providers is account-level and separate from the Agent Space's own Capabilities tab).
Renumbers the setup flow to align Setup Overview with the actual step headings: Step 1 (MCP Server Deployment), Step 2 (Connect the MCP server to your Agent Space), Step 3 (Create the redshift-support-specialist skill), Step 4 (Create the Custom Agent, no longer marked optional), Step 5 (How to Use the Skill). Consolidates all setup prerequisites (Agent Space, deploy tools, IAM permissions to deploy) into a single Prerequisites section before Step 1, with detail pointers to deployment/README.md. Moves Limitations, Agent Types, Architecture, and How the Pieces Fit Together to the end of the document, after Skill Contents and before License, so the step-by-step setup flow reads uninterrupted. Updates cross-references in custom-agents/redshift-support-specialist/README.md and deployment/README.md to match the renumbered step anchors.
The Lambda's IAM execution role had no explicit name in the SAM template, so CloudFormation auto-generated one with a random suffix (e.g. redshift-mcp-RedshiftMcpFunctionRole-LWxE1aGLRtLI) -- leaving no stable name to grant database-level permissions against. Earlier docs also referenced the Lambda *function* name instead of its *execution role*, and used the IAM: prefix (for IAM users) instead of IAMR: (for IAM roles) when deriving the Redshift database username. - sam-app/template.yaml: adds an explicitly named RedshiftMcpFunctionRole (redshift-mcp-lambda-execution-role) and points the function's Role property at it. Adds GrantSysMonitorCommand and GrantTableInfoCommand stack outputs that generate the exact GRANT SQL with the real deployed role name. - deployer-permissions-policy.json: updates the IamRoleManagement and PassRoleToLambda resource ARNs to the new role name. - deployment/README.md, README.md, sam-app/README.md: correct IAM: to IAMR: for IAM roles, document the sys:monitor and SVV_TABLE_INFO grants needed for the skill to read other users' query data, and point to the new stack outputs (or deploy.sh's printed commands) instead of assuming a fixed role name. Verified against a live deployment: confirmed the previously-deployed stack's Lambda execution role name via `aws lambda get-function`, and validated the updated SAM template with `sam validate --lint`.
- Correct Step 3 to describe two upload options instead of three, and number the GitHub import steps consistently - Fix skill upload navigation path to Knowledge -> Skills -> Add Skill for both GitHub import and zip upload options - Remove outdated background-mode parenthetical from Step 4 intro
default to background mode. Background execution only when the user explicitly asks, after cluster/database scope is confirmed (SKILL.md Core Rules 10/11, custom agent Section 0) - Treat empty query results as healthy outcomes with a friendly PASS message instead of reporting them as failed - Bump skill to 1.7.0 and custom agent to 1.1.0 with changelogs
…agent with async model Skill v1.8.0 / custom agent v1.3.0. - Remove Capability 4 (Disaster Recovery) and 5 (Incident Detection): both were reference-guidance-only with no live data path. Removed pending a proper CloudWatch MCP server integration. All alarm/DR references scrubbed from instructions and reference docs; orphaned reference files deleted (preserved in the v1.7.0 tag) - Rewrite custom agent Section 0 to 'no scope, no run': custom agents always execute as asynchronous invocations, so scope must come from the invocation prompt; missing scope ends the run with a 'Scope required' report instead of guessing - Document interactive (skill in Chat) vs autonomous (custom agent) use, canceling a background run, downloading the HTML report from Artifacts, the capability-discovery prompt, and version numbers - Fix DevOpsAgentRole trust policy to allow both service/* and agentspace/* SourceArn patterns; raise samconfig capabilities to CAPABILITY_NAMED_IAM
…th detailed breakdowns
…1.8.0 - audit: 98/100 Grade A, 0 criticals, 0 warnings, passed - functional: 94.4% with-skill vs 15.3% without (+79.2%), PARETO_BETTER, passed - trigger: 100% trigger precision, 100% no-trigger precision (2 runs/query), passed - unified report: 0.87 Grade B, passed: true - reword one trigger query to include the skill's operational-review trigger vocabulary
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds the redshift-support-specialist skill (v1.8.0) and its companion custom agent (v1.3.0): Amazon Redshift domain expertise for AWS DevOps Agent, plus a ready-to-use serverless deployment of the
awslabs.redshift-mcp-serverMCP server the skill relies on.Skill capabilities (all live via the six Redshift MCP tools — no AWS CLI access, no CSV uploads, read-only):
What's included:
skills/redshift-support-specialist/— SKILL.md, references, query templates, report templates, thresholds, evals (with committed Agent Skill Eval results)custom-agents/redshift-support-specialist/— system prompt for an autonomous custom agent with a "no scope, no run" guard (custom agents execute as async invocations, so scope must come from the invocation prompt; missing scope ends the run with a "Scope required" report instead of guessing)skills/redshift-support-specialist/deployment/— serverless MCP server deployment (Lambda zip + API Gateway with SigV4/IAM auth, via AWS SAM or plain CLI; no containers, no ECR, no VPC), including a documented fix for the DevOps Agent SigV4 role trust policy (agentspace/*SourceArn is required in addition to the documentedservice/*)llms.txtupdatedType of change
Testing
Agent Skill Eval (results committed under
skills/redshift-support-specialist/evals/):audit: 98/100, Grade A — 0 criticals, 0 warnings, passedfunctional: 94.4% with-skill vs 15.3% without-skill assertion pass rate (+79.2%), classified PARETO_BETTER (quality up, ~50% fewer tokens), passedtrigger: 100% trigger precision and 100% no-trigger precision (6 queries, 2 runs each), passedreport: 0.87, Grade B,"passed": trueManual DevOps Agent testing (live Agent Space + deployed MCP server against real provisioned clusters and Serverless workgroups):
License confirmation