Skip to content

Strengthen application security and multi-tenant isolation guidance #640

Description

@devantler

Goal

Complement existing GitHub security scanning with design-time and test-time application security guidance for customer software and SaaS.

Scope

Threat modeling, OWASP ASVS/API Security Top 10, authentication, authorization (object-level checks), tenant boundaries, secure session management, secrets, webhook validation, rate limits, file uploads, secure defaults and incident response. Include cross-tenant data-access test cases and explicit risk escalation.

Mandatory adoption-first workflow

  1. Inventory existing first-party skills and identify precise gaps; avoid duplication.
  2. Search reputable maintained third-party skills first. Evaluate quality, license/redistribution, security, compatibility, maintenance, provenance and scope.
  3. Prefer adopting a suitable upstream skill with an immutable version/commit pin and a reviewable automated update process; upstream changes must be summarized and tested before bumping the pin. Align with existing issues Bundled upstream skills: reviewable updates and clear redistribution terms #581, Updater PRs state what changed upstream for each moved skill #583 and Review and safely release the temporary git-commit upstream pin #598 rather than duplicating their implementation.
  4. Only author a new first-party skill when documented evaluation shows no adequate candidate. Before implementation, research authoritative standards, primary sources, realistic scenarios, failure modes, acceptance tests and maintenance ownership.
  5. Author first-party skills using the repository's skill-authoring skill and applicable Agent Skills specification/industry conventions; validate metadata, progressive disclosure, triggering, examples and automated evaluation. Never silently claim legal compliance.

Acceptance criteria

  • Audit existing coverage first and compare maintained upstream skills.
  • Threat model and automated test examples demonstrating blocked cross-tenant access.
  • Guidance triggers on new endpoints, auth changes, tenant data access and payment webhooks.
  • No redundant scanning instructions.

Activity

  1. coderabbitai commented on Nov 4, 2026

    @coderabbitai
    🔗 Related PRs

    #613 - chore: adopt the Marketplace skill updater [merged]


    📝 Issue Planner

    Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.

    • Create Plan

    🧪 Issue enrichment is currently in open beta.

    You can configure auto-planning by selecting labels in the issue_enrichment configuration.

    To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

    issue_enrichment:
      auto_enrich:
        enabled: false

    💬 Have feedback or questions? Drop into our discord!

  2. moved this to 📥 Backlog in 🌊 Project Boardon Nov 4, 2026
  3. added theissue type on Nov 8, 2026
  4. devantler commented on Nov 10, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer (interactive Claude Code session)

    Research brief

    Gap. Agents here have guidance for scanning code and hardening pipelines, but none for the application layer: who may read which record, how tenants are kept apart, and how to prove it with a test before the code ships.

    Evidence.

    Where it shows up. The business-site customer portal, KHoist's data product API, the request handlers of the SvelteKit sites, and tenant applications on the shared platform.

    Acceptance scenarios.

    • Should load: "Add an endpoint so a customer can download their invoice" -> access is decided from the signed-in customer's tenant, not from the identifier in the request, and a test shows another tenant gets nothing.
    • Should load: "We are adding company accounts: people see only their own company's projects" -> a short threat model, a tenant-scoping design, and isolation tests for every route that touches tenant data.
    • Should load: "Handle the payment provider's webhook" -> the signature is checked over the raw body, old or repeated deliveries are refused or ignored, and provider specifics are left to the provider's own skill.
    • Should load: "Let users upload attachments" -> size and type limits, storage outside the served path, tenant-scoped names.
    • Near miss: "Harden this workflow file" -> github-actions-hardening. "Set up code scanning" -> codeql. "Is this dependency vulnerable?" -> dependency-scanning. "Onboard a tenant namespace" -> gitops-tenant-onboarding. "Customer data is leaking right now" -> incident-response.
    • Must not load: "Fix the typo on the pricing page".

    Already covered by. github-actions-hardening, codeql, secret-scanning, dependency-scanning and dependabot cover pipelines and scanning. gitops-tenant-onboarding covers tenants at the cluster level. stripe-best-practices covers one payment provider. incident-response covers the response once something has gone wrong. None covers application access rules, tenant boundaries in data, or tests for them.

    Would something else fix it better? Partly. A cross-tenant test in each product's CI is the real control, and each product's own instructions stay the authority for its tenant model. But nothing tells an agent to write that test or what it must prove. That is reusable guidance, so a skill is the right carrier, and the test it asks for is the enforcement.

    Out of scope. Scanning setup (the skills above). Personal-data duties (#638). Consumer and payment disclosures (#639). Each product's actual roles, tenant model and escalation contacts, which stay in that product's own instructions.

    Decision record: build application-security

    Candidates. Reviewed 2026-11-10.

    Candidate Owner and licence Reviewed at Gates Scores Verdict
    security-review Sentry (getsentry/skills), repository Apache-2.0, the skill's references CC BY-SA 4.0 commit e94f2221 Licence: not passable today. The skill's own licence file says its references derive from the OWASP Cheat Sheet Series under share-alike terms, and #581 owns that redistribution review F 1, M 2, C 2, T 1, U 1 Reject for now. It reviews finished code for known vulnerability classes. It has no threat model, no tenant design and no webhook guidance
    security-model Sentry (getsentry/skills), Apache-2.0, adapted from openai/codex-security commit e94f2221 Not taken to the gates: rejected on fit F 0, M 2, C 1, T 1, U 1 Reject. It writes a policy file that tells scanners what to report. It needs uv and bundled Python helpers, and its text names Sentry's own disclosure address and tools
    security-review github/awesome-copilot, MIT commit 227949a7 Not taken to the gates: rejected on fit F 0, M 2, C 2, T 0, U 1 Reject. It is a scanner procedure that includes secret and dependency checks, which this plugin already has. The issue rules out duplicate scanning
    threat-model-analyst github/awesome-copilot, MIT commit 227949a7 Not taken to the gates: rejected on fit and size F 1, M 2, C 2, T 0, U 0 Reject. Threat modelling only, as a multi-agent report generator of 17 files and 335 KB. Far more than a team adding one endpoint will run
    owasp-security agamm/claude-code-owasp, MIT, one maintainer commit 1ae29cb2 Compatibility fails first: its frontmatter uses a when_to_use field the open skill format does not define, so the copy fails this marketplace's validation, and bundled copies are never rewritten. Licence not established: its largest reference restates OWASP material that is published under share-alike terms, under an MIT label F 1, M 1, C 0, T 2, U 1 Reject. The closest in content: it maps findings to the standards and mentions tenants, but it is a review procedure and has no isolation test design
    VibeSec-Skill BehiSecc/VibeSec-Skill, Apache-2.0 commit 0590993b Compatibility fails: the skill's name is not a valid skill name, so it fails validation unmodified F 1, M 1, C 0, T 0, U 2 Reject. One long secure-coding file, unchanged since February 2026
    Trail of Bits collection trailofbits/skills, CC BY-SA 4.0 commit 470419e1 (from the coverage page) Licence: waits on #581, as the coverage page records not scored Reject for now. Audit workflows for security researchers, not design guidance

    What was read. Sentry security-review: the entry file, its licence file and five references (authorization, authentication, API security, business logic, file security), searched for tenant, object-level, webhook, rate-limit, upload and threat-model coverage. Sentry security-model: the entry file and its specification. awesome-copilot: both entry files and the folder sizes. agamm and VibeSec: the entry files, the folder layout, and the same coverage search. Candidates rejected on fit, format or licence were not read line by line for correctness, and I make no correctness claim about them.

    Why build. No candidate covers the acceptance scenarios, and the two closest cannot be bundled today (format, licence). The gap is rated high on the coverage page, and the customer portal is being built now.

    Standards the skill is written against (checked 2026-11-10 in the standards' own repositories): OWASP ASVS 5.0.0 (released 2025-05-30), chapters V2, V4 to V8 and V16, and the OWASP API Security Top 10 2023. The skill cites them by chapter and item and restates none of their text.

    Known defects. None yet: the skill is being written. Every example in it will be run before the pull request is offered for review.

    Description cost. Reported on the pull request, with the target of 400 characters or fewer.

    Acceptance scenarios. Results are recorded on the pull request after the skill is installed from its head.

    What would change this decision. #581 clearing share-alike redistribution would make Sentry's security-review a candidate for the review-time half, beside this skill. A maintained upstream skill that covers tenant isolation design and its tests, in the open format, would replace this one.

    🤖 Generated with Claude Code

  5. moved this from 📥 Backlog to 🏃🏻‍♂️ In Progress in 🌊 Project Boardon Nov 10, 2026
  6. moved this from 🏃🏻‍♂️ In Progress to ✅ Done in 🌊 Project Boardon Nov 11, 2026
  7. devantler commented on Nov 11, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer (interactive Claude Code session)

    Done in #682, released as agentic-engineering 7.5.0. This note completes the decision record above.

    Acceptance criteria.

    • Existing coverage was audited and seven upstream skills were compared before building; the comparison is on the coverage page and in the record above.
    • The skill has a filled threat model and runnable tests that show cross-tenant access being refused, in Go and TypeScript. Removing the tenant check makes them fail.
    • Its description names endpoints, forms, sign-in, roles, tenant data, webhooks and uploads. A fresh agent chose it from the description for an invoice endpoint, company accounts, a payment webhook and uploads.
    • It sends scanning, pipeline hardening, cluster tenancy and incident response to the skills that own them, and repeats none of their instructions.

    Acceptance scenarios, as tried. A fresh agent followed the installed skill for an invoice read and delete route and for a webhook receiver, in three passes. The first two found real defects, among them a webhook example that lost an event when handling failed; all were fixed and confirmed in the third pass. The review then raised 17 more points, also fixed. The trial notes on #682 list what was run.

    Description cost. 398 characters.

    Known limits. The review fixes were verified by running the examples and removing each protection once, not by another fresh-agent pass. Whether agents load the skill in daily work is not measured yet; that is #651. Each product still has to write its own isolation tests.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    None yet

    Start date

    None yet

    Target date

    None yet

    Effort

    None yet

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions