Skip to content

Improve template JSON schema for editor validation - #7579

Open
Mzack9999 wants to merge 2 commits into
devfrom
4950-jsonschema
Open

Improve template JSON schema for editor validation#7579
Mzack9999 wants to merge 2 commits into
devfrom
4950-jsonschema

Conversation

@Mzack9999

@Mzack9999 Mzack9999 commented Jul 24, 2026

Copy link
Copy Markdown
Member

Summary

  • Revamp nuclei-jsonschema.json generation with richer field docs, examples, deprecations, and anyOf required combinations for Template/Info/HTTP and all remaining protocols (DNS, TCP, File, Headless, SSL, Websocket, Whois, Code, Javascript) plus workflows/matchers/extractors
  • Fix schema $id and emit markdownDescription for Monaco/VS Code autocomplete; add make genschema

Closes #4950

Summary by CodeRabbit

  • New Features
    • Expanded JSON Schema coverage across templates, workflows, protocols, matchers, extractors, and request types, including richer field documentation with examples, defaults, and deprecation notes.
    • Added/strengthened validation for required-field combinations and supported configuration shapes.
  • Developer Experience
    • Added a dedicated schema regeneration command to rebuild the JSON Schema output only.
    • Improved schema output for more consistent identifiers and markdown-friendly descriptions.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 769ad874-be63-4725-9824-6e81ac480b63

📥 Commits

Reviewing files that changed from the base of the PR and between 1364270 and 0a0a9a1.

⛔ Files ignored due to path filters (1)
  • nuclei-jsonschema.json is excluded by !**/*.json
📒 Files selected for processing (22)
  • Makefile
  • cmd/docgen/docgen.go
  • pkg/model/model.go
  • pkg/model/model_schema.go
  • pkg/operators/extractors/extractors_schema.go
  • pkg/operators/matchers/matchers_schema.go
  • pkg/protocols/code/code_schema.go
  • pkg/protocols/dns/dns_schema.go
  • pkg/protocols/file/file_schema.go
  • pkg/protocols/headless/headless_schema.go
  • pkg/protocols/http/http.go
  • pkg/protocols/http/http_method_types.go
  • pkg/protocols/http/http_schema.go
  • pkg/protocols/javascript/js_schema.go
  • pkg/protocols/network/network_schema.go
  • pkg/protocols/ssl/ssl_schema.go
  • pkg/protocols/websocket/websocket_schema.go
  • pkg/protocols/whois/whois_schema.go
  • pkg/templates/templates.go
  • pkg/templates/templates_schema.go
  • pkg/utils/schema/helpers.go
  • pkg/workflows/workflows_schema.go
🚧 Files skipped from review as they are similar to previous changes (22)
  • pkg/model/model_schema.go
  • pkg/protocols/network/network_schema.go
  • pkg/protocols/ssl/ssl_schema.go
  • pkg/protocols/dns/dns_schema.go
  • pkg/protocols/javascript/js_schema.go
  • pkg/protocols/file/file_schema.go
  • pkg/protocols/headless/headless_schema.go
  • pkg/protocols/whois/whois_schema.go
  • pkg/protocols/http/http_method_types.go
  • pkg/workflows/workflows_schema.go
  • pkg/operators/extractors/extractors_schema.go
  • pkg/protocols/websocket/websocket_schema.go
  • pkg/templates/templates_schema.go
  • Makefile
  • pkg/protocols/code/code_schema.go
  • pkg/model/model.go
  • pkg/operators/matchers/matchers_schema.go
  • cmd/docgen/docgen.go
  • pkg/protocols/http/http_schema.go
  • pkg/utils/schema/helpers.go
  • pkg/protocols/http/http.go
  • pkg/templates/templates.go

Walkthrough

The change adds shared JSON Schema composition helpers and metadata for templates, Info sections, workflows, operators, and protocol requests. It updates schema serialization with stable identifiers and description conversion, and adds a Makefile target for JSON-schema-only regeneration.

Changes

JSON Schema generation

Layer / File(s) Summary
Shared schema composition helpers
pkg/utils/schema/helpers.go
Adds reusable property metadata, examples, descriptions, schema-extension, and AnyOf-required helpers.
Template, Info, and workflow schemas
pkg/model/*, pkg/templates/*, pkg/workflows/*
Adds schema metadata, examples, required combinations, and extension methods for template-related entities.
HTTP request schema
pkg/protocols/http/*
Defines HTTP request metadata and required combinations, integrates them into schema extension, and exposes HTTP method examples.
Protocol and operator schemas
pkg/operators/..., pkg/protocols/{code,dns,file,headless,javascript,network,ssl,websocket,whois}/*
Adds JSON Schema metadata and extension methods for operator and protocol request types.
Schema serialization and generation wiring
cmd/docgen/docgen.go, Makefile
Adds stable schema identifiers, recursive description conversion, formatted output, and a genschema target that generates only nuclei-jsonschema.json.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Makefile
  participant docgen
  participant jsonschemaReflector
  Makefile->>docgen: build and run genschema
  docgen->>jsonschemaReflector: reflect schema with stable BaseSchemaID
  jsonschemaReflector-->>docgen: encoded schema
  docgen->>docgen: update $id and description keys
  docgen-->>Makefile: write nuclei-jsonschema.json
Loading

Poem

I’m a rabbit with schemas tucked neat,
Metadata hops in a repeat.
Required fields dance, examples glow,
Stable IDs help editors know.
genschema thumps—JSON’s complete!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main change: improving template JSON schema for editor validation.
Linked Issues check ✅ Passed The changes add richer descriptions, examples, required combinations, and deprecated-field metadata across the schema as requested.
Out of Scope Changes check ✅ Passed The modified files all support schema generation and validation improvements and align with the issue scope.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 4950-jsonschema

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/docgen/docgen.go`:
- Around line 89-91: Update the description handling in the docgen map
transformation so it retains the original "description" entry while also adding
the "markdownDescription" Monaco extension; remove the deletion from the branch
guarded by parent != "properties", preserving the existing conversion scope.

In `@pkg/protocols/http/http_schema.go`:
- Line 383: Correct the spelling of “concurreny” to “concurrency” in the Threads
description string, leaving the rest of the text unchanged.

In `@pkg/protocols/network/network_schema.go`:
- Around line 92-95: Update networkRequestAnyOfRequired to use a single
schema.RequiredCombos entry that requires both “host” and “inputs” together,
replacing the separate schema.Require entries so requests containing only one
field no longer validate.

In `@pkg/templates/templates_schema.go`:
- Line 428: Update the workflow template schema requirement near
schema.Require("workflows") to use the same base requirement as protocol
templates, enforcing the documented id + info + workflows contract while
retaining workflows as required.

In `@pkg/utils/schema/helpers.go`:
- Around line 53-65: Update ExtendSchema’s oneOf construction to preserve array
item schemas for referenced array properties such as info.author and info.tags.
When building each branch, carry over the original array Items constraint (or
expose it through PropertyMetadata) so string-array branches still reject
non-string elements, while retaining the existing parent Ref and Type clearing
behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a752a221-e54a-4576-87f6-6bf3c9c0e316

📥 Commits

Reviewing files that changed from the base of the PR and between bcf2089 and 1364270.

⛔ Files ignored due to path filters (1)
  • nuclei-jsonschema.json is excluded by !**/*.json
📒 Files selected for processing (22)
  • Makefile
  • cmd/docgen/docgen.go
  • pkg/model/model.go
  • pkg/model/model_schema.go
  • pkg/operators/extractors/extractors_schema.go
  • pkg/operators/matchers/matchers_schema.go
  • pkg/protocols/code/code_schema.go
  • pkg/protocols/dns/dns_schema.go
  • pkg/protocols/file/file_schema.go
  • pkg/protocols/headless/headless_schema.go
  • pkg/protocols/http/http.go
  • pkg/protocols/http/http_method_types.go
  • pkg/protocols/http/http_schema.go
  • pkg/protocols/javascript/js_schema.go
  • pkg/protocols/network/network_schema.go
  • pkg/protocols/ssl/ssl_schema.go
  • pkg/protocols/websocket/websocket_schema.go
  • pkg/protocols/whois/whois_schema.go
  • pkg/templates/templates.go
  • pkg/templates/templates_schema.go
  • pkg/utils/schema/helpers.go
  • pkg/workflows/workflows_schema.go

Comment thread cmd/docgen/docgen.go
Comment thread pkg/protocols/http/http_schema.go Outdated
Comment on lines +92 to +95
var networkRequestAnyOfRequired = []schema.RequiredCombos{
schema.Require("host"),
schema.Require("inputs"),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Require host and inputs together.

Lines 92-95 emit anyOf: [{required:["host"]}, {required:["inputs"]}], so a request with only one field validates. Use one combination requiring both fields.

Proposed fix
 var networkRequestAnyOfRequired = []schema.RequiredCombos{
-	schema.Require("host"),
-	schema.Require("inputs"),
+	schema.Require("host", "inputs"),
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
var networkRequestAnyOfRequired = []schema.RequiredCombos{
schema.Require("host"),
schema.Require("inputs"),
}
var networkRequestAnyOfRequired = []schema.RequiredCombos{
schema.Require("host", "inputs"),
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/protocols/network/network_schema.go` around lines 92 - 95, Update
networkRequestAnyOfRequired to use a single schema.RequiredCombos entry that
requires both “host” and “inputs” together, replacing the separate
schema.Require entries so requests containing only one field no longer validate.

Comment thread pkg/templates/templates_schema.go Outdated
Comment thread pkg/utils/schema/helpers.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Nuclei YAML template JsonSchema

1 participant