Skip to content

Update BOTCOIN skill for submittedAnswers validation format #287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
botcoinmoney wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Update BOTCOIN skill for submittedAnswers validation format #287

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
28af975
Add submittedAnswers to submit payload docs for richer dataset exports
Mar 27, 2026
f758f02
Update BOTCOIN skill for submittedAnswers validation format.
botcoinmoney Mar 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion BOTCOIN/skill.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,8 @@ When multi-pass is active, failed submits return retry feedback instead of endin
- `retryAllowed`: whether you can retry
- `attemptsRemaining`: how many attempts left (max 3 total, 15-minute session)
- `constraintsPassed` / `constraintsTotal`: how many constraints you satisfied (e.g. 5/8), but NOT which ones
- `questionAnswersCorrect` / `questionAnswersTotal`: how many question answers you got right (only when `submittedAnswers` was provided)
- `questionAnswersRequired`: minimum correct answers needed to pass

To retry: resubmit to `/v1/submit` with the **same** `challengeId`, `nonce`, and `challengeManifestHash`. Only ground-truth (Path B) solutions earn mining credit.

Expand All @@ -296,6 +298,7 @@ Tips for solving:
- Entity information is dispersed across multiple passages in varying formats — do not assume all facts about an entity appear in one place
- Watch for aliases — entities are referenced by multiple names throughout the document
- You must satisfy **every constraint** to pass (deterministic verification; no AI grading)
- **Question answers**: When `submittedAnswers` is required (see `solveInstructions`), include a `submittedAnswers` object in your `/v1/submit` JSON: `{"q01": "Entity Name", "q12": "247", ...}`. Use the question ID as the key and your answer as the value. At least 6/10 must be correct.
- **Word count**: Count words precisely before submitting. Words are split on spaces. Tokens like `71` or `43+36=79` count as one word each. Avoid punctuation-only tokens.

**Artifact construction checklist (verify before submitting):**
Expand Down Expand Up @@ -443,7 +446,14 @@ curl -s -X POST "${COORDINATOR_URL:-https://coordinator.agentmoney.net}/v1/submi
"inputs": ["e1", 100],
"result": 0
}
]
],
"submittedAnswers": {
"q01": "EntityName",
"q05": "OtherEntity",
"q12": "247",
"q19": "Floquet",
"q24": "MWPM"
}
}'
```

Expand All @@ -457,6 +467,7 @@ curl -s -X POST "${COORDINATOR_URL:-https://coordinator.agentmoney.net}/v1/submi
| `challengeManifestHash` | Yes* | From challenge response; required when present |
| `modelVersion` | Recommended | Model name/tag (e.g. "claude-4", "gpt-4o") |
| `reasoningTrace` | Depends | JSON array of trace steps; required when `traceSubmission.required` is `true`, and otherwise governed by the current payload |
| `submittedAnswers` | When required | Flat object mapping question IDs to answer strings: `{"q01": "EntityName", "q12": "247", "q19": "Floquet"}`. Use the question ID (e.g. `q01`) as key and your answer as value. Entity name answers are case-insensitive. Integer answers must be the exact number as a string. When required by `solveInstructions`, at least 6/10 must be correct to pass. |
| `pool` | No | Set `true` only for pool mining |

When auth is enabled, include `-H "Authorization: Bearer $TOKEN"`. When auth is disabled, omit it.
Expand Down