feat(skill): add the external-integration skill to the bundled pack - #528
Merged
Conversation
Calling a service someone else runs, from fetching the current documentation for the version you are on to deciding whether repeating a failed request is safe. The measured basis for the first half: over eight Python libraries and eleven models, code generated against an API that changed after training ran in the target environment 43% of the time, and 66% with the current page in the prompt, so a citation is the largest single improvement available and still leaves a third wrong. The finishing move is one recorded response. The second half is the part no published skill covers: translating payloads at the adapter so an additive change upstream is not an outage, classifying every failure with "do not retry" as the default, treating retry safety as a property of the request rather than of the error, bounding retries to one layer and a budget, stating what callers get while the service is down, and naming the hosts and credential the integration may use. Vendor specifics stay out by rule, with the reason written into the body: providers now publish and maintain their own instructions for agents, so a section here about a named API would be a changelog we sign up to maintain. references/failures.md carries the per-transport catalogue: what each failure looks like on the wire, which class it maps to, and what is known about the write afterwards. Retrieval rows cover the seven objectives it must be offered for, and four rows now assert it is not offered for a neighbour's objective. Signed-off-by: Ion Alpha <contact@ionalpha.io>
…his skill's rows The domain-language rows landed on main while this branch was open. Reset the table to main's copy so the merge is clean, then re-add this skill's rows on top of it in the next commit. Signed-off-by: Ion Alpha <contact@ionalpha.io>
…in's table Seven rows for the objectives this skill must be offered for, and five rows that assert it is not offered for a neighbour's objective. Two descriptions moved to keep the ranking honest now that the pack holds eight skills and offers five. This skill's opening clause no longer says "calls something", which was scoring on "make this thing easier to call" and displacing the skill that objective belongs to. interface-depth's description now says that making something easier to call is what depth means, which is its own subject stated in the words a person types, and it wins that row back on the match rather than on the tie-break. Signed-off-by: Ion Alpha <contact@ionalpha.io>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
external-integrationto the bundled pack: the craft of calling a service someone else runs, from fetching their current documentation for the version you are actually on through to deciding whether repeating a failed request is safe. It ships withreferences/failures.md, a per-transport catalogue of what each failure looks like on the wire, which class it maps to, and what is known about the write afterwards.Seven retrieval rows state the objectives it must be offered for, and five existing rows now assert it is not offered for a neighbour's objective, so the scope split against
structural-boundaries,interface-depth,contract-designandsystematic-debuggingis checked on every build rather than asserted in a review.Why
The pack could tell you where a change goes, how deep a module should be, what a contract may promise, what to call things, and whether to take a dependency, but not what to do at the one boundary where the code on the other side is not ours.
The first half of the skill has a number behind it. Over eight Python libraries and eleven models, code generated for an API that changed after the models were trained ran in the target environment 43% of the time; with the current documentation page in the prompt, 66%. Fetching the page is the largest single improvement available and it still leaves a third of the code broken, and one in six of the calls that stayed wrong was the call that page marked deprecated. So the skill's spine is that a citation is not a verification: the integration is settled by one real response, recorded.
The second half is the part the published work stops short of. Translation at the adapter so an additive change upstream is not an outage, a class on every failure with "do not retry" as the default, retry safety treated as a property of the request rather than of the error, retries bounded to one layer and a budget, a stated behaviour for the service being down, and a named set of hosts and a scoped credential. All three of the mechanisms it leans on already execute here:
fault.Classifytreats an unclassified error as terminal, the shared transport retries 408, 429 and 5xx with cappedRetry-Afterand refuses to replay a body it cannot rewind, andnetguardis default-deny at connect.Vendor specifics stay out by rule, and the body carries the reason rather than the assertion: providers now publish and maintain their own agent instructions alongside their documentation, so a section here about a named API would be a changelog we sign up to maintain and lose.
How to verify
go test ./skill/...TestPackIsRetrievableruns the real ranker over the real pack against every row inretrieval.txt, including the five new must-not rows.TestEveryPackSkillStatesItsTriggersfails a skill no row expects, andTestPackProseIsAuthoredreads both new files.Reading it is the other half:
skill/bundled/skills/external-integration/SKILL.mdat 224 lines, andreferences/failures.mdat 83.Notes for reviewers
One neighbour's description moved, and the reason is worth a look. The pack now holds eight skills and offers five, so a tie on match score is broken alphabetically, and this skill's original opening clause ("calls something you do not run") tied with
interface-depthon "make this thing easier to call" and took its place. Two edits fix it on the match rather than on the sort: this description now opens with "depends on a system you do not run", andinterface-depthnow says that making something easier to call is what depth means, which is its own subject in the words a person actually types.The rest of the description is tuned rather than freely written, since it is the only text discovery loads. Five must-not rows hold; rows about versioning and pinning do not, because "the version you are actually on" is genuinely part of this subject, and forcing those would have cost the description more than the check is worth.
Authored from a research pass over the published skills read from their own repositories, the API-evolution and agent-hallucination measurements, and the operator literature on retries. No six-word sequence in either new file is shared with any published skill on the subject or with the rest of our own pack, checked mechanically before commit.