Skip to content

fix(bots): close stack regressions and the review findings - #192

Merged
haydenshively merged 3 commits into
fix/normalize-position-join-keyfrom
fix/stack-fixes
Sep 1, 2026
Merged

fix(bots): close stack regressions and the review findings#192
haydenshively merged 3 commits into
fix/normalize-position-join-keyfrom
fix/stack-fixes

Conversation

@haydenshively

@haydenshively haydenshively commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #189. Fixes, not features. Two passes live here:

  1. A regression sweep over feat(midnight-liquidation): support loan-as-collateral markets #184/feat(swaps): rank candidates on an interpolated venue cost curve #187/fix(bots): treat a reverted send as economic, not a send failure #188/fix(bots): normalize the position join key across log events #189, asking one narrow question — "did we break what the bots already did?" — which found four answers.
  2. The triage of the 20 automated review findings those PRs drew once they went ready. Verified one by one; nine fixed, four declined with reasoning on the threads.

Filed as its own PR so the mistakes and their fixes are legible together rather than rewritten into the PRs that introduced them.


Part 1 — the regression sweep

1. Detection-only blue could broadcast liquidations 🔴

main stack before this PR
gate order if (venues.length === 0) return { kind: 'no_config' } first unwrap resolution first, swap-free branch, then the gate
swap-free guard resolution.steps.length > 0 && isAddressEqual(…) isAddressEqual(…) — guard dropped

swapFreePlan returns kind: 'swap', and blue's tick takes that straight to simulatesubmit. So with zero venues two paths that were unreachable now reached the broadcaster.

Fix: swapFreeWithoutVenues on composeMultiVenueQuoting, default false — a caller with no swap-free path refuses immediately. Midnight opts in. Default-deny lives in the package deliberately, so a future third caller inherits the safe posture rather than repeating this.

2. …and armed backoff on a transient read failure

Same reorder: an unwrapper throw became kind: 'failed' instead of no_config, and blue's tick arms per-position exponential backoff on failed. Refusing before the unwrap chain fixes this and #1 together, and spends no reads doing it.

3. A sibling's execution revert cancelled a real backoff

backoffExempt is keyed by position, so an execution revert on candidate B suppressed the backoff candidate A's sendRejected (nonce / funds / RPC) had armed. New backoffForced set overrides the exemption: an economic verdict on one candidate's plan refutes nothing about broken machinery.

4. Phase A.5 spent RPC on suppressed positions

On main both suppression gates preceded quoteFor, so a suppressed position cost zero I/O. prepareRoutes now takes a suppressed predicate, and isCooled/isBackedOff are defined once so the two phases cannot drift. Suppressed candidates map to unknown, not no_route — claiming they need no route would price them at zero and let them outrank a costed sibling.


Part 2 — the review triage

Fixed

5. The venue-less swap-free exception was still too broad (caught on this PR — a hole in fix #1). swapFreePlan serves two shapes, and the other one — an unwrap chain that merely lands on the loan token, PT-USDC in a USDC market — still moves assets. With Midnight wiring both an ERC-4626 and a Pendle unwrapper on Base, an ALLOW_BAD_DEBT_ONLY deployment could broadcast an ordinary liquidation. The exception is now zero-step only; venue-enabled callers keep both shapes.

6. Candidate ranking trusted curves that quoting refuses. costRoutes checked clamped but not ageMs and not completeness, while estimatedOut is consumed there as an absolute level against a current oracle — the one term staleness decays. A stale or partial curve therefore drove the position_cap and fall_through_bound cutoffs. Both consumers now share one exported curveIsTrusted(estimates, venues), so the divergence is gone structurally rather than patched twice. (MAX_PREDICTION_AGE_MS is consolidated into MAX_COST_LEVEL_AGE_MS, which is the same question with the same answer.)

7. Pendle PT routes were resolved twice per tick — both rate-limited hosted API calls, the first discarded. New optional Unwrapper.previewTokenOut returns the post-unwrap token without building calldata: Pendle answers from its TTL-cached markets list with zero HTTP, ERC-4626 from its memoized asset(). Phase A.5 only ever needed the pair. Resolution itself stays uncached, since the calldata is amount-bound. If any unwrapper lacks the seam the preview returns null and the caller falls back to a full resolve, so a partial walk can never report the wrong pair.

8. Phase A.5 resolved routes serially, in discovery order, with the whole phase awaited before ranking — so one slow route delayed every later candidate. Now partitioned synchronously and fanned out through Promise.all.

9. A retry-worthy sibling no longer loses to a suppressed one. Every suppression set is keyed by position while candidates are per (slot, mode), so a sibling's no_route suppressed a position whose other candidate ended floor_unmet — the one outcome meant to retry every block as the LIF ramps. New retryWorthy set, with backoffForced still overriding.

10. The cooldown verdict is snapshotted for the tick. Unlike block-keyed backoff, CooldownStore.shouldSkip is wall-clock and could flip between phases, leaving a candidate quoted with a route never warmed — which, because scoreNetOfRouteCost fails open per position, drops the whole alternative set to gross ordering.

11. Revert-streak state expires between episodes. Labels are market:borrower and outlive a borrow, so a stale startedAt made the first revert of a new episode report a false 15-minute crossing — or, if the old entry had escalated, yield ongoing forever and never warn. New REVERT_STREAK_EPISODE_GAP_MS, deliberately separate from the escalation threshold: silence and streak length are unrelated quantities.

12. tx.submit_failed carries a candidate discriminator. Optional SubmitArgs.correlation, spread beside id. Additive and inert for the other four callers; stops short of PendingEntry, since a send that fails here never reaches pending.

13. Docs and conventions. The swap-free "iff" guarantee (passesRouteQuality is a floor, not a band); the selectorConstant overclaim (all Error(string) share one selector); two stale README rows (ALLOW_BAD_DEBT_ONLY, and mode fall-through); four new test helpers to arrow constants plus a JSDoc an inserted const had orphaned.

Declined, with reasoning on the threads


Verification

23 new tests, each proven against the mutation that should break it — the source reverted to this branch's parent while the new tests stood, and every non-control assertion failed.

  • Typecheck clean across swaps / bot-kit / both liquidators; pnpm lint 0 warnings 0 errors; knip clean.
  • pnpm test: 2895 passed. The 4 fork/e2e suites fail locally on a missing RPC_URL_8453 only — identical set before and after; CI holds the secret and runs them.
  • One test the sweep flagged as weakened is restored: expect(unwrapper.probed).toHaveLength(0) had become toBeGreaterThan(0), i.e. updated to match the new behavior rather than catching it. It is now a pair — one per posture.
  • TIB-2026-08-28-midnight-loan-as-collateral.md is amended twice: the pre-gate ordering is opt-in, and the exception is zero-step only.

Deliberately not changed

With venues enabled, a collateral === loan market now yields a swap-free plan where main fell through to a doomed aggregator quote. That is main's unwrapOnlyPlan extended to the zero-step case, it is strictly better, and blue's sweepCalls already handles the duplicate-skim. Flagging rather than reverting.

🤖 Generated with Claude Code

@haydenshively haydenshively self-assigned this Sep 1, 2026
@haydenshively
haydenshively marked this pull request as ready for review September 1, 2026 03:51
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T04:01:54.356717Z 93b3597 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

haydenshively and others added 2 commits August 31, 2026 22:52
A regression sweep over #184/#187/#188/#189 found four behaviors the stack
changed without meaning to. None are new functionality; each restores what
the bots did on main.

Detection-only blue could broadcast. The no-venues gate moved to AFTER unwrap
resolution and the swap-free branch lost its `steps.length > 0` guard, so a
venue-less deployment returned `kind: 'swap'` — which the tick takes straight
to simulate+submit. docker-compose defaults Robinhood (4663) to
ALLOW_DETECTION_ONLY with a funded key, documented as skipping every routed
liquidation. `swapFreeWithoutVenues` (default false) restores the immediate
refusal; midnight opts in, because its loan-as-collateral slots need no route
and ALLOW_BAD_DEBT_ONLY is a supported posture there.

That same reorder downgraded a transient unwrapper RPC failure from
`no_config` (skip) to `failed` (arms backoff), pushing a deliberately unarmed
deployment into a suppression state machine it never entered. Refusing before
the unwrap chain fixes both at once, and spends no reads doing it.

A send REJECTION now arms backoff even when a sibling execution-reverted.
Both sets are keyed by position, so `backoffExempt` was cancelling the backoff
a broken nonce/funds/RPC send earned; the position then re-sent every block
while the send machinery was still broken.

Phase A.5 no longer resolves routes for suppressed positions. It ran ahead of
the cooldown/backoff gates, so a backed-off position spent an uncached read
per candidate per tick — breaking backoff's contract that it bounds API and
RPC usage under a backlog, and putting that latency in front of the first
send of a maturity burst.

Each of the 8 new tests was verified to fail against the pre-fix source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The midnight opt-in was untested: deleting `swapFreeWithoutVenues: true` broke
no test, because the existing no-venues case uses a collateral that is not the
loan token. Adds the loan-as-collateral case that pins it, verified to fail
when the flag is removed.

TIB-2026-08-28 asserted the swap-free short-circuit runs before the no-venues
gate as an unconditional package property. It is now opt-in, and the record
says so — including why, since that ordering is what let a venue-less blue
return a broadcastable plan.

Also: `Address` over an inline hex template in the blue fixture; one home for
the `swapFreeWithoutVenues` rationale instead of four; `isCooled`/`isBackedOff`
defined once rather than open-coded in both phase A.5 and phase B (their only
guarantee of agreement was a comment, which referenced a symbol not in scope);
`probe.warmed` counts candidates that actually resolved, so its ratio no longer
drifts under a backlog; README's probe-gating claim narrowed to match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

Comment thread bots/midnight-liquidation/src/quotes.ts
@haydenshively
haydenshively marked this pull request as draft September 1, 2026 03:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae3a024d6a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bots/midnight-liquidation/src/runner/tick.ts
@haydenshively
haydenshively marked this pull request as ready for review September 1, 2026 03:59
Triaged all 20 Codex/Devin findings across #184, #187, #188, #189 and #192.

Fixed:
- the venue-less swap-free exception admitted unwrap-only plans, so an
  ALLOW_BAD_DEBT_ONLY deployment could broadcast an asset-moving liquidation
- candidate ranking trusted stale and incomplete curves that quoting refuses;
  both now share one exported `curveIsTrusted`
- a Pendle PT resolved its unwrap chain twice per tick, both hosted API calls;
  new `previewTokenOut` seam answers phase A.5 from cache
- phase A.5 resolved routes serially, in discovery order
- a sibling's `no_route` suppressed a position whose other candidate was
  `floor_unmet`, which is meant to retry every block as the LIF ramps
- the wall-clock cooldown verdict could flip mid-tick, leaving a candidate
  quoted but unpriced
- revert-streak state never expired, so a reused label reported false crossings
- `tx.submit_failed` carried no candidate discriminator
- docs: the swap-free "iff" guarantee, `selectorConstant`, two README rows

Declined, with reasoning for the threads: narrowing the backoff exemption to
post-maturity plans (a regression — the sets are per-position while the mode is
per-candidate), `stopAfterWinner`, and the sub-1e-18 rate truncation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@haydenshively haydenshively changed the title fix(bots): close regressions found reviewing the stack fix(bots): close stack regressions and the review findings Sep 1, 2026
@haydenshively
haydenshively merged commit 7d07f26 into main Sep 1, 2026
7 checks passed
@haydenshively
haydenshively deleted the fix/stack-fixes branch September 1, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants