Skip to content

fix(bots): close stack regressions and the review findings #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
haydenshively merged 3 commits into from
Sep 1, 2026
Merged

fix(bots): close stack regressions and the review findings #192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b760271
fix(bots): close regressions found reviewing the stack
haydenshively Sep 1, 2026
93b3597
address review of the stack fixes
haydenshively Sep 1, 2026
5283c74
address the automated review of the stack
haydenshively Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 42 additions & 2 deletions bots/blue-liquidation/test/quotes.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { Logger } from '@repo/bot-kit'
import type { RateLimitedClient, Venue, VenuePair, VenueSelector } from '@repo/swaps'
import type { RateLimitedClient, Unwrapper, Venue, VenuePair, VenueSelector } from '@repo/swaps'
import type { Address } from 'viem'

import { getAddress } from 'viem'
import { describe, expect, it } from 'vitest'
Expand Down Expand Up @@ -89,13 +90,39 @@ function fakeSelector(
return { selector, refreshed }
}

// An unwrapper converting the collateral straight to the loan token, recording what it was asked
// about — so a test can assert the read was never spent.
const unwrapsToLoan = (): Unwrapper & { probed: Address[] } => {
const probed: Address[] = []
return {
kind: 'fake-erc4626',
probed,
async resolve({ token }) {
probed.push(token)
return {
step: {
tokenIn: COLLATERAL,
tokenOut: LOAN,
target: TARGET,
value: 0n,
callData: '0x12345678',
amountIn: { source: 'balance', offset: 4n }
},
expectedAmountOut: 1000n,
amountOutMinimum: 1000n
}
}
}
}

function compose(
selector: VenueSelector,
overrides: {
venues?: ('0x' | '1inch')[]
excludeCollaterals?: `0x${string}`[]
logger?: Logger
httpClient?: RateLimitedClient
unwrappers?: readonly Unwrapper[]
} = {}
) {
return composeQuoting({
Expand All @@ -106,13 +133,26 @@ function compose(
venues: overrides.venues ?? ['0x'],
baseUrls: {},
maxRouteImpactBps: 500,
unwrappers: [],
unwrappers: overrides.unwrappers ?? [],
excludeCollaterals: overrides.excludeCollaterals ?? [],
logger: overrides.logger ?? NOOP_LOGGER
})
}

describe('composeQuoting (Blue lens-projection adapter)', () => {
it('passes no swap-free opt-in, so zero venues never yields a broadcastable plan', async () => {
// `kind: 'swap'` goes straight to simulate+submit in the tick, and Blue supports running unarmed
// (ALLOW_DETECTION_ONLY). It has no swap-free liquidation path, so it leaves the package's
// `swapFreeWithoutVenues` off — even a collateral that unwraps straight to the loan token.
const unwrapper = unwrapsToLoan()
const { selector, refreshed } = fakeSelector(['0x'])
const { quoteFor } = compose(selector, { venues: [], unwrappers: [unwrapper] })
expect(await quoteFor(PLAN, OUT, LABEL)).toEqual({ kind: 'no_config', firmCalls: 0 })
// The refusal precedes unwrap resolution, so it also spends no detection read.
expect(unwrapper.probed).toHaveLength(0)
expect(refreshed).toHaveLength(0)
})

it('returns no_config (and never probes) for an excluded collateral', async () => {
const { selector, refreshed } = fakeSelector(['0x'])
const { quoteFor } = compose(selector, { excludeCollaterals: [COLLATERAL] })
Expand Down
8 changes: 5 additions & 3 deletions bots/midnight-liquidation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Environment variables:
| `ONEINCH_API_KEY` | cond. | — | Enables the `1inch` venue when set. Read at point of use; never stored on config or logged. |
| `ENABLE_LIFI` | no | `false` | Enables the keyless `lifi` venue. Also implicitly enabled when `LIFI_API_KEY` is set. |
| `LIFI_API_KEY` | no | — | Optional; LiFi routes keyless, a key only raises its rate limits (and enables the venue). Read at point of use; never logged. |
| `ALLOW_BAD_DEBT_ONLY` | no | `false` | When no venue is enabled, the bot refuses to start unless this is `true` (then it runs bad-debt-only: discovers positions, realizes bad debt, never swap-liquidates). |
| `ALLOW_BAD_DEBT_ONLY` | no | `false` | When no venue is enabled, the bot refuses to start unless this is `true` (it then discovers positions, realizes bad debt, and liquidates loan-as-collateral slots, which need no route; it never swap-liquidates, and an unwrap chain counts as a swap here). |
| `ZEROX_BASE_URL` / `ONEINCH_BASE_URL` / `LIFI_BASE_URL` | no | public | Optional venue API host overrides. |
| `EXCLUDE_COLLATERALS` | no | — | Comma-separated collateral addresses the bot must never seize/hold — skipped (no quote) even in a listed market. |
| `MAX_FEE_GWEI` | no | `300` | Hard max fee cap used by the pending transaction queue. |
Expand Down Expand Up @@ -164,7 +164,8 @@ There is no swap config file. Instead:
collateral→loan pair, a background job requests
indicative quotes from every enabled venue at the `PROBE_LADDER` sizes and caches each venue's rate
curve, which is then interpolated at the actual seize size. This probe is **gated** to pairs that
have a liquidatable position and cached for `PROBE_STALE_MS`, and runs on its own `PROBE_HTTP_RPS`
have a liquidatable position that is neither cooled-down nor backed off, is cached for
`PROBE_STALE_MS`, and runs on its own `PROBE_HTTP_RPS`
budget — so venues' tight rate limits (~1 req/sec) are respected and quiet markets cost nothing. It
is started, never awaited: a pair is warmed for the next tick rather than delaying this one.
- **When a position is liquidatable**, the bot firm-quotes once against the pre-chosen best venue and
Expand Down Expand Up @@ -439,7 +440,8 @@ valid && gateAllows && hasDebt && !locked && (block.timestamp > maturity || !hea
- Post-maturity healthy positions use post-maturity mode, where LIF ramps from `1e18` to `maxLif` over
60 minutes and the RCF cap is disabled.
- Post-maturity **unhealthy** positions open both on-chain gates, so the bot builds both candidate
plans and keeps the one with the higher expected surplus. Normal mode pays the full `maxLif`
plans and **retains both**, ranked best-first and attempted in order — a mode that fails falls
through to the other in the same tick rather than forfeiting the position. Normal mode pays the full `maxLif`
immediately while the post-maturity LIF is still ramping, so `plan.built { postMaturityMode: false }`
on a matured position shortly after maturity is expected behavior, not a mode-selection bug; once the
ramp completes, ties resolve to post-maturity (its gate cannot close if the price recovers).
Expand Down
10 changes: 9 additions & 1 deletion bots/midnight-liquidation/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -369,6 +369,13 @@ async function main() {
data: encodeExec(market, borrower, plan, swapPlan)
},
label,
// One position submits several `(slot, mode)` alternatives under one label, so without this
// their `tx.submit_failed` rows are indistinguishable — the same discriminator every other
// stage already emits.
correlation: {
collateralIndex: plan.collateralIndex,
postMaturityMode: plan.postMaturityMode
},
maxFeePerGas: fees.maxFeePerGas,
maxPriorityFeePerGas: fees.maxPriorityFeePerGas,
blockNumber
Expand All @@ -385,7 +392,8 @@ async function main() {
routing: {
resolveRoute,
warmRoute: venueSelector.refresh,
routeCost: venueSelector.select
routeCost: venueSelector.select,
venues
},
logger
})
Expand Down
39 changes: 25 additions & 14 deletions bots/midnight-liquidation/src/quotes.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import type {
} from '@repo/swaps'
import type { Address } from 'viem'

import { composeMultiVenueQuoting, resolveUnwraps } from '@repo/swaps'
import { composeMultiVenueQuoting, previewUnwrapChain, resolveUnwraps } from '@repo/swaps'
import { isAddressEqual } from 'viem'

import type { LiquidationPlan } from './sizing/plan'
Expand Down Expand Up @@ -37,9 +37,11 @@ const candidateOf = (plan: LiquidationPlan) => ({
*
* `resolveRoute` exposes just the pair half of that pipeline, for the tick's phase A.5, so the expensive
* half is not duplicated: the probe refresh the composer then drives for the same pair is absorbed by
* the selector's staleness gate, phase A.5 having already warmed it. Resolving twice is free for plain
* collateral (the unwrappers memoize their negatives per token) but does cost one extra amount-dependent
* read per candidate for a genuinely exotic one — an `eth_call`, never a venue call.
* the selector's staleness gate, phase A.5 having already warmed it. It walks
* {@link previewUnwrapChain} rather than resolving, so an exotic collateral costs NO amount-dependent
* work here — a PT's pair comes off the TTL-cached markets list and a vault share's off the memoized
* `asset()`, where resolving would have spent a hosted Pendle request whose calldata is then discarded
* and re-fetched at quote time. Falls back to a full resolve only if some unwrapper lacks the seam.
*/
export function composeQuoting(deps: {
httpClient: RateLimitedClient
Expand Down Expand Up @@ -76,6 +78,8 @@ export function composeQuoting(deps: {
...rest,
executor,
unwrappers,
// Loan-as-collateral slots need no route, and `ALLOW_BAD_DEBT_ONLY` is supported here.
swapFreeWithoutVenues: true,
Comment thread
haydenshively marked this conversation as resolved.
// The composer owns the probe refresh now: it runs after unwrap resolution so probes price the
// tradable underlying, not an exotic collateral the venues can't quote.
refresh: selector.refresh,
Expand All @@ -87,18 +91,25 @@ export function composeQuoting(deps: {
async resolveRoute(plan, out, label) {
const collateral = out.market.collateralParams[plan.collateralIndex]
if (!collateral || excluded(collateral.token)) return null
const resolution = await resolveUnwraps(unwrappers, {
const previewed = await previewUnwrapChain(unwrappers, {
token: collateral.token,
amountIn: plan.seizedAssets,
executor,
stopToken: out.market.loanToken,
correlation: { id: label, ...candidateOf(plan) }
stopToken: out.market.loanToken
})
if (isAddressEqual(resolution.token, out.market.loanToken)) return null
return {
pair: { collateral: resolution.token, loan: out.market.loanToken },
amountIn: resolution.amountIn
}
// `amountIn` is a probe-interpolation input, so the seize is close enough: it feeds a cost
// estimate, never an encoded min-out. The full resolve is the fallback, and only it threads the
// chain's worst-case output.
const { token, amountIn } =
previewed !== null
? { token: previewed, amountIn: plan.seizedAssets }
: await resolveUnwraps(unwrappers, {
token: collateral.token,
amountIn: plan.seizedAssets,
executor,
stopToken: out.market.loanToken,
correlation: { id: label, ...candidateOf(plan) }
})
if (isAddressEqual(token, out.market.loanToken)) return null
return { pair: { collateral: token, loan: out.market.loanToken }, amountIn }
},

async quoteFor(plan, out, label) {
Expand Down
37 changes: 31 additions & 6 deletions bots/midnight-liquidation/src/runner/revert-streak.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,17 @@ import type { Hex } from 'viem'
*/
export const REVERT_STREAK_ESCALATE_MS = 15 * 60_000

/**
* How long a label may go with no recorded revert before the next one starts a FRESH streak.
*
* Independent of {@link REVERT_STREAK_ESCALATE_MS}, which measures how long one streak has run — a
* stuck position is re-attempted every sweep, so silence means it stopped being attempted at all, not
* that it is stuck harder. Sized against the sweep period rather than the incentive ramp: ~30 Base
* blocks, loose enough to ride out a tick that skipped the position, far tighter than the minutes or
* hours between two separate borrow episodes under one `market:borrower` label.
*/
export const REVERT_STREAK_EPISODE_GAP_MS = 60_000

/** What {@link RevertStreakStore.record} learned about the streak the just-recorded revert extends. */
export type RevertStreak = {
/** Consecutive execution-reverted sends, this one included. */
Expand All @@ -21,9 +32,11 @@ export type RevertStreak = {
/** The 4-byte selector this send reverted with, absent when the payload carried none. */
selector: Hex | undefined
/**
* True while every send in the streak reported the same selector — much stronger evidence of a
* structural fault (a closed gate, malformed calldata, an estimator discrepancy) than a mixed
* streak, which reads as ordinary min-out shortfalls against whichever pool the route hit.
* True while every send in the streak reported the same 4-byte selector. Evidence of a structural
* fault (a closed gate, malformed calldata, an estimator discrepancy) rather than ordinary min-out
* shortfalls, but WEAK evidence in one direction: every `require` string shares `0x08c379a0` and
* every arithmetic fault `0x4e487b71`, so unrelated failures can hold it true. Read it with the
* decoded `reason` on the corresponding `tx.submit_failed`, which does distinguish them.
*/
selectorConstant: boolean
/**
Expand All @@ -46,6 +59,10 @@ export type RevertStreak = {
* provider-side estimator behaviour, and pool state that moved in between. It only reports; it never
* suppresses.
*
* A streak spans one EPISODE: {@link REVERT_STREAK_EPISODE_GAP_MS} of silence starts a fresh one, so
* a label reused by a later borrow neither inherits an escalation nor reports a crossing it did not
* earn.
*
* In-memory only, like the shared `Backoff` and `CooldownStore` — chain truth wins on restart. Entries
* for a position that recovers to non-liquidatable are never re-checked and linger until process exit:
* the same accepted, bounded leak `createBackoff` documents at its canonical home.
Expand All @@ -60,6 +77,7 @@ export type RevertStreakStore = {
type Entry = {
count: number
startedAt: number
lastAt: number
selector: Hex | undefined
constant: boolean
escalated: boolean
Expand All @@ -70,22 +88,29 @@ type Entry = {
* incentive shape — a wall-clock LIF ramp — and no second consumer exists yet.
*/
export const createRevertStreakStore = (
opts: { escalateAfterMs?: number; now?: () => number } = {}
opts: { escalateAfterMs?: number; episodeGapMs?: number; now?: () => number } = {}
): RevertStreakStore => {
const escalateAfterMs = opts.escalateAfterMs ?? REVERT_STREAK_ESCALATE_MS
const episodeGapMs = opts.episodeGapMs ?? REVERT_STREAK_EPISODE_GAP_MS
const now = opts.now ?? (() => Date.now())
const streaks = new Map<string, Entry>()

return {
record: (label, selector) => {
const at = now()
const previous = streaks.get(label)
const prior = streaks.get(label)
// A gap past {@link REVERT_STREAK_EPISODE_GAP_MS} is a NEW episode, not one long streak: nothing
// was being declined in between (a competitor cleared the position, or it stopped being
// liquidatable). Labels are `market:borrower` and so are reused across borrow episodes, and
// without this a stale `startedAt` makes the first revert of the next one report a false
// crossing — or, if the old entry had escalated, yield `ongoing` forever and never warn at all.
const previous = prior && at - prior.lastAt <= episodeGapMs ? prior : undefined
const count = (previous?.count ?? 0) + 1
const startedAt = previous?.startedAt ?? at
const constant = previous ? previous.constant && previous.selector === selector : true
const durationMs = at - startedAt
const past = durationMs > escalateAfterMs
streaks.set(label, { count, startedAt, selector, constant, escalated: past })
streaks.set(label, { count, startedAt, lastAt: at, selector, constant, escalated: past })
return {
count,
durationMs,
Expand Down
Loading
Loading