Repository navigation
Conversation
…pty data (rendering issue)
…w-sgho-boost, message:none is correct final state
…spread
The fetchMeritTimeRange return builder used truthy spread
(name && {name}) which stripped name:'' and message:[]
from cached objects. Next cron cycle loaded these as
undefined → isCachedTimeRangeComplete flagged them
as missing → infinite retry loop → Worker 429 → OOM.
Same class of bug as d9cad34 but in the write path
instead of the read/check path.
…mpted, not missing Three truthy-gate bugs causing infinite retry loop: 1. fetchMeritTimeRange: name=undefined → result.name stays undefined (now: name ?? '' writes empty string = attempted) message=[] not written to result (now: always writes []) 2. fetchMeritTimeRange return builder: truthy spread strips name:'' and message:[] from cache writes (now: !== undefined) 3. isCachedTimeRangeComplete: endDate:'' treated as missing via truthy check (now: !== undefined, aligned with name/message) 4. loadCachedMeritCampaignMetadata: hasRequiredFields required endDate non-empty, blocking entries with endDate:'' from entering validatedTimeRanges (now: endDate !== undefined suffices, matching isCachedTimeRangeComplete semantics) Root cause: ethereum-new-sgho-boost is a 404 page — all fields come back empty. Truthy gates filtered these as 'not attempted' instead of 'attempted, no data', causing every cron cycle to retry → Worker 429 → memory growth → OOM.
… single-source Map key construction (AAV-786)
…ish memory safety checklist - persistenceService: max entries for marketRowHashes(500), marketConfigHashes(500), oraclePriceHashes(2000) - merit-api: roundEstimateCache max 200, shrinkCampaignMetadataCache() with stale-key removal + max 500 + FIFO overflow eviction - server: RSS restart guard log level warn→error with heap/external details - docs: new memory-leak-checklist.md (3-layer defense principle, per-item checklist, Key Format single-source) - AGENTS.md: Memory Safety Rule referencing checklist - tests: merit-cache-shrink.test.ts (6 tests covering stale removal, self- prefix, max entries, FIFO eviction)
…-786) Introduce named key builder functions in @internal/aave-shared-contracts to eliminate 25 inline template-literal Map/Set key constructions across the codebase. This prevents format-mismatch bugs (like the oraclePriceHashes shrink no-op caused by inconsistent key formats). Key functions added to shared-contracts/src/keys.ts: - normalizeAddress: unified address normalization (toLowerCase + trim) - spokeKey: chainId:spokeAddress - v4SpokeCacheKey: spokeAddress:hubAddress - v3PriceKey / v4PriceKey: oracle lean price cache keys - v3OnchainKey / v4OnchainKey: onchain data reserve keys - chainTokenKey / chainSymbolKey: token/symbol lookup keys - topologySortKey: topology dedup/sort key - v4ReserveId / aaveProReserveId: V4 reserve identifiers Consumer changes: - oracleService: 6 inline keys → spokeKey + v3PriceKey + v4PriceKey - aave-rpc-infra: 4 inline keys → spokeKey + v4ReserveId + aaveProReserveId - addressBookRegistry: 3 inline keys → spokeKey + topologySortKey - onchainDataService: 4 inline keys → v4SpokeCacheKey + v3OnchainKey + v4OnchainKey - aave-fetcher: 3 inline keys → chainTokenKey + chainSymbolKey - merkl-api: 6 inline keys → chainTokenKey + chainSymbolKey - v4-topology: 1 inline key → topologySortKey - v4-fetcher: 1 inline key → v4ReserveId - concurrent-fetch: 1 inline key → v3OnchainKey Local normalizeAddress in aave-rpc-infra and onchainDataService replaced by shared-contracts import (single definition).
Adds 6 tests covering shrinkHashMaps and shrinkOraclePriceHashes: - stale key removal (not in current batch) - max entries + FIFO overflow eviction - empty input clears all entries Exports shrink functions and test helpers with ts-prune-ignore-next.
…sessment Complete inventory of all 38 in-memory caches across backend + packages, with per-item Domain/TTL/Max/Shrink status and accept/reject rationale. Identifies 6 caches needing max entries (P3) and documents why 2 are acceptable without max (rate limiters).
…ache/ProviderPool sizing - DynamicRpcCache: max 50 (chainId ~20, 2.5× headroom) - ProviderPool: max 150 (≥ DynamicRpcCache.max × avg URLs per chain ~3) - leanPriceCache: max 500, V4_RESERVE_TOKEN_CACHE: max 100 - poolCache: max 50, v4SpokeCache: max 80 - docs: update inventory table (6 🟡→🟢), add cache audit rule to AGENTS.md
…che invalidate as shrink layer - Rename layer 2 from 'TTL' to 'TTL/Shrink' with explicit distinction - FIFO overflow is Size layer execution, not a shrink strategy - DynamicRpcCache: invalidate is shrink layer (driven by ProviderPool health), upgrade 🟡→🟢
…yer-progression loop (AAV-822) Replace the if (urls.length === 0) branch in executeWithAutoRpc with unified needsMore-based layer progression: hardcoded → viem → dynamic. All chains now follow the same path. Hardcoded chains with all RPCs suppressed fallback to viem/chains URLs, then DynamicRpcCache URLs. Normal path has zero overhead (needsMore=false skips extra layers). New: needsMore(), mergeUrls() helpers. ADR-0027 updated. Closes AAV-795.
… overflow loops - Add fifoEvict<K>(map, maxEntries) to @internal/aave-shared-contracts - Fix falsy-key bug: use iterator done check instead of !oldestKey - Replace inline while-loop FIFO overflow in: persistenceService (3), oracleService (2), onchainDataService (3), dynamicRpcCache (2), merit-api (2), token-price-resolver (1), merklForecastService (2) - Keep ProviderPool inline (needs 3-map联动 deletion) - Add 7 fifoEvict tests including falsy key coverage - Update memory-leak-checklist.md with fifoEvict section
…rade hono 4.12.25 - tests/**/*.test.ts → tests/*.test.ts in 4 packages (shared-contracts, fetcher, rpc-infra, root): ** does not expand in CI sh -c (no globstar), causing Could not find errors on GitHub Actions - hono override ^4.11.9 → ^4.12.25: fixes 4 moderate GHSA vulnerabilities (xrhx-7g5j-rcj5, 3hrh-pfw6-9m5x, f577-qrjj-4474, 2gcr-mfcq-wcc3)
- New script: scripts/check-no-globstar-in-test.sh scans all package.json test scripts for **/ patterns that won't expand in CI bash - Added to ci:remote pipeline (pre-push hook enforcement) - Updated AGENTS.md: Architecture Rules + Validation Gate - Added to .gitignore whitelist
…ey, v4OnchainKey, aaveProReserveId v3OnchainKey: normalize poolAddress + tokenAddress v4OnchainKey: normalize tokenAddress (spoke/hub already done) aaveProReserveId: normalize underlying token address All callers already pass lowercase addresses, so zero runtime behavior change. This is a defensive consistency fix ensuring all key constructor functions provide the same normalizeAddress guarantee.
…emory-leak-checklist
…R cap errors - Add TARGET_TOTAL_APR as 4th CampaignForecastType / ForecastCampaignTypeLite - Map 7 Target Total APR distributionMethods (AAVE_NET_APR, AAVE_V4_NET_APR, ERC4626_APR, ERC4626_SPREAD_CAPPED, ERC4626_TARGET_APR_WITH_MERKL, SOFR_SPREAD_RATCHET, DEEL_DISTRIBUTION) to TARGET_TOTAL_APR - Remove L3 mode mapping — mode is budget-bound strategy, not type signal - Rename extractMaxApr → extractAprCap(campaign, campaignType): TARGET_TOTAL_APR reads distributionSettings.targetAPR, others read distributionSettings.apr - Extend aprCap guard to MAX/FIX/TARGET_TOTAL_APR in buildForecastState - Serialize targetAPR in CampaignSnapshotLite (both fetcher + backend) - Add budgetBoundMode, spreadCap fields to MerklCampaignBreakdown - Add TARGET_TOTAL_APR rules to merklApiContract (matches MAX pattern) - Update ADR-0024: 2-level mapping, L3 removed, Target Total APR section Fixes AAV-833 (3 Merkl campaigns continuously reporting Missing APR cap)
…iveRewards - New module: brevis-distributed-so-far.ts reads tokenCumulativeRewards(token) via Multicall3 on submitAddr/submitChainId (NOT claimAddr/claimChainId) - Uses getAaveRpcUrlsByChainId() for real RPC URLs with fallback rotation - BigInt-safe normalization: whole-part division + remainder separately - Enriched in fetchBrevisAprs() post-processing, not in gRPC parse path - distributedSoFarUsd added to BrevisCampaignBreakdown type + prune whitelist - API serialization passthrough (no percentage scaling for USD fields) - Schema fingerprint updated: e90ee7642b64 → 6b86418edd07 - 5 integration tests with real Optimism RPC + contract Fixes: AAV-848, AAV-849, AAV-850
…evisForecast - New brevisForecastService.ts: cron-write/API-read-only memory store - metaController.ts: brevisForecast sub-block in GET /api/meta/side-data - marketsService.ts: setBrevisForecastEntries on markets cron - BrevisCampaignBreakdown: remove distributedSoFarUsd field - fetchBrevisAprs returns BrevisProcessedData (index + brevisDistributedSoFar) - enrichDatasetWithIncentiveData simplified back to RuntimeReserveData[] - prune whitelist and schema fingerprint updated - API docs + memory-leak-checklist updated Refs: AAV-858, AAV-859, AAV-860, AAV-861
- Remove brevisForecast side-data sub-block; Brevis items now merged into forecast.items[] alongside Merkl items - Rename distributedSoFarUsd -> distributedSoFar (align with Merkl) - Add optional endTimestamp to Brevis forecast items (from breakdown campaignEndedAt, Unix seconds) - Add 1h chain call cache to fetchBrevisDistributedSoFar (TTL + 2h lazy prune + FIFO overflow max=100) - ForecastResponseItem.endTimestamp now optional (Merkl always has it, Brevis may not) - Side-data partial count 5 -> 4 - Update tests, API docs, memory-leak-checklist - Frontend handoff doc in aaveapy-doc repo
Enforce that tests involving on-chain contracts or third-party APIs (Merkl/Brevis/CoinGecko) must use real URLs/contract addresses, not mocks. API contract changes must be verified against dev/staging.
- BrevisCampaignBreakdown: campaignId required, add campaignType/aprCap, remove budgetNormalizedAmount/budgetTokenSymbol from shared-contracts type - brevis-api.ts: set campaignType=FIX, aprCap=campaignApr at build time; keep budget* in fetcher-local type for enrich, prune strips them - ForecastResponseItem.endTimestamp: required (was optional) - brevisForecastService: skip items without endTimestamp - incentive-prune: preserve campaignType + aprCap in Brevis breakdown - Update tests for required campaignId and endTimestamp
…AV-867) Brevis gRPC campaign.type is an action type, not a distribution type. All Aave campaigns are fixed APR + fixed budget (no Dutch auction/MAX). Comment documents this for future maintainers.
scaleBrevisBreakdown now applies ×100 scale to aprCap (matching Merkl behavior). Previously aprCap was passed through as raw ratio, causing frontend to display 0.024 instead of 2.4%.
…price lookup The local chainTokenKey used hyphen separator (59144-0x...) while shared-contracts uses colon (59144:0x...). This mismatch caused tokenPrices.get() to always return undefined, making distributedSoFar undefined for all Brevis campaigns, which then got filtered out by brevisForecastService.
…h-risk areas - Lesson Learned: Map key must use shared utility function, never re-implement locally (brevis-distributed-so-far chainTokenKey mismatch) - High-Risk Areas: add brevis-distributed-so-far.ts
…es dynamic, spreadCap removal (AAV-877/878/879/880) - Remove spreadCap from MerklCampaignBreakdown type (AAV-877) - Pass budgetBoundMode from fetcher rawMode through to API output (AAV-878) - Refactor ApiMerklBreakdown from Pick<> whitelist to direct equality (AAV-878) - Add budgetBoundMode param to getBreakdownFieldRule/getForecastFieldRule (AAV-879) - TARGET_TOTAL_APR+FIX_APR uses FIX rule (omit plannedDaily, no requiredDaily) (AAV-879) - Update ADR-0024 with budgetBoundMode consequences - Add 11 new tests covering budgetBoundMode field rules and serialization
- BaseCampaignBreakdown: positionCap (number) → positionCapNative (string, Merkl raw amount) + positionCapUsd (number, Merit/Brevis USD) - extractPositionCapFromCampaign: remove targetTokenPrice/targetTokenDecimals params, return raw string - buildReserveUnderlyingLookup: simplify Map→Set (only needs reserve existence check) - Delete ReserveUnderlyingInfo interface - fetchMerklCampaignDetails: can now extract positionCapNative directly - Merit/Brevis: positionCap → positionCapUsd - OpenAPI schema + tests updated
Heap snapshot analysis confirmed ~1K uncollected Markets/Chains query string instances accumulating per day. V3 AaveClient was a module-level singleton, but urql's fetchExchange and Client retain Operation objects + Response references after each .query().toPromise() call. Creating a fresh client per cron cycle lets the old one (and its urql internals) be GC'd. Matches the V4 pattern already in v4-fetcher.ts. Measured impact (3 cron cycles, settled state): heap: 68MB → 47MB (-31%), old_space: 57MB → 36.5MB (-36%) RSS: 430-450MB → 143MB (-67%)
…ixed 4h production monitoring (243 samples) confirms memory is stable: - heap: 50-52MB (no growth), old_space: 39-40MB (no growth) - RSS: 154-159MB (no growth), https free sockets: non-accumulating - Previously reported ~1.2 MB/h was sampling bias (peak vs settled) Removed: - MEMORY_DIAG env var and all gated code paths - /api/debug/heap-snapshot, heap-snapshot/:filename, heap-stats endpoints - --expose-gc from Dockerfile CMD - rssDelta()/rssMark from aave-fetcher (both CLI and fetchMarketsData) - logOldSpaceDiff/tryGc/logHeapDiff/withHeapTrace/maybeHeapSnapshot from updateScheduler - heap-diff block in marketsService - 10min heap detail interval in server.ts Kept (production monitoring / defensive code): - 📊 Memory 60s periodic log with cache stats + undici/nodeAgent stats - --max-old-space-size=512, MALLOC_ARENA_MAX=2, RSS_RESTART_THRESHOLD_MB=800 - maxSockets=10, maxFreeSockets=2 for http/https globalAgent - undici globalDispatcher with connection limits
…rcent conversion Both rayStringToPercent (onchainDataService.ts) and rayToPercent (aave-rpc-infra/index.ts) were dividing by 10^27 (returning RAY decimal) instead of 10^25 (returning percent = RAY decimal x 100). This caused baseBorrowRate for 47 V3 reserves to be 100x too low (e.g., USDS returned 0.055 instead of 5.5), breaking frontend rate simulation. Fix: change 10n**21n to 10n**19n in both functions. TDD: 8 tests for rayStringToPercent, 11 tests for rayToPercent. Verified on live dev server: USDS baseBorrowRate=5.5. Closes AAV-1108, AAV-1109. Parent: AAV-1106.
…t value Updated example from 0 = 0% to 5.5 = 5.5% to reflect the fixed percent unit.
… CI auto-revert Fix: 4 places in marketsApiSerialize.test.ts used old positionCap instead of positionCapUsd (renamed in AAV-1075). Also fix scaleMeritCampaignBreakdown type constraint. Prevention: (1) Add test:typecheck to pre-commit hook so test file type errors are caught at commit time. (2) Add CI auto-revert job: if CI fails on direct push (not PR), automatically revert the commit and create an issue. This is the safety net for git push --no-verify bypassing the pre-push hook.
pre-commit now auto-fixes: bin-paths (./node_modules/.bin/X -> npx --no-install X), globstar (tests/**/*.test.ts -> tests/*.test.ts). pre-push runs ci (build+test, non-fixable) then auto-fixes bin-paths, globstar, and audit. If pre-push auto-fix changes files, it amends the last commit and asks user to push again. Non-auto-fixable checks (build, typecheck, test, prune, workspace-coverage) still fail immediately. Verified: simulated bin-paths and globstar violations were both auto-fixed successfully.
- Create units.ts: rayToRatio/rayToPercent/FIELD_UNITS/SERIALIZER_RULES - Fix V4 RPC borrowApy: rayToRatio instead of rayToPercent - Refactor onchainDataService to use shared rayToPercent - Add invariant tests: units.test.ts + unitsConsistency.test.ts - Update AGENTS.md with Unit & Precision Safety Rule
…AAV-1106)" This reverts commit a562f32.
Records the architectural decision for units.ts, FIELD_UNITS registry, three-layer test safety net, and AAV-1106 systemic fix.
CI failed because rayStringToPercent re-export was flagged as unused by ts-prune (only tests import it). Tests now import rayToPercent directly from @internal/aave-shared-contracts. Also fix RuntimeReserveData to Record cast in unitsConsistency.test.ts.
Shallow clone (fetch-depth: 1 default) caused git revert to treat all 370 files as new additions from the commit, deleting the entire repo. fetch-depth: 0 ensures full history for correct revert.
…e dead code S1: Delete local rayToPercent from aave-rpc-infra + its test file; merge unique test values (5.5%, 35%, 0.001%) into shared units.test.ts. S2/P2: ADR-0035 incorrectly said re-export, actual impl directly imports. S3: Remove unused typeKeys variable + unused RuntimeReserveData import in units.test.ts.
…atio merit-api.ts had a local function meritAprPercentToRatio(percent) that was identical to percentToRatio(percent) from units.ts. Replaced all 10 call sites with the shared function and deleted the local definition.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
# Conflicts: # .claude/commands~HEAD # .github/dependabot.yml # .github/workflows/auto-approve-remediation-pr.yml # .github/workflows/ci.yml # .gitignore # AGENTS.md # README.md # backend/package-lock.json # backend/package.json # backend/scripts/dev-clean.sh # backend/scripts/dev-entry.sh # backend/src/cacheTtl.ts # backend/src/config.ts # backend/src/controllers/coingeckoController.ts # backend/src/controllers/marketsController.ts # backend/src/controllers/merklForecastController.ts # backend/src/controllers/metaController.ts # backend/src/middleware/cacheHeaders.ts # backend/src/server.ts # backend/src/services/ethProviderService.ts # backend/src/services/marketsApiSerialize.ts # backend/src/services/marketsService.ts # backend/src/services/merklForecastService.ts # backend/src/services/onchainDataService.ts # backend/src/services/updateScheduler.ts # backend/src/types/index.ts # backend/tests/marketsApiSerialize.test.ts # backend/tests/merklForecastService.test.ts # backend/tests/startup.test.ts # docs/api/FRONTEND-SYNC-CHANGES.md # docs/api/api-documentation.md # docs/api/brevis-supplement.md # docs/api/markets-api-sdk-field-validation.md # docs/api/native-apr-calculation.md # docs/backend/HARDCODE-AND-EXTERNAL-IMPORTS.md # docs/backend/data-freshness-mechanism.md # docs/backend/data-precision-comparison.md # docs/backend/rpc-endpoints.md # docs/ci-security-automation.md # docs/deploy/cloudflare-complete-guide.md # docs/deploy/deploy.md # docs/deploy/railway-three-service-architecture.md # docs/development-best-practices.md # docs/merkl-merit-cache-architecture.md # docs/redundant-fields-and-calculations-audit.md # docs/reusable/README.md # docs/reusable/caching-data-freshness-patterns.md # docs/reusable/ci-git-hooks-patterns.md # package-lock.json # package.json # packages/aave-fetcher/src/brevis-api.ts # packages/aave-shared-config/index.d.ts # packages/aave-shared-config/index.js # scripts/sync-coingecko-platform-map.ts # src/index.ts # src/merit-api.ts # src/merkl-api.ts # workers/package-lock.json # workers/package.json # workers/src/browser-pool.ts
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
- seoController: add sanitiseGroupCol() whitelist barrier so CodeQL can trace that GROUP BY/ORDER BY columns never come from raw user input - seo.ts / swagger.ts: add rateLimitMiddleware to satisfy missing-rate-limit alerts - seoIntegration.test.ts: mirror production router setup with rate limiter - workers/src/index.ts: return generic error instead of raw error.message
…workflow - rateLimit.ts: delegate to express-rate-limit (recognised by CodeQL) instead of hand-rolled Map-based implementation. Call sites unchanged. Gains standard RateLimit-* response headers automatically. - codeql.yml: explicit workflow scanning both main and railway on push, PR, and weekly schedule so issues surface before merge PRs.
The managed CodeQL setup already passes after the express-rate-limit fix. Custom workflow caused SARIF upload conflicts. Railway branch scanning should be enabled via GitHub Settings UI instead.
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
railwaybranch (active development branch) intomainmain, covering: memory leak fixes, V3/V4 SDK per-fetch client creation, unit conversion system (ADR-0035), Merkl/Brevis incentive improvements, recently ended campaign support, position cap fields, node-fetch removal (undici), CI hardening, and various bug fixesdevbranch is a subset ofrailwayand can be deprecated after this mergeKey Changes
Memory & Performance
Data & API
positionCapNative+positionCapUsdfields (AAV-1075)lastEndedCampaignnested object replacingrecentlyEnded*scalars (AAV-1044)opportunityIdUnit Conversion (ADR-0035)
FIELD_UNITSregistry in@internal/aave-shared-contracts/units.tsrayToRatio/rayToPercent/percentToRatiofrom single sourceInfrastructure
node-fetch, use built-in fetch/undici (AAV-1064)