Skip to content

testcases for append entries hot spin - #28

Merged
codecasualty merged 1 commit into
mainfrom
fix/append_spin
Sep 25, 2026
Merged

codecasualty merged 1 commit into
mainfrom
fix/append_spin

Conversation

@codecasualty

Copy link
Copy Markdown
Owner

Fix:- WAL Bloat, Replication & Election Bugs

Problem

After ~43 hours of chaos testing, node3's WAL grew to 478 MB because:

  1. Followers rewrote resent AppendEntries batches to the WAL even when there was no conflict.
  2. The leader's replication loop hot-spun, resending unchanged batches ~4,400 times/sec.

This eventually caused WAL replay to exhaust the 512 MB heap and put the node into a crash-restart loop.

What Changed

Follower

  • Implemented Raft §5.3 per-entry conflict detection.
  • Skip entries already present with the same term or entries below the snapshot
  • Only truncate and append the conflicting/new tail
  • Added WAL durability barrier for retries
  • Corrected matchIndex and commitIndex bounds

Leader

  • Added a 20 ms backoff when nextIndex doesn't advance.
  • Prevented hot-spinning on repeated successful responses.

Election

  • A node becomes leader only if it is still a candidate in the same term.
  • Prevents two leaders from being elected in the same term.

WAL Replay

  • Replay is now index-aware.
  • Duplicate/old WAL records are safely collapsed into a clean log.

@codecasualty
codecasualty merged commit 390df50 into main Sep 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant