Remove all-zeros page WAL compression to reduce redundant WAL on extension#946
Open
markrui wants to merge 1 commit into
Open
Remove all-zeros page WAL compression to reduce redundant WAL on extension#946markrui wants to merge 1 commit into
markrui wants to merge 1 commit into
Conversation
…nsion Revert the Neon hack in XLogRecordAssemble that compressed an all-zeros new page into a single whole-page hole (hole_offset = 0, hole_length = BLCKSZ), and restore vanilla PostgreSQL's standard pd_lower/pd_upper hole computation. This special compression existed only to make the per-block zero-page FPIs that Neon emitted on relation extension cheap. Neon now logs a single marker FPI for the last extended block instead of one FPI per block, so those redundant per-block WAL records are gone and the compression hack is no longer needed. Removing it cuts WAL volume during relation extension, improves write performance, and keeps the WAL parseable by unmodified PostgreSQL.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove all-zeros page WAL compression to reduce redundant WAL on extension
Summary
Revert the Neon-specific branch in
XLogRecordAssemblethat compresses anall-zeros new page into a single whole-page hole (
hole_offset = 0,hole_length = BLCKSZ), and restore vanilla PostgreSQL's standardpd_lower/pd_upperhole computation.This is the PostgreSQL half of a two-part change. The companion Neon PR
switches relation extension to emit a single marker FPI for the last
extended block instead of one full-page image per block, which makes this
compression hack unnecessary.
Background: why this hack exists
When PostgreSQL extends a relation, Neon's SMGR emits one
XLOG_FPIrecord per newly allocated block, each containing an all-zero page image.
An all-zero page fails vanilla PostgreSQL's standard hole-elision check in
XLogRecordAssemble— which (for aREGBUF_STANDARDpage) requirespd_lower >= SizeOfPageHeaderData && pd_upper > pd_lower && pd_upper <= BLCKSZ— because a zeroed page has
pd_lower = pd_upper = 0. With no hole elided,each such FPI would otherwise carry a full 8 KB body.
To keep those per-block zero FPIs cheap, postgres#327 (commit
8d60df0cd637829e01f62013ff10450f5b25a626, "Optimize stroing zero FPI inWAL", 2023-11) made a two-part change:
Producer side —
src/backend/access/transam/xloginsert.c: aPageIsNew()branch inXLogRecordAssemblethat treats the whole pageas a hole, so the FPI body shrinks to nothing:
Reader side —
src/backend/access/transam/xlogreader.c: a matchingexception in the FPI hole cross-check so that such whole-page-hole
records (
hole_offset == 0 && hole_length == BLCKSZ) are not rejectedon replay.
It works, but the producer-side records deliberately diverge from the
upstream WAL format (a record vanilla PostgreSQL's redo would reject),
documented in
core_changes.mdunder "WAL-log an all-zeros page as onelarge hole".
Why it is now removable
The hack only ever existed to make the per-block zero-page FPIs cheap.
The companion Neon change stops emitting per-block zero FPIs altogether:
it logs a single marker FPI for the last extended block, and that marker
page is
PageInit'd. AfterPageInitthe page haspd_lower = SizeOfPageHeaderData,pd_upper = BLCKSZ, so the standardhole computation elides all but the 24-byte page header — essentially the
same tiny FPI the hack produced (a 24-byte header vs. the hack's 0-byte
body), without any special-casing.
With the extension path no longer emitting all-zeros standard FPIs, the
PageIsNew()branch is no longer exercised. Removing it is saferegardless: any residual all-zeros
REGBUF_STANDARDpage would simplyfall through to the standard path and be logged as a valid full 8 KB
image, rather than a non-vanilla whole-page-hole record. Removing it:
WAL format (these records become parseable by an unmodified
pg_waldump/ redo path), andcore_changes.md.Changes
src/backend/access/transam/xloginsert.c: drop thePageIsNew()branchin the
REGBUF_STANDARDpath and de-nest the standard hole computation.The result is identical to upstream PostgreSQL 17.
This reverts only the producer side of postgres#327. The reader-side
exception in
xlogreader.cis intentionally kept (see Compatibility).So this PR restores vanilla WAL format for newly produced extension
FPIs; the redo path stays deliberately more lenient than vanilla.
Compatibility
WAL already produced by the old code (records with
hole_offset == 0 && hole_length == BLCKSZ, i.e. an empty FPI body) isstill present in safekeepers. To keep replaying it, this PR keeps the
reader-side exception postgres#327 added to the FPI hole cross-check in
xlogreader.c; removing it would make redo reject those historicalrecords. New extension WAL no longer produces such records (the companion
Neon change emits a
PageInit'd marker page that elides via the standardcondition), so the reader exception is dead weight only for fresh WAL and
exists purely for backward compatibility. Existing safekeeper-stored WAL
does not need to be rewritten.
Dependency / ordering
This must land together with the companion Neon marker-FPI PR. Merging
this revert alone, while Neon still emits per-block zero FPIs, would
temporarily inflate extension WAL (those FPIs lose their compression).
Recommended order: merge this PR first, then bump the Neon
vendor/postgres-vXXsubmodule to this PR's merged commit.Testing
make check(core regression): 222 / 222 passed.References
8d60df0cd637829e01f62013ff10450f5b25a626, "Optimizestroing zero FPI in WAL", 2023-11) — added the
PageIsNewhole hack toxloginsert.cand the matching reader exception inxlogreader.c64890594a) — submodule bump that picked up the hacksrc/backend/access/hash/hashpage.c_hash_alloc_buckets— vanillaPostgreSQL's long-standing "log only the last block" extension pattern,
which the Neon side now mirrors