Skip to content

Remove all-zeros page WAL compression to reduce redundant WAL on extension#946

Open
markrui wants to merge 1 commit into
neondatabase:REL_17_STABLE_neonfrom
markrui:REL_17_STABLE_neon
Open

Remove all-zeros page WAL compression to reduce redundant WAL on extension#946
markrui wants to merge 1 commit into
neondatabase:REL_17_STABLE_neonfrom
markrui:REL_17_STABLE_neon

Conversation

@markrui

@markrui markrui commented Jun 15, 2026

Copy link
Copy Markdown

Remove all-zeros page WAL compression to reduce redundant WAL on extension

Summary

Revert the Neon-specific branch in XLogRecordAssemble that compresses an
all-zeros new page into a single whole-page hole (hole_offset = 0,
hole_length = BLCKSZ), and restore vanilla PostgreSQL's standard
pd_lower/pd_upper hole computation.

This is the PostgreSQL half of a two-part change. The companion Neon PR
switches relation extension to emit a single marker FPI for the last
extended block instead of one full-page image per block, which makes this
compression hack unnecessary.

Background: why this hack exists

When PostgreSQL extends a relation, Neon's SMGR emits one XLOG_FPI
record per newly allocated block, each containing an all-zero page image.
An all-zero page fails vanilla PostgreSQL's standard hole-elision check in
XLogRecordAssemble — which (for a REGBUF_STANDARD page) requires
pd_lower >= SizeOfPageHeaderData && pd_upper > pd_lower && pd_upper <= BLCKSZ
— because a zeroed page has pd_lower = pd_upper = 0. With no hole elided,
each such FPI would otherwise carry a full 8 KB body.

To keep those per-block zero FPIs cheap, postgres#327 (commit
8d60df0cd637829e01f62013ff10450f5b25a626, "Optimize stroing zero FPI in
WAL", 2023-11) made a two-part change:

  1. Producer sidesrc/backend/access/transam/xloginsert.c: a
    PageIsNew() branch in XLogRecordAssemble that treats the whole page
    as a hole, so the FPI body shrinks to nothing:

    if (PageIsNew(page))
    {
        bimg.hole_offset = 0;
        cbimg.hole_length = BLCKSZ;   /* treat the whole page as a hole */
    }
  2. Reader sidesrc/backend/access/transam/xlogreader.c: a matching
    exception in the FPI hole cross-check so that such whole-page-hole
    records (hole_offset == 0 && hole_length == BLCKSZ) are not rejected
    on replay.

It works, but the producer-side records deliberately diverge from the
upstream WAL format (a record vanilla PostgreSQL's redo would reject),
documented in core_changes.md under "WAL-log an all-zeros page as one
large hole".

Why it is now removable

The hack only ever existed to make the per-block zero-page FPIs cheap.
The companion Neon change stops emitting per-block zero FPIs altogether:
it logs a single marker FPI for the last extended block, and that marker
page is PageInit'd. After PageInit the page has
pd_lower = SizeOfPageHeaderData, pd_upper = BLCKSZ, so the standard
hole computation elides all but the 24-byte page header — essentially the
same tiny FPI the hack produced (a 24-byte header vs. the hack's 0-byte
body), without any special-casing.

With the extension path no longer emitting all-zeros standard FPIs, the
PageIsNew() branch is no longer exercised. Removing it is safe
regardless: any residual all-zeros REGBUF_STANDARD page would simply
fall through to the standard path and be logged as a valid full 8 KB
image, rather than a non-vanilla whole-page-hole record. Removing it:

  • restores byte-compatibility of extension FPIs with vanilla PostgreSQL's
    WAL format (these records become parseable by an unmodified
    pg_waldump / redo path), and
  • lets one section be deleted from Neon's core_changes.md.

Changes

  • src/backend/access/transam/xloginsert.c: drop the PageIsNew() branch
    in the REGBUF_STANDARD path and de-nest the standard hole computation.
    The result is identical to upstream PostgreSQL 17.

This reverts only the producer side of postgres#327. The reader-side
exception in xlogreader.c is intentionally kept (see Compatibility).
So this PR restores vanilla WAL format for newly produced extension
FPIs; the redo path stays deliberately more lenient than vanilla.

Compatibility

WAL already produced by the old code (records with
hole_offset == 0 && hole_length == BLCKSZ, i.e. an empty FPI body) is
still present in safekeepers. To keep replaying it, this PR keeps the
reader-side exception postgres#327 added to the FPI hole cross-check in
xlogreader.c; removing it would make redo reject those historical
records. New extension WAL no longer produces such records (the companion
Neon change emits a PageInit'd marker page that elides via the standard
condition), so the reader exception is dead weight only for fresh WAL and
exists purely for backward compatibility. Existing safekeeper-stored WAL
does not need to be rewritten.

Dependency / ordering

This must land together with the companion Neon marker-FPI PR. Merging
this revert alone, while Neon still emits per-block zero FPIs, would
temporarily inflate extension WAL (those FPIs lose their compression).
Recommended order: merge this PR first, then bump the Neon
vendor/postgres-vXX submodule to this PR's merged commit.

Testing

  • Clean rebuild of PostgreSQL v17 from this branch.
  • make check (core regression): 222 / 222 passed.

References

  • postgres#327 (8d60df0cd637829e01f62013ff10450f5b25a626, "Optimize
    stroing zero FPI in WAL", 2023-11) — added the PageIsNew hole hack to
    xloginsert.c and the matching reader exception in xlogreader.c
  • neon#5910 (64890594a) — submodule bump that picked up the hack
  • src/backend/access/hash/hashpage.c _hash_alloc_buckets — vanilla
    PostgreSQL's long-standing "log only the last block" extension pattern,
    which the Neon side now mirrors

…nsion

Revert the Neon hack in XLogRecordAssemble that compressed an all-zeros
new page into a single whole-page hole (hole_offset = 0,
hole_length = BLCKSZ), and restore vanilla PostgreSQL's standard
pd_lower/pd_upper hole computation.

This special compression existed only to make the per-block zero-page
FPIs that Neon emitted on relation extension cheap. Neon now logs a
single marker FPI for the last extended block instead of one FPI per
block, so those redundant per-block WAL records are gone and the
compression hack is no longer needed. Removing it cuts WAL volume during
relation extension, improves write performance, and keeps the WAL
parseable by unmodified PostgreSQL.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant