Skip to content

[Performance] Compress web bundles with Brotli at build time instead of gzip on the fly #101250

Closed
Closed
[Performance] Compress web bundles with Brotli at build time instead of gzip on the fly#101250
Assignees
justinpersaudroryabrahamcollectioneur
Labels
DailyKSv2Task

Description

@melvin-bot
melvin-bot
bot
opened on Sep 15, 2026

Problem

NewDot web assets travel S3 → CloudFront → Cloudflare → browser. The deploy job uploads dist as plain uncompressed files, and CloudFront gzips them on the fly, so every client receives gzip.

The production build is 96 JavaScript files, 41.5 MB raw (11.6 MB gzipped), plus an 8 MB WebAssembly module (3.2 MB gzipped). A first sign-in downloads 6.2 MB (main, vendors, authScreens) before the app can render. With 22 deploys in the last 30 days and main/authScreens rotating on nearly every deploy, returning users re-download up to 6.2 MB each time:

authScreens.bundle.js   2921 KB
vendors.bundle.js       1847 KB
main.bundle.js          1478 KB

Solution

Compress every js/css/html/json/svg/wasm/ttf file with Brotli level 11 at build time, upload the .br copies next to the originals, and let the CDN serve the .br copy to browsers that accept it. Every browser since 2017 supports Brotli; anything else keeps getting gzip exactly as today.

Measured on the current build (gzip 9 is what CloudFront serves today, byte for byte):

                                                              gzip 9   brotli 11          saving
canvaskit.wasm                                              3 215 KB    2 470 KB             23%
authScreens.bundle.js                                       2 921 KB    1 987 KB             32%
vendors.bundle.js                                           1 847 KB    1 399 KB             24%
main.bundle.js                                              1 478 KB    1 066 KB             28%
illustrations.bundle.js                                       787 KB      643 KB             18%
heicTo.bundle.js                                              604 KB      416 KB             31%

First sign-in (112 compressible files)                      15.14 MB    11.34 MB   -3.8 MB (25%)
Re-download after a deploy (main + vendors + authScreens)   6 246 KB    4 452 KB   -1.8 MB (29%)

Time saved at different network speeds:

Speed       Scenario                                6.2 MB gzip  Brotli saves
5 Mbit/s    weak 3G, 4G                                    10 s         2.9 s
10 Mbit/s   average 3G, poor 4G, congested Wi-Fi            5 s         1.5 s
30 Mbit/s   typical 4G                                    1.7 s         0.5 s
100 Mbit/s  5G, home Wi-Fi                                0.5 s        0.15 s

This is a download-time improvement only — the amount of JavaScript the browser parses and runs is unchanged. Images and woff2 fonts are excluded because they are already compressed internally.

Implementation steps

  1. Add a compression plugin to the rsbuild config so every js/css/wasm/ttf file gets a .br twin, compressed once with Brotli 11. The plugin compresses in parallel, so build overhead is ~15–25 s.
  2. Upload the .br files to S3 with Content-Encoding: br and the real Content-Type — a few extra lines in deploy.yml next to the existing s3 cp.
  3. In CloudFront: add a function that checks whether the browser accepts Brotli and, if so, serves .js.br instead of .js. Browsers that don't accept it keep getting gzip as today.
  4. In Cloudflare: enable Brotli on the new.expensify.com zone, otherwise Cloudflare keeps asking the origin for gzip only and step 3 never kicks in.

If NewDot hosting moves entirely to Cloudflare (dropping CloudFront/S3), steps 1 and 2 are unchanged and the CloudFront function becomes a Cloudflare Worker doing the same thing.

Rollback: remove the CloudFront function. Everything goes back to gzip on the fly and the .br files sit unused.

Decisions already settled in the thread

  • Why build time, not on the fly: Brotli 11 is 60–150× slower than gzip, so it can't run per request.
  • Why level 11 over 9: across main/vendors/authScreens, Brotli 9 saves 20% vs gzip and Brotli 11 saves 28%. Level 11 costs 5–12 s per big chunk at build time, which is acceptable for a one-time cost.
  • Why Brotli over zstd: brotli 11 beats zstd 19 on all three main chunks (for example authScreens: 2 032 KB vs 2 178 KB).
  • Decompression cost: gzip 9 is 7.3 ms vs brotli 11 at 8.5 ms for the whole bundle — ~1 ms extra, off the JS thread.

Slack thread: https://expensify.slack.com/archives/C05LX9D6E07/p1789482193542349

Activity

  1. melvin-bot
    assigned
    collectioneur
    ,
    roryabraham
    and
    justinpersaud
    on Sep 15, 2026
  2. melvin-bot
    added
    WeeklyKSv2
    on Sep 15, 2026
  3. melvin-bot
    added a parent issue on Sep 15, 2026
  4. mountiny
    added
    Task
    on Sep 16, 2026
  5. mountiny
    added this to [#whatsnext] #qualityon Sep 16, 2026
  6. mountiny
    moved this to HIGH in [#whatsnext] #qualityon Sep 16, 2026

  7. roryabraham commented on Sep 18, 2026

    @roryabraham
    roryabraham
    on Sep 18, 2026
    Contributor

    Next step: awaiting PR from @collectioneur

  8. melvin-bot
    added
    ReviewingHas a PR in review
    on Sep 21, 2026

  9. Julesssss commented on Sep 30, 2026

    @Julesssss
    Julesssss
    on Sep 30, 2026
    Contributor

    Next step: awaiting PR from @collectioneur

    Deployed today.

  10. melvin-bot
    added
    DailyKSv2
    and removed
    ReviewingHas a PR in review
    WeeklyKSv2
    on Sep 30, 2026

  11. melvin-bot commented on Sep 30, 2026

    @melvin-bot
    melvin-botBot
    on Sep 30, 2026 – with Melvin Bot
    Author

    @justinpersaud / @roryabraham @collectioneur

    The solution for this issue has been 🚀 deployed to production 🚀 in version 9.4.99-2 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

  12. 27 remaining items

  13. justinpersaud commented on Oct 22, 2026

    @justinpersaud
    justinpersaud
    on Oct 22, 2026
    Contributor

    I marked it ready for review now and assigned you both to review

  14. roryabraham commented on Oct 22, 2026

    @roryabraham
    roryabraham
    on Oct 22, 2026
    Contributor

    PR merged. I was hoping to CP it to staging and test, but there were conflicts that I didn't feel like resolving. So I suppose we'll see if it worked on staging tomorrow!

  15. MelvinBot commented on Oct 24, 2026

    @MelvinBot
    MelvinBot
    on Oct 24, 2026
    Contributor

    Status after [No QA] Serve the web build's Brotli 11 twins from the Cloudflare Worker reached staging (9.5.20-0, tested 2026-10-23):

    The Worker serves the Brotli twins correctly, but Chrome, Edge and Firefox don't get them on staging. Those browsers send zstd in Accept-Encoding. Cloudflare then decodes the twin and re-encodes it as zstd, so main is 1.46 MB instead of 1.13 MB. That's about the same as production today (1.45 MB zstd).

    test-staging sends br for the same request. So a Cloudflare compression setting on the staging and production hostnames probably prefers zstd. Someone with access to that Cloudflare setting needs to check it. roryabraham couldn't find it with their access.

    main-e27519ace5234214.bundle.js on staging, by Accept-Encoding:

    Accept-Encoding Encoding Size
    gzip, deflate, br, zstd (Chrome) zstd 1.46 MB
    gzip, deflate, br (Safari) br 1.13 MB
    gzip gzip 1.60 MB
    identity none 5.66 MB
    Same file on test-staging, Chrome header br 1.13 MB
    Checks that pass on staging
    • The br body decodes to the same 5.66 MB file.
    • The CSS and service-worker.js are sent as br with a -br ETag.
    • Strong and weak -br ETags in If-None-Match return 304 with no body, for every Accept-Encoding tested.
    • /main-….bundle.js.br, .%62r and .BR return 404.
    • Navigating to a route ending in .br returns the app shell.
    • An older build's bundle loads from R2 with Cloudflare's own br. That's expected, since earlier deploys didn't upload the twins.

    I tested with curl only, not in a browser.

    justinpersaud will follow up on Monday.

  16. justinpersaud commented on Oct 26, 2026

    @justinpersaud
    justinpersaud
    on Oct 26, 2026
    Contributor

    @roryabraham I updated the compression rule if you want to try again

  17. roryabraham commented on Oct 26, 2026

    @roryabraham
    roryabraham
    on Oct 26, 2026
    Contributor

    @MelvinBot retest and see if it's now working as expected on staging

  18. MelvinBot commented on Oct 26, 2026

    @MelvinBot
    MelvinBot
    on Oct 26, 2026
    Contributor

    Yes, it works on staging now. Chrome, Edge and Firefox get the Brotli 11 twins. With the Chrome Accept-Encoding header, main is now 1.13 MB as br. On Friday it was 1.46 MB as zstd.

    Files on staging, requested with the Chrome header (gzip, deflate, br, zstd), tested 2026-10-26:

    File Encoding Size ETag
    main-9b0ae0deeaf48455.bundle.js br 1.13 MB -br
    vendors-ef86f26f1a53b40e.bundle.js br 1.44 MB -br
    heicTo-f009d6ccb336231a.bundle.js br 0.43 MB -br
    main.f7d14882f3.css br 2.2 KB -br
    Other checks
    • Safari's header (gzip, deflate, br) also gets the same 1.13 MB br body for main.
    • gzip alone gets 1.60 MB gzip, and identity gets the 5.66 MB original.
    • The decoded br body has the same SHA-256 hash as the original file.
    • If-None-Match with the -br ETag returns 304 with no body.
    • For comparison, production main gets Cloudflare's own br at 1.41 MB, with no -br ETag.
    • One minor change: a client that sends only zstd now gets the uncompressed file. Real browsers always also send gzip or br, so this doesn't affect them.

    I tested with curl only, not in a browser.

    view run

  19. melvin-bot
    added
    DailyKSv2
    and removed
    ReviewingHas a PR in review
    WeeklyKSv2
    on Oct 26, 2026

  20. melvin-bot commented on Oct 26, 2026

    @melvin-bot
    melvin-botBot
    on Oct 26, 2026 – with Melvin Bot
    Author

    @justinpersaud / @roryabraham @collectioneur

    The solution for this issue has been 🚀 deployed to production 🚀 in version 9.5.20-5 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

  21. roryabraham commented on Oct 26, 2026

    @roryabraham
    roryabraham
    on Oct 26, 2026
    Contributor

    @MelvinBot now that is on prod, check if it's working on production as well or @justinpersaud needs to make compression rule changes in prod as well

  22. MelvinBot commented on Oct 26, 2026

    @MelvinBot
    MelvinBot
    on Oct 26, 2026
    Contributor

    Yes, it works on production too. justinpersaud doesn't need to change any more compression rules. Chrome, Edge and Firefox now get the Brotli 11 files on new.expensify.com. With Chrome's Accept-Encoding header, main is 1.13 MB as br. Before the deploy, it was about 1.41 MB.

    Files on production (9.5.20-5), requested with Chrome's header (gzip, deflate, br, zstd), tested 2026-10-26:

    File Encoding Size ETag
    main-9b0ae0deeaf48455.bundle.js br 1.13 MB -br
    vendors-f654f236e5229163.bundle.js br 1.44 MB -br
    heicTo-f009d6ccb336231a.bundle.js br 0.43 MB -br
    main.f7d14882f3.css br 2.2 KB -br
    Other checks
    • Safari's header (gzip, deflate, br) gets the same 1.13 MB br file for main.
    • gzip alone gets a 1.60 MB gzip file.
    • The decoded br file is byte-for-byte the same as on staging.
    • If-None-Match with the -br ETag returns 304 with no body.

    I tested with curl only, not in a browser.

    view run

  23. roryabraham commented on Oct 26, 2026

    @roryabraham
    roryabraham
    on Oct 26, 2026
    Contributor

    Closing since this is complete with no payments due

  24. roryabraham
    closed this as completedon Oct 26, 2026
  25. github-project-automation
    moved this from HIGH to Done in [#whatsnext] #qualityon Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

DailyKSv2Task

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions