perf(@angular/build): implement semaphore backpressure throttling in JavaScriptTransformer - #33371
Conversation
3622686 to
cb9a0cf
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces a semaphore-based backpressure throttle mechanism (#runWithThrottle) to limit the concurrency of JavaScript transformation tasks in JavaScriptTransformer. While this is a great addition to prevent resource saturation, there are a couple of critical issues identified in the review. First, if maxThreads is 0, the throttle will deadlock immediately; we should ensure maxConcurrent is at least 1. Second, if the transformer is closed while tasks are pending, those tasks will hang indefinitely. To resolve this, the pending tasks queue should store both resolve and reject callbacks so they can be properly rejected and cleaned up upon closure.
…JavaScriptTransformer Throttle active esbuild transformation requests higher in the pipeline using an asynchronous semaphore queue bounded to maxThreads * 2. In large monorepo builds (thousands of files), esbuild crawls the import graph via parallel goroutines much faster than Node.js workers can process downleveling and linking. Unthrottled onLoad calls flood libuv's file read pool and accumulate thousands of source buffers in Piscina's task queue. Throttling active requests higher in the chain keeps libuv's I/O pool free, caps Buffer memory overhead at ~10MB–30MB, and ensures file buffers remain short-lived for quicker GC reclamation.
cb9a0cf to
c57a542
Compare
|
This pull request has been automatically locked due to inactivity. Read more about our automatic conversation locking policy. This action has been performed automatically by a bot. |
Throttle active esbuild transformation requests higher in the pipeline using an asynchronous semaphore queue bounded to maxThreads * 2.
In large monorepo builds (thousands of files), esbuild crawls the import graph via parallel goroutines much faster than Node.js workers can process downleveling and linking. Unthrottled onLoad calls flood libuv's file read pool and accumulate thousands of source buffers in Piscina's task queue.
Throttling active requests higher in the chain keeps libuv's I/O pool free, caps Buffer memory overhead at ~10MB–30MB, and ensures file buffers remain short-lived for quicker GC reclamation.