diff --git a/.jules/bolt.md b/.jules/bolt.md new file mode 100644 index 0000000..4b79790 --- /dev/null +++ b/.jules/bolt.md @@ -0,0 +1,3 @@ +## 2024-06-18 - Sliding Window withTaskGroup over Static Chunking +**Learning:** In Swift structured concurrency, processing high-volume tasks using `withTaskGroup` with static chunking (e.g. creating chunks of array elements) limits throughput due to tail latency. The execution pauses while waiting for the slowest task in the current chunk before starting the next chunk. +**Action:** Use a sliding window approach with an iterator instead of static chunking. Populate the `withTaskGroup` with the initial `maxConcurrency` tasks, then continuously loop over `group` results with `for await`, adding a new task from the iterator each time one completes to maintain maximum parallel execution. diff --git a/Sources/Cacheout/Memory/ProcessMemoryScanner.swift b/Sources/Cacheout/Memory/ProcessMemoryScanner.swift index 3f8e728..ea77da6 100644 --- a/Sources/Cacheout/Memory/ProcessMemoryScanner.swift +++ b/Sources/Cacheout/Memory/ProcessMemoryScanner.swift @@ -97,29 +97,35 @@ actor ProcessMemoryScanner { /// /// Returns the collected entries and the count of EPERM failures. private func scanPIDs(_ pids: [pid_t]) async -> (entries: [ProcessEntryDTO], epermCount: Int) { - // Chunk PIDs to cap concurrency at maxConcurrency. - let chunks = stride(from: 0, to: pids.count, by: maxConcurrency).map { - Array(pids[$0..