[medjed02]

Summary:

We are moving a production profile cache with batched reads (about 32 keys per request) to CacheLib. Calling find() per key makes each lookup wait on its own DRAM misses: the bucket slot, the chain head and the item.

findMany(keys, handles) gives the same handles, stats and promotion as calling find() per key, but overlaps the cache misses of different keys. It walks the batch in groups of 8: the bucket slots of the next group are prefetched while the chain heads of the current group are prefetched and looked up. Stats and the clock are updated once per batch. DRAM misses go to NVM as in find(); those lookups are issued without waiting on each other. The keys must stay valid during the call.

The chain head is read from the slot without the bucket lock (4-byte compressed pointers only, annotated for TSAN), so it may be stale. It goes to a new PtrCompressor::prefetch(), which, unlike unCompress(), does not validate the pointer and skips offsets outside the slab. The handles that findMany() overwrites are released after the bucket lock is dropped, since releasing them can run item callbacks.

Happy to change the API or split this into smaller PRs.

Test Plan:

• FindManyTest: handles, refcounts, expiry and stats match find(); lookups race with inserts and removes; a remove callback uses the cache while findMany() overwrites the last handle to a removed item (this deadlocks if the old handle is released under the bucket lock).
• NvmCacheTest.FindMany: a DRAM hit, an NVM hit and a miss in one batch.
• MemoryAllocatorTest.PrefetchStale: prefetch() of pointers into an uncarved slab and past the end of a slab; unCompress() rejects these in debug builds.
• Full ctest passes in a Release build; FindManyTest and MemoryAllocatorTest also pass in a Debug build.
• cachelib/benchmarks/FindManyBench.cpp, 40M items × 500 B, Zipf 0.99, batches of 32, 16 threads on AMD EPYC 7713: findMany() uses about 34% less CPU per key than find() (558 vs 846 ns/key, faster in 3/3 blocks). Groups of 4 to 8 keys are a plateau; 16, 32 and a per-key pipeline are 2–10% slower (1M and 40M items, 1 and 16 threads).

[meta-codesync]

@AlnisM has imported this pull request. If you are a Meta employee, you can view this in D131231523.