Add unified cache

[NikitaEvs]

Summary

PoC of the unified cache model with new/delete integration, custom Allocator support and Uncompressed cache and Marks cache integrations.

Features

• Allocations which go through the new/delete methods or custom BuddyAllocator instances can trigger cache entries eviction from the Uncompressed cache and Marks cache
• All allocations discussed above use the global memory arena that is initialized on the startup
• The trigger for evictions is set up using a threshold of the consumed memory

Main known problems

• Memory tracker isn't working with this schema
• There are async evictions so there is no guarantee that memory will be released instantly in case of a memory shortage
• Majority of allocations are done through custom Allocator instances (for example PODArray use Allocator class), so it is needed to change Allocator instances to BuddyAllocator to make these structures use BuddyArena
• Buddy allocator schema isn't very efficient
• Potential high contention on the global cache policy and global allocator state
• Some allocations can be done aside from BuddyArena (e.g. static constructors cannot use BuddyArena)

Implementation details

The high-level architecture of the PoC:

More specific diagram:

You can find components from the picture above in the following places:

• Global allocator state - BuddyArena class
• Global cache policy - LRUUnifiedCacheGlobal class
• Custom Allocator instance - BuddyAllocator class
• new/delete integration (and other places in the same file)
• MarksCache integration
• UncompressedCache integration

Test plan

The main components of the allocator were tested using separate unit tests, WIP.

Benchmarks

WIP

[rschu1ze]

(could be written more succinctly using the KiB/MiB/GiB suffix literals in base/base/units.h)

[rschu1ze]

(not urgent but it would be nice if we could ask the OS for the total available memory here and then allocate a percentual fraction of it, e.g. 90%)

[rschu1ze]

I guess l. 659, l. 660 and l. 662 should be moved into the (private) constructor of the buddy arena (which get's called once when BuddyArena::instance() is first called)

[NikitaEvs]

Should we move the allocator arena parameters inside the private constructor? We can otherwise configure it using settings

[rschu1ze]

For the final version, it would be nice to add a comment that explains the layout of the meta/directory structure.

[rschu1ze]

I am not sure if I understand the reason for below code in this method.

[NikitaEvs]

Added more detailed comments, will add more in the final version

[rschu1ze]

In case the locking becomes a bottleneck, you could try our new futex-based lock implementation (ClickHouse#44924). Or, alternatively, stride the lock, i.e. introduce multiple locks (e.g. one per equally-large range of the lowest level of the buddy allocator)

[serxa]

We do not have a faster std::mutex reimplementation yet, only faster std::shared_mutex (i.e. DB::SharedMutex). And I guess we won't get better performance than ordinary std::mutex in the same way.

[rschu1ze]

I remember our discussion that the memory tracker doesn't know about allocations at runtime using the buddy allocator (i.e. it just sees the initial allocation of the buddy allocator).

You added instrumentation for memory tracking here. Does that solve the issue?

[NikitaEvs]

For now we have a hack with CurrentMemoryTracker::free right after the initial arena allocation. It should add support of memory tracker instrumentation for the project