Set JemallocAllocatorOptions::num_arenas=8

Upstream commit ID: facebook/mysql-5.6@f902321
PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951)

Summary:
RocksDB's allocator for omitting block cache data from core dumps (`NewJemallocNodumpAllocator()`) recently added support for sharding allocation requests across multiple arenas (facebook/rocksdb#11400). Sharding can fix high contention on per-arena locks when thread caching is avoided for RocksDB blocks (e.g., when using MALLOC_CONF='tcache_max:4096' and rocksdb_block_size=16384). Sharding increases fragmentation, however.

This diff hardcodes MyRocks to use eight shards when `--rocksdb_cache_dump=false`. The test plan shows the impact of this change on a workload that triggers block cache misses as fast as possible. We might consider trying more values besides one and eight, or make it configurable in the future.

Differential Revision: D46786850

fbshipit-source-id: 56878aa14a4e9d95697a8af2dfa29b9d96ef6c6c

storage/rocksdb/ha_rocksdb.cc

@@ -6702,6 +6702,12 @@ static int rocksdb_init_internal(void *const p) {
       alloc_opt.limit_tcache_size = true;
       alloc_opt.tcache_size_lower_bound = block_size / 4;
       alloc_opt.tcache_size_upper_bound = block_size;
+      // MyRocks commonly runs with reduced `tcache_max`, which prevents block
+      // allocations from using `tcache`, despite the generous bounds set just
+      // above. In this case jemalloc arena contention can be a bottleneck.
+      // Using multiple arenas alleviates the bottleneck with the downside of
+      // increasing fragmentation.
+      alloc_opt.num_arenas = 8;
       rocksdb::Status new_alloc_status =
           rocksdb::NewJemallocNodumpAllocator(alloc_opt, &memory_allocator);
       if (!new_alloc_status.ok()) {