CI: Tune the ccache size

[reneme]

Notes

A clean build of the coverage job (that builds most, if not all the things) currently results in a cache of 250MiB. Namely 600 * 0.42. The entire build/test job ran 36m 34s on a cold cache. And 29m 19s on the warm cache (after adding a commit that changes some non-C++ file).

First round, cold cache:

ccache --show-stats
  Cacheable calls:   894 / 898 (99.55%)
    Hits:             14 / 894 ( 1.57%)
      Direct:          0 /  14 ( 0.00%)
      Preprocessed:   14 /  14 (100.0%)
    Misses:          880 / 894 (98.43%)
  Uncacheable calls:   4 / 898 ( 0.45%)
  Local storage:
    Cache size (GB): 0.3 / 0.6 (42.04%)
    Hits:             14 / 894 ( 1.57%)
    Misses:          880 / 894 (98.43%)

Second round, warm cache (after building jitterentropy and ESDM):

ccache --show-stats
  Cacheable calls:   1055 / 1061 (99.43%)
    Hits:             175 / 1055 (16.59%)
      Direct:         161 /  175 (92.00%)
      Preprocessed:    14 /  175 ( 8.00%)
    Misses:           880 / 1055 (83.41%)
  Uncacheable calls:    6 / 1061 ( 0.[57](https://github.com/randombit/botan/actions/runs/12866174366/job/35868214050#step:6:57)%)
  Local storage:
    Cache size (GB):  0.3 /  0.6 (42.04%)
    Hits:             175 / 1055 (16.[59](https://github.com/randombit/botan/actions/runs/12866174366/job/35868214050#step:6:59)%)
    Misses:           880 / 1055 (83.41%)

... dependency builds on warm cache produced 161 hits and not a single new miss. That tallys and should prove that the dependency builds benefit from ccache.

Second round, warm cache (after build botan)

ccache --show-stats
  Cacheable calls:   1788 / 1796 (99.55%)
    Hits:             840 / 1788 (46.98%)
      Direct:         826 /  840 (98.33%)
      Preprocessed:    14 /  840 ( 1.67%)
    Misses:           948 / 1788 (53.02%)
  Uncacheable calls:    8 / 1796 ( 0.45%)
  Local storage:
    Cache size (GB):  0.3 /  0.6 (47.68%)
    Hits:             840 / 1788 (46.98%)
    Misses:           948 / 1788 (53.02%)

... another 665 new cache hits and 68 misses. I didn't expect so many misses, frankly. Perhaps we're including build.h (containing BOTAN_VERSION_VC_REVISION) too often?
Cache size stayed below 300MB on the second run (600MB * 0.48 = 288MB).

[randombit]

Something else to explore in this area is using sccache on all platforms and using their GHA backend. This (AIUI) stores each individual cached compilation as an entry in the GHA cache. We could then eliminate the need for cache keys/etc, which would be helpful as in fact many of our builds do compile the same file, with the same flags and same compiler. Also sccache will do the cache updates as compilation occurs - so the cache is updated even if the build fails, and concurrent CI jobs that share the same inputs can make use of the cache. Finally evictions are less catastrophic since they evict a single file's cache rather than an entire job.

The main problem I've heard about with this is that GHA cache is really slow to access. Some people were seeing 3-4 seconds per read... maybe things have gotten better recently.

Some background here https://xuanwo.io/en-us/reports/2023-04/

[reneme]

Something else to explore in this area is using sccache on all platforms and using their GHA backend.

Oh, that sounds interesting. But for now, I'd just like to get a baseline for what we currently have. I'm going to re-run this (with warm caches) and see what the hit rates look like. Also to get an idea whether the ccache usage for building jitterentropy and ESDM actually work properly.