Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref(proguard): Properly size the in-memory cache #1424

Merged
merged 2 commits into from
Apr 4, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions crates/symbolicator-proguard/src/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,10 @@ impl ProguardService {
download_svc: Arc::clone(&self.download_svc),
};

self.cache.compute_memoized(request, cache_key).await
self.cache
.compute_memoized(request, cache_key)
.await
.map(|item| item.1)
}

/// Downloads a source bundle for the given scope and debug id.
Expand Down Expand Up @@ -141,7 +144,9 @@ pub struct FetchProguard {
}

impl CacheItemRequest for FetchProguard {
type Item = ProguardMapper;
/// The first component is the estimated memory footprint of the mapper,
/// computed as 2x the size of the mapping file on disk.
type Item = (u32, ProguardMapper);

const VERSIONS: CacheVersions = PROGUARD_CACHE_VERSIONS;

Expand All @@ -168,10 +173,18 @@ impl CacheItemRequest for FetchProguard {
}

fn load(&self, byteview: ByteView<'static>) -> CacheEntry<Self::Item> {
Ok(Self::Item::new(byteview))
let weight = byteview.len().try_into().unwrap_or(u32::MAX);
// NOTE: In an extremely unscientific test, the proguard mapper was slightly less
// than twice as big in memory as the file on disk.
let weight = weight.saturating_mul(2);
Ok((weight, ProguardMapper::new(byteview)))
}

fn use_shared_cache(&self) -> bool {
false
}

fn weight(item: &Self::Item) -> u32 {
item.0.max(std::mem::size_of::<Self::Item>() as u32)
}
}
6 changes: 5 additions & 1 deletion crates/symbolicator-service/src/caching/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ pub struct Caches {
pub sourcefiles: Cache,
/// Store for minidump data symbolicator failed to process, for diagnostics purposes
pub diagnostics: Cache,
/// Proguard mapping files.
pub proguard: Cache,
}

Expand Down Expand Up @@ -294,7 +295,10 @@ impl Caches {
config,
config.caches.downloaded.into(),
max_lazy_redownloads,
default_cap,
// We use 2GiB as the in-memory cache size for Proguard files.
// Note that a Proguard mapper can take up hundreds of MB
// in memory.
2 * 1024 * 1024 * 1024,
loewenheim marked this conversation as resolved.
Show resolved Hide resolved
)?,
})
}
Expand Down
Loading