Skip to content

Commit

Permalink
fix(repair) add jitter to handle edge case where all the following ar…
Browse files Browse the repository at this point in the history
…e true:

- we're caught at least 1024 slots behind or turbine isn't working
- the current bottom slot is skipped
- the next slot is skipped
- the slot 25 slots ahead is also skipped

with jitter, we only run into a problem if there are also 40 consecutive slots that are skipped, which is practically impossible
  • Loading branch information
dnut committed Jan 15, 2025
1 parent 6c90312 commit c8546b8
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions src/shred_network/repair_service.zig
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ pub const RepairService = struct {
report: MultiSlotReport,
thread_pool: RequestBatchThreadPool,
metrics: Metrics,
rng: std.Random.DefaultPrng,

pub const RequestBatchThreadPool = HomogeneousThreadPool(struct {
requester: *RepairRequester,
Expand Down Expand Up @@ -109,6 +110,7 @@ pub const RepairService = struct {
.report = MultiSlotReport.init(allocator),
.thread_pool = RequestBatchThreadPool.init(allocator, maxRequesterThreads()),
.metrics = try registry.initStruct(Metrics),
.rng = std.Random.DefaultPrng.init(0),
};
}

Expand Down Expand Up @@ -222,8 +224,10 @@ pub const RepairService = struct {

// eagerly request the next unknown slot in case turbine is laggy
try repairs.append(.{ .HighestShred = .{ slot + 1, 0 } });
// request 10 seconds ahead to detect if caught behind
try repairs.append(.{ .HighestShred = .{ slot + 25, 0 } });

// request ahead to detect if caught behind. use jitter to avoid skipped slots
const num_slots_ahead = self.rng.random().intRangeAtMost(u32, 10, 50);
try repairs.append(.{ .HighestShred = .{ slot + num_slots_ahead, 0 } });

self.metrics.oldest_slot_needing_repair.set(oldest_slot_needing_repair);
self.metrics.newest_slot_needing_repair.set(newest_slot_needing_repair);
Expand Down

0 comments on commit c8546b8

Please sign in to comment.