Skip to content

Commit

Permalink
doc(sdk): Add moooooar documentation.
Browse files Browse the repository at this point in the history
This patch adds documentation that explains how `AsVectorSubscriber`
works.
  • Loading branch information
Hywan committed May 15, 2024
1 parent c6dadd2 commit bea77ed
Show file tree
Hide file tree
Showing 2 changed files with 78 additions and 7 deletions.
58 changes: 55 additions & 3 deletions crates/matrix-sdk/src/event_cache/linked_chunk/as_vector.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,61 @@ use super::{
type ChunkLength = usize;

pin_project! {
/// A type used to transform a `Stream<Item = Vec<Update<Item, Gap>>>` into
/// a `Stream<Item = Vec<VectorDiff<Item>>>`. Basically, it helps to consume
// a [`LinkedChunk<CAP, Item, Gap>`] as if it was an [`ObservableVector<Item>`].
/// A type that transforms a `Stream<Item = Vec<Update<Item, Gap>>>` —given by
/// [`UpdateSubscriber`]— into a `Stream<Item = Vec<VectorDiff<Item>>>` —this
/// type—. Basically, it helps to consume a [`LinkedChunk<CAP, Item, Gap>`] as
/// if it was an [`eyeball::ObservableVector<Item>`].
///
/// How this type transforms `Update` into `VectorDiff`? There is no internal
/// buffer of kind [`eyeball_im::ObservableVector<Item>`], which could have been
/// used to generate the `VectorDiff`s. They are computed manually.
///
/// The only buffered data is pairs of [`ChunkIdentifier`] and [`ChunkLength`].
/// The following rules must be respected:
///
/// * A chunk of kind [`ChunkContent::Gap`] has a length of 0,
/// * A chunk of kind [`ChunkContent::Items`] has a length equals to its number
/// of items,
/// * The pairs must be ordered exactly like the chunks in [`LinkedChunk`], i.e.
/// the first pair must represent the first chunk, the last pair must
/// represent the last chunk.
///
/// The only thing this algorithm does is maintaining the pairs:
///
/// * [`Update::NewItemsChunk`] and [`Update::NewGapChunk`] are inserting a new
/// pair with a chunk length of 0 at the appropriate index,
/// * [`Update::RemoveChunk`] is removing a pair,
/// * [`Update::PushItems`] is increasing the length of the appropriate pair by
/// the number of new items, and is potentially emitting [`VectorDiff`],
/// * [`Update::DetachLastItems`] is decreasing the length of the appropriate pair
/// by the number of items to be detached; no [`VectorDiff`] is emitted,
/// * [`Update::ReattachItems`] and [`Update::ReattachItemsDone`] are
/// respectively muting or unmuting the emission of [`VectorDiff`] by
/// [`Update::PushItems`].
///
/// The only `VectorDiff` that are emitted are [`VectorDiff::Insert`] or
/// [`VectorDiff::Append`] because a [`LinkedChunk`] is append-only.
///
/// `VectorDiff::Append` is an optimisation when numerous `VectorDiff::Insert`s
/// have to be emitted at the last position.
///
/// `VectorDiff::Insert` need an index. To compute this index, the algorithm
/// will iterate over all pairs to accumulate each chunk length until it finds
/// the appropriate pair (given by [`Update::PushItems::position_hint`]). This
/// is _the offset_. To this offset, the algorithm adds the position's index of
/// the new items (still given by [`Update::PushItems::position_hint`]). This is
/// _the index_. This logic works for all cases as long as pairs are maintained
/// according to the rules hereinabove.
///
/// That's a pretty memory compact and computation efficient way to map a
/// `Stream<Item = Vec<Update<Item, Gap>>>` into a `Stream<Item =
/// Vec<VectorDiff<Item>>>`. The larger the `LinkedChunk` capacity is, the fewer
/// pairs the algorithm will have to handle, e.g. for 1'000 items and a
/// `LinkedChunk` capacity of 128, it's only 8 pairs, be 256 bytes.
///
/// [`LinkedChunk`]: super::LinkedChunk
/// [`ChunkContent::Gap`]: super::ChunkContent::Gap
/// [`ChunkContent::Content`]: super::ChunkContent::Content
pub struct AsVectorSubscriber<Item, Gap> {
// The inner `UpdatesSubscriber`.
#[pin]
Expand Down
27 changes: 23 additions & 4 deletions crates/matrix-sdk/src/event_cache/linked_chunk/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

#![allow(dead_code)]

//! A linked chunk is the underlying data structure that holds all events.
/// A macro to test the items and the gap of a `LinkedChunk`.
/// A chunk is delimited by `[` and `]`. An item chunk has the form `[a, b,
/// c]` where `a`, `b` and `c` are items. A gap chunk has the form `[-]`.
Expand Down Expand Up @@ -107,17 +109,33 @@ use updates::*;
/// Errors of [`LinkedChunk`].
#[derive(thiserror::Error, Debug)]
pub enum Error {
/// A chunk identifier is invalid.
#[error("The chunk identifier is invalid: `{identifier:?}`")]
InvalidChunkIdentifier { identifier: ChunkIdentifier },
InvalidChunkIdentifier {
/// The chunk identifier.
identifier: ChunkIdentifier,
},

/// A chunk is a gap, and it was expected to be an items.
#[error("The chunk is a gap: `{identifier:?}`")]
ChunkIsAGap { identifier: ChunkIdentifier },
ChunkIsAGap {
/// The chunk identifier.
identifier: ChunkIdentifier,
},

/// A chunk is an items, and it was expected to be a gap.
#[error("The chunk is an item: `{identifier:?}`")]
ChunkIsItems { identifier: ChunkIdentifier },
ChunkIsItems {
/// The chunk identifier.
identifier: ChunkIdentifier,
},

/// An item index is invalid.
#[error("The item index is invalid: `{index}`")]
InvalidItemIndex { index: usize },
InvalidItemIndex {
/// The index.
index: usize,
},
}

/// Links of a `LinkedChunk`, i.e. the first and last [`Chunk`].
Expand Down Expand Up @@ -719,6 +737,7 @@ impl<const CAP: usize, Item, Gap> LinkedChunk<CAP, Item, Gap> {
self.updates.as_mut()
}

/// Get a `Stream<Item = Vec<VectorDiff<Item>>>` of this `LinkedChunk`.
pub fn subscribe_as_vector(&mut self) -> Option<AsVectorSubscriber<Item, Gap>> {
let mut initial_chunk_lengths = VecDeque::new();

Expand Down

0 comments on commit bea77ed

Please sign in to comment.