tvrg / stream-kmerge

K-way merge for rust streams

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Fantastic library! A design question - Why not store the stream separately and index them in the heap element?

twitu opened this issue · comments

Thanks for this fantastic generic implementation of a stream kmerge. It is almost exactly what I was looking for to solve a specific problem. Although because of lifetimes issues, I couldn't use it as is. But because it's well written I could modify relevant portions to make it work for me.

I have question regarding a key design choice. Firstly for any stream kmerge, the heap element needs to track the stream from which an element came from. That way when the element is popped that stream can be called to get the next element and it can be put back in the heap immediately. This is almost fundamental to this whole approach.

To that end your heap element directly stores the stream.

pub struct HeadTail<S>
where
    S: Stream,
{
    head: S::Item,
    tail: S,
}

My only question is how did you decide between this and storing the stream separately say in a Vec and storing the index in the heap element. Something like this.

pub struct HeadTail<T>
{
    head: T,
    stream_index: usize,
}

pub struct KWayMergeBy<S, C>
where
    S: Stream,
    S: Unpin,
    C: Compare<HeadTail<S>>
{
    initial: Option<JoinAll<StreamFuture<S>>>,
    next: Option<S>,
    stream_store: Vec<S>,
    heap: BinaryHeap<HeadTail<S::Item>, C>,
}