support array of links

[Tony Przygienda]

it is incredibly difficult to write generics code over the current adapters since if an object is linked on multiple queues I may want to write generics saying "manipulate queue[x] link" and now it's almost impossible to express with generics basically collapsing due to all the lifetime constrains on cursors and elements

it would be much simpler if we have array of the same adapter, e.g. LinkList and then I can pass reference to the adapter by using the index into the array where the function interanlly can index it on the structure rather than trying to write a generic over any of those adapters

[Amanieu d'Antras]

Could you show an example of what you are trying to achieve and your proposed solution? I'm having trouble picturing it without some code examples.

[Cormac Relf]

A more standard use case is building a skip list. Each node has a tower of links, up to a (typically constant bounded) random height. You can make an adapter for links at a specific index across all nodes, by expanding the macro code for intrusive_adapter, adding a height (not max height) field to the adapter struct, and using that height to calculate first the offsets within the skip link struct, then the skip link struct within the node array, then the array within the node. To pick one linked list, you initialise it with MyAdapter::new(h), and then it walks your nodes using an offset based on h, threading a list through links at the same height in each.

(And, of course, IRL you would actually build a tower of cursors to navigate through it efficiently using the skip distances.)

Gist for the adapter: https://gist.github.com/cormacrelf/eb54684ab717ad104c4c3f1ddaa2e624

[Cormac Relf]

Here's my report on using intrusive linked lists to implement a skip list. I got it to work with some changes.

You can't easily build a skip list using intrusive_collections::LinkedList as-is because there is no way to pivot a node pointer from one list to another. To efficiently navigate and mutate a skip list you need to take a cursor at some height, and get another cursor at the same node but for the list threaded through at a different height. Equivalently, you could take a cursor at some height, and "advance" it by setting the current node O(1). That advance function can't just be "move_next until current == target" because then navigating the skip list would be slower than just using one linked list.

This can't be done as it stands, because:

• Obviously there's no built-in unsafe fn advance_to function to change the current node of a cursor
• You can get the PointerOps::Pointer out through CursorMut::as_cursor(...).clone_pointer(), so that's a start
• You can't, however, get the &'a mut LinkedList out of the CursorMut. You need this to construct a new cursor on the same list with a different current pointer via LinkedList::cursor_mut_from_ptr.

I think unsafe fn advance_to would be a great addition. Bikeshed the name if you like. I suppose ideally it'd work on *const PointerOps::Value so users of Rc don't need to rehydrate and dehydrate / bump strong pointers and the like (unless that's not how the pointerops impl works). I've been surprised before at the microsecond-scale performance hit that doing this excessively can cause. It would form a consistent API where the unsafe methods assuming that a node is already linked in a particular list use *const Value, whereas PointerOps::Pointer is for nodes being inserted. I'm also sensing that the PointerOps::Pointer APIs have ownership / ref count implications whereas *const Value does not.

Here are the main search routines of a working skip list implementation.

// Additions to intrusive_collections::linked_list
impl<'a, A: Adapter> CursorMut<'a, A> {
    #[inline]
    pub fn current_as_ptr(&self) -> Option<*const <A::PointerOps as PointerOps>::Value> {
        Some(unsafe { self.list.adapter.get_value(self.current?) })
    }
    #[inline]
    pub unsafe fn advance_to(&mut self, node: Option<*const <A::PointerOps as PointerOps>::Value>) {
        unsafe {
            self.current = node.map(|n| self.list.adapter.get_link(n));
        }
    }
}

// Skip list code (SkipAdapter is defined roughly as in the gist above)
const MAX_HEIGHT: usize = 10;
const TOWER_SIZE: usize = MAX_HEIGHT + 1;

struct SkipList {
    lists: [LinkedList<SkipAdapter>; TOWER_SIZE],
    rng: super::RopeRng,
}

struct Node {
    value: RefCell<String>,
    height: Cell<u8>,
    skip_links: [SkipLink; TOWER_SIZE],
}

// Note that in this skip list, each link has a usize width.
// The use case is more like a rope (big log-time-insert string) than an equivalent to BTreeMap.
// In a skip-list-based map, you would just use the linked node's key instead of an explicit width.
struct SkipLink {
    link: LinkedListLink,
    skip_width: Cell<usize>,
}

struct TowerCursorMut<'a> {
    inner: [SubCursor<'a>; TOWER_SIZE],
    rng: &'a mut super::RopeRng,
}

struct SubCursor<'a> {
    cursor: CursorMut<'a, SkipAdapter>,
    position: Cell<usize>,
}

impl SkipList {
    // fn new(width) -> Self { ... init all lists with a single node ... }

    fn cursor_mut(&mut self, offset: usize, stick_end: bool) -> TowerCursorMut {
        let lists = &mut self.lists;
        const UNINIT: MaybeUninit<SubCursor> = MaybeUninit::uninit();
        let mut tower = [UNINIT; TOWER_SIZE];
        for (height, list) in lists.iter_mut().enumerate() {
            let mut cursor = list.cursor_mut();
            cursor.move_next();
            assert!(!cursor.is_null());
            tower[height].write(SubCursor {
                cursor,
                position: 0.into(),
            });
        }
        let mut cursor = TowerCursorMut {
            inner: unsafe { tower.map(|sub| sub.assume_init()) },
            rng: &mut self.rng,
        };
        cursor.seek(offset, stick_end);
        cursor
    }
}

impl TowerCursorMut<'_> {
    fn seek(&mut self, mut offset: usize, stick_end: bool) {
        let mut height = self.head_height();
        let mut pivot = self.inner[MAX_HEIGHT].cursor.current_as_ptr();
        while height > 0 {
            height -= 1;
            let i = height;
            let c = &mut self.inner[I];
            // Safety: invariant: nodes are always linked up to their own height.
            // The pivot is set by a node found via move_next on a cursor at height+1,
            // this cursor is at height. Therefore pivot is linked in this list too.
            unsafe {
                c.cursor.advance_to(pivot);
            }
            loop {
                let next = c.cursor.get().unwrap();
                let next_link = &next.skip_links[i];
                let skip = next_link.skip_width.get();
                let is_linked = next_link.link.is_linked();
                if offset > skip || (!stick_end && offset == skip && is_linked) {
                    offset -= skip;
                    c.cursor.move_next();
                    if c.cursor.get().is_none() {
                        panic!("reached the end of the list");
                    }
                } else {
                    c.position.set(offset);
                    pivot = c.cursor.current_as_ptr();
                    break;
                }
            }
        }
    }
}