slab + intrusive collections

[Alkis Evlogimenos]

I am trying to use a slab instead of arena and use intrusive collections to the elements in the slab. But I can't seem to make this work without polluting the top level type with a lifetime.

Sample code:

struct Item {
  link: RBTreeLink,
  val: u64,
}

intrusive_adapter!(ItemAdapter<'a> = &'a Item: Item { link: RBTreeLink });
impl<'a> KeyAdapter<'a> for ItemAdapter<'a> {
  type Key = u64;
  fn get_key(&self, i: &'a Item) -> u64 { -i.val }
}

struct TopLevel {
  slab: Slab<Item>,  // storage for items
  ids: HashMap<u64, usize>,  // id to item map (through its index in the slab)
  ord: RBTree<ItemAdapter<'a>>,  // ordered items: what lifetime to put here?!
}

If I add lifetime to TopLevel then it will need to be specified whenever TopLevel is a member of another struct.

Is what I am trying to do possible? If so how?

[Amanieu d'Antras]

There's no way to do it without a lifetime on TopLevel. Here is what it would look like:

struct TopLevel<'a> {
  slab: &'a Slab<Item>,
  ids: HashMap<u64, usize>,
  ord: RBTree<ItemAdapter<'a>>,
}

let slab = Slab::new(...);
let top_level = TopLevel {
  slab: &slab,
  ids: ...,
  ord: ...,
};

By the way, this is the same pattern that is used in the Rust compiler. The compiler crates a GlobalContext struct with a lifetime which is then passed on to almost every other function in the compiler.

[Alkis Evlogimenos]

I see. Having the slab outside is a lot better! Thanks @Amanieu!

[Alkis Evlogimenos]

One more question. I tried to implement a stateful adapter such that I stop dealing with references and deal with usize instead which I would use to index into the slab. I didn't manage to make it work. Is there an example where this is done?

[Amanieu d'Antras]

It would look something like this:

struct MyAdapter<'a>(&'a Slab<Item>);
impl<'a> Adapter for MyAdapter<'a> {
    type Link = RBTreeLink;
    type Value = Item;
    type Pointer = usize;
    #[inline]
    unsafe fn get_value(&self, link: *const RBTreeLink) -> usize {
        let ptr = container_of!(link, Item, link);
        // ptr is a pointer to an element in the slab, convert it to an index somehow
    }
    #[inline]
    unsafe fn get_link(&self, value: usize) -> *const RBTreeLink {
        &self.0.get_from_index(value).link
    }
}

[Alkis Evlogimenos]

Thanks Amanieu. I went as far as that and got stuck at the comment. I opened an issue against slab, maybe this API is possible to implement: tokio-rs/slab#45

[Alkis Evlogimenos]

I have a working implementation now except for one limitation: when composing data structures together, the inner nodes cannot be modified. For example:

struct Bucket {
  link: RBTreeLink,
  list: LinkedList,
  ...
}

struct Item {
  link: LinkedListLink,
  ...
}

I want to be able to get a Bucket from an RBTree and then add Items to its linked list. Is it possible to add a get_mut to CursorMut for each container? I understand that this is unsafe since the user can break the data structure's invariants by getting a mutable reference to the item but I don't see another way. What do you think?

[Amanieu d'Antras]

You're supposed to use inner mutability here. Basically, put the Bucket.list in a RefCell (or UnsafeCell).

[Alkis Evlogimenos]

Thanks @Amanieu. Closing.