Support for `allocator_api` feature and `allocator-api2` crate

[Folke Behrens]

Hi, would you be interested in adding support for the allocator_api feature (for nightly) and the allocator-api2 crate (for stable)? I'm using a-a2 for my AST types and wanted to fuzzy test a code generator, but there doesn't seem to be a way to inject an allocator into Arbitrary::arbitrary().

I'm thinking Unstructured could provide an allocator() method that returns the allocator passed to Unstructured::with_allocator(data, my_allocator). Problem is that this requires Unstructured to be generic. I don't know if it's enough to set Global as default value to not break the API.

Instead of supporting a custom allocator directly Unstructured could expose some generic extra state that could contain the allocator. That's more flexible.

[Folke Behrens]

I made some progress on other parts of my project and realized that the (current) allocator API is too "invasive" may not be worth the hassle, and because String isn't even covered yet I will continue without using an arena allocator.

[Nick Fitzgerald]

To test an allocator with arbitrary, I recommend doing something like this:

struct Size(usize);

impl<'a> Arbitrary<'a> for Size {
    // clamp as necessary...
}

struct Align(usize);

impl<'a> Arbitrary<'a> for Align {
    // round to pow 2 and clamp as necessary...
}

#[derive(Arbitrary)]
enum Op {
    Alloc { size: Size, align: Align },
    Dealloc { allocation: usize },
    Realloc { allocation: usize, size: Size, align: Align },
    // ...
}

fn test_allocator(allocator: &impl Allocator, ops: &[Op]) {
    let mut allocs = vec![];

    for op in ops {
        match op {
            Op::Alloc { size, align } => {
                let layout = Layout::new(size, align);
                let ptr = allocator.allocate(layout);
                allocs.push((layout, ptr));
            }
            Op::Dealloc { allocation } => {
                if allocs.is_empty() {
                    continue;
                }
                let i = allocation % allocs.len();
                let (layout, ptr) = allocs.swap_remove(i).unwrap();
                allocator.deallocate(ptr, layout);
            }
            // ...
        }
    }

    // Free any remaining allocations in allocs...
}

libfuzzer_sys::fuzz_target!(|ops: Vec<Op>| {
    let allocator = MyAllocator::new();
    test_allocator(&allocator, &ops);
});

[Folke Behrens]

Right, thanks! Sorry, I should have explained better. I wanted to implement Arbitrary for types that have <A: Allocator>, like Vec and Box, because my types have such containers. I guess I could have kept it simple for tests and only impl Arbitrary for MyType<Global>. In the end I decided to rip out the allocator stuff because too many other types and functions would need to be made generic and because String<A>, probably the most important type, isn't going to be possible for another year.

That's all. Still, adding some generic extra state to Unstructured could be useful. Maybe something for v2.