Would it be feasible to add support for pinned ByteArray#?
anka-213 opened this issue · comments
I am working with a project that creates a large immutable long-living structure that unfortunately contains a bunch of Data.ByteString
s. I was hoping to be able to easily reduce GC load by wrapping the creation of this structure with a
fmap getCompact $ compact =<< createStructure
but that of course gives the "Pinned ByteArray# objects cannot be compacted" exception.
I can think of two possible solutions (other than trying to eliminate all bytestrings from the code), but both might add way too much complexity:
- Either being aware of the ByteString library specifically and that it is safe to copy in that specific case. (or even just copy the relevant part of the ByteArray#)
- Or allowing them to still live outside the compact region, but then the region is no longer actually compact, so it might make it a lot more tricky for the garbage collector?
I guess another option would be to make an unsafe version of the compact
function, where you say "trust me, it's safe to copy these pinned ByteArray#".
It's not safe to copy the pinned bytearray backing a ByteString
. Look at the definition of ByteString
and ForeignPtr
:
data ByteString = BS {-# UNPACK #-} !(ForeignPtr Word8) -- payload
{-# UNPACK #-} !Int -- length
data ForeignPtr a = ForeignPtr Addr# ForeignPtrContents
That Addr#
will point to the wrong byte array if you do a deep copy that includes the ForeignPtrContents (most likely the PlainPtr data constructor).