Increase max value size
chjj opened this issue · comments
Values are currently capped at 1024 bytes. There are two reasons for this:
- We try to save as much space as possible on disk by compressing pointers and packing everything into bits. Only 10 bits are currently reserved for sizes.
- Having a max value size lends itself to a much simpler C api, where liburkel need not do any allocation and the user can simply stack allocate a 1024 byte array before a call.
This may be unacceptable for more generic uses of the urkel tree. If this were to change, we would need to:
- Write a migration for older databases.
- Make some breaking API changes.
Yep, this'd be very useful - I was thinking about using it instead of our tiny home-grown key-value multiprocess & multithreading storage DB, but we have values of several megabytes in size quite regularly, so we can't switch to urkel.
I would like to know how hard it is to change the value size to 4096 KB, to allow it storage SQLITE page data. (this will need 12 bits size)