simerplaha / SwayDB

Persistent and in-memory key-value storage engine for JVM that scales on a single machine.

Home Page:https://swaydb.simer.au

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Create exact header size instead of defaulting to 127 bytes

simerplaha opened this issue · comments

commented

Overview

A Blocks header size cannot be larger than 127 bytes so for each Block a header (Array) get initialised of size 127.

val header = Slice.of[Byte](Byte.MaxValue)

Issue

Even though this byte size (127 bytes) in terms of max file-size (2.GB) is small the in-memory cost of creating Array[Byte] of size 127 is still expensive.

Solutions

  1. Find the actual maximum header size required by randomly running tests with various configurations. My guess is that maximum wouldn't be any larger than 60 bytes with minimum being 2 bytes.
  2. Or Each data-block should provide it's expected header size so header array sizes are exact.