msiebuhr / MetricBase

play/experimentation/single-binary version of Graphite

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

LevelDB: Listing all metrics causes a full scan of all data.

msiebuhr opened this issue · comments

This could work well with a rebuild of the internals of LevelDB, in style of OpenTSDB.

Assign all metrics a 3-byte value and store them with a known prefix in the table, eg:

M:<metric name> = 000
M:<metric2 name> = 001
...

Then store all metrics under the short prefix:

D:000:<timestamp> = value

Speed: We will either need to look up in the first table all the time (or, realistically, keep it in memory). Either way, it will be smaller and way faster to go through.

Size: LevelDB has prefix compression, so shortening keys isn't really that important (I guess). But it's hard arguing against not putting the data there in the first place.

Removed LevelDB for now, as it didn't really do any good for now.