LevelDB: Listing all metrics causes a full scan of all data.
msiebuhr opened this issue · comments
Morten Siebuhr commented
This could work well with a rebuild of the internals of LevelDB, in style of OpenTSDB.
Assign all metrics a 3-byte value and store them with a known prefix in the table, eg:
M:<metric name> = 000
M:<metric2 name> = 001
...
Then store all metrics under the short prefix:
D:000:<timestamp> = value
Speed: We will either need to look up in the first table all the time (or, realistically, keep it in memory). Either way, it will be smaller and way faster to go through.
Size: LevelDB has prefix compression, so shortening keys isn't really that important (I guess). But it's hard arguing against not putting the data there in the first place.
Morten Siebuhr commented
Removed LevelDB for now, as it didn't really do any good for now.