reverbrain / eblob

Eblob is an append-only low-level IO library, which saves data in blob files. Created as low-level backend for elliptics

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Defrag: defragmentation on dedicated device

shaitan opened this issue · comments

Problem

Defragmentation is connected with a lot of disk io activity and it requires 2 * blob_size as a free space on the device. Currently, defragmentation kills disk of backend and all read/write operations with backends are dramatically slowed down. Also free space requirement reserves space on each backend device, as a result we have 2 * blob_size * number of backends reserved but mostly unused space.

Solution

Make defragmentation on dedicated device. Lets make comparison of defragmentation on dedicated device vs backend's device:

Measurement dedicated device backend's device
disk io 1 read and 1 write at the backend's device and 3 read and 3 write at the dedicated device 3 read, 3 write and 1 move at the backend's device
free space requirement for one server 2 * blob_size on dedicated device 2 * blob_size * number of backends

Since defragmentation will have most of disk io at dedicated disk, backend's disk will be free for clients requests.