containerd / accelerated-container-image

A production-ready remote container image format (overlaybd) and snapshotter based on block-device.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Userspace convertor] Unexpectedly high number of disk read/written

WaberZhuang opened this issue · comments

What happened in your environment?

We found a large amount of disk IO during the conversion process, e.g. 60MB of disk IO when the original OCI layer is only 8KB.

We have identified the reason for this. The convertor does mkfs to initialize an ext4 filesystem but we missed a feature called 'Lazy Block Group Initialization'. Then, every time it opens/closes the ext4 filesystem, it reads through the bitmap/inode table in each block group. Otherwise, we use a simple cache to increment the IO block size.

We need to change the mkfs to fix it, which may affect the layer deduplication.

What did you expect to happen?

No response

How can we reproduce it?

Convert an overlaybd image and use the option --reserve

$du -sh tmp_conv/0015_sha256\:ec84879c7faf9af7be642c7673a291fdc9926c7c57e3c775c60bfa3cad0095ed/{writable_data,layer.tar}
65M	tmp_conv/0015_sha256:ec84879c7faf9af7be642c7673a291fdc9926c7c57e3c775c60bfa3cad0095ed/writable_data
8.0K	tmp_conv/0015_sha256:ec84879c7faf9af7be642c7673a291fdc9926c7c57e3c775c60bfa3cad0095ed/layer.tar

What is the version of your Accelerated Container Image?

v1.0.2

What is your OS environment?

centos 8

Are you willing to submit PRs to fix it?

  • Yes, I am willing to fix it.