Latest ECS optimized ami-02507631a9f7bc956 storage configuration changed

[Carlos Olmos]

I just launched a new spot fleet request with the latest AMI ami-02507631a9f7bc956 for my ECS cluster.
I found that the storage configuration of this AMI no longer matches the documentation:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-ami-storage-config.html states that:

Amazon ECS-optimized Amazon Linux AMIs from version 2015.09.d and later launch with an 8-GiB volume for the operating system that is attached at /dev/xvda and mounted as the root of the file system. There is an additional 22-GiB volume that is attached at /dev/xvdcz that Docker uses for image and metadata storage.

And that:

You can increase these default volume sizes by changing the block device mapping settings for your instances when you launch them; however, you cannot specify a smaller volume size than the default.

When creating the Spot Fleet Request via the web console I found that /dev/xvdcz is not an available volume device, although /dev/xvdbz is available.

After the EC2 instance was created I found that vgs is no longer installed in the base O.S. and that docker is using a different Storage Driver (devicemapper changed to overlay2).

This is the way "docker info" reported the storage in previous AMI ami-0bf2fb355727b7faf:

$sudo docker info
...
Server Version: 18.06.1-ce
Storage Driver: devicemapper
 Pool Name: docker-docker--pool
 Pool Blocksize: 524.3kB
 Base Device Size: 10.74GB
 Backing Filesystem: ext4
 Udev Sync Supported: true
 Data Space Used: 3.503GB
 Data Space Total: 42.42GB
 Data Space Available: 38.92GB
 Metadata Space Used: 6.701MB
 Metadata Space Total: 46.14MB
 Metadata Space Available: 39.44MB
 Thin Pool Minimum Free Space: 4.242GB
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
 Library Version: 1.02.135-RHEL7 (2016-11-16)
Logging Driver: json-file
Cgroup Driver: cgroupfs
...

And this is how is now:

$ sudo docker info
...
Server Version: 18.06.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
...

What's the correct process to extend the size of the docker partition for image and metadata storage in the new AMI?
We also use the extra volume /dev/xvdcz to add encryption to our container storage. What's the recommended approach now?

[Eric Sun]

Hi, thanks for raising the question

Please note that ami-0bf2fb355727b7faf is an Amazon Linux image and ami-02507631a9f7bc956 is an Amazon Linux 2 image. AMI default storage configurations are different between Amazon Linux AMI and Amazon Linux 2 AMI

From our document for Amazon Linux 2 image here. By default, the Amazon Linux 2-based Amazon ECS-optimized AMIs (Amazon ECS-optimized Amazon Linux 2 AMI, Amazon ECS-optimized Amazon Linux 2 (arm64) AMI, and Amazon ECS GPU-optimized AMI) ship with a single 30-GiB root volume.

While for Amazon Linux image document here. Amazon Linux image will have 8G /dev/xvda and 22G /dev/xvdcz volume by default as you mentioned above.

To keep your ECS cluster's volume setup consistent, you could try to use the latest AMI under the tab Amazon Linux AMI in this link instead of Amazon Linux 2

[Carlos Olmos]

Thanks for the quick reply and the reference.