support for 0-byte file creation rates?

[Glenn K. Lockwood]

I am trying to use elbencho to test the file create rate of a file system and hit two problems. First, when I just try to make as many files as I can within a timelimit,

$ ~/src/elbencho/bin/elbencho --timelimit 60 --threads 1 --size 0 --block 0 --write --delfiles outputfile.{0..999999}
-bash: /global/homes/g/glock/src/elbencho/bin/elbencho: Argument list too long

I know why this is happening, but I am wondering if there is another way to create a bunch of files without expressing each one on the command line.

When I reduce the number of files, I get a different error:

$ ~/src/elbencho/bin/elbencho --timelimit 60 --threads 1 --size 0 --block 0 --write --delfiles tmp.{0..100000}
ERROR: File size must not be 0 when benchmark path is a file. File: tmp.0

Is the creation of empty files unsupported? It's a common benchmark requirement for file systems (though not a very sensible one...), and I am trying to find the worst-case performance for when users scp a directory full of tiny (empty) files to different file systems (again, not a sensible thing to do, but our users love doing it).

[Glenn K. Lockwood]

I am starting to realize that this may not have been an intended use case for elbencho. It appears that all files are held open during the test:

$ ~/src/elbencho/bin/elbencho --timelimit 60 --threads 1 --size 4096 --block 4096 --write --delfiles tmp.{0..100000}
ERROR: Unable to open benchmark path: tmp.4093; SysErr: Too many open files

If it's significant work to add this functionality to elbencho, I can make due with mdtest instead.

[Sven Breuner]

Hi @glennklockwood, indeed when you specify the filenames directly as parameters, then elbencho opens all of them in the beginning and shares the open filedescriptors across all the threads. That makes this mode more appropriate for the case where you're working with a limited number of large files (or block devices).

When you do sequential reads/writes in this mode (i.e. you don't specify "--rand") then elbencho will equally partition the total amount of data between the files.
For example
elbencho -w -t 8 -b 1m -s 10g /mnt/somestorage/file{1..4}
...means we have 4x10GB=40GB and 8 threads to write the data. So the first thread will write the first 5GB (=40GB div 8) of the first file, the 2nd thread will write the 2nd 5GB of the first file, the 3rd thread will write the 1st 5GB of the 2nd file and so on.

When you add --rand e.g. to check 4KB random reads in large files, for example
elbencho --rand -r -t 8 -b 1m -s 10g /mnt/somestorage/file{1..4}
...then each thread will randomly select the next file and randomly select the next offset. So in other words, the threads jump not only randomly within a file, but also randomly between the given files.

What you're looking for is described under elbencho --help-multi. In this mode, the main difference is that you use one (or optionally multiple) directories as parameters instead of using actual filenames. And you specify the number of subdirectories (-n) that each thread should create and the number of files (-N) that each thread should put into each of its directories.

So to get to your single thread creating 1 million files in a single dir, you would set one directory per thread (-n 1) and 1 million files per dir (-N 1000000):
elbencho -d -w -t 1 -s 4k -b 4k -n 1 -N 1000000 /mnt/somestorage/somedir
If you would set 2 threads, then each of them would create a directory with 1 million files inside. I you would want 2 threads that together create one million files in a single dir then you would use the "--dirsharing" option like this:
elbencho -d -w -t 1 -s 4k -b 4k -n 1 -N 500000 --dirsharing /mnt/somestorage/somedir

I see you also set a timelimit. With that, you probably want to run file creation and deletion in two separate commands instead of a single one to make sure that the deletion actually runs and full completes like this:
elbencho -d -w -t 1 -s 4k -b 4k -n 1 -N 1000000 --timelimit 60 /mnt/somestorage/somedir
elbencho -F -t 1 -s 4k -b 4k -n 1 -N 1000000 --nodelerr /mnt/somestorage/somedir

[Glenn K. Lockwood]

This is fantastic! Thanks for the explanation. For some reason I thought --help-multi was about running across multiple clients, not multiple directories, so I never bothered looking in there. I tried out

mkdir somedirs
./bin/elbencho --threads 1 \
               --size 0 \
               --files 10000 \
               --mkdirs \
               --write \
               --delfiles \
               --deldirs \
               --dirsharing \
               ./somedirs

and it did exactly what I had hoped.