Montage: A General System for Buffered Durably Linearizable Data Structures

Montage is a system for building fast buffered persistent data structures, developed at the University of Rochester by H. Wen, W. Cai, M. Du, L. Jenkins, B. Valpey, and M. L. Scott. The work has been submitted to arXiv (link). You can also find its short version on DISC' 20 as a brief announcement (link).

The testing harness is based on J. Izraelevitz's parHarness.

Montage: A General System for Buffered Durably Linearizable Data Structures

1. Required Libraries

Source code of most of required libraries are provided in ./ext, which includes Mnemosyne, libitm.so, Pronto, MOD (which contains nvm_malloc and immutable data structures), Ralloc, and threadcached. Most of them are customized for the experiment. nvm_malloc is vanilla. Please refer to ext/README.md for more information of those libraries.

Other than those in ./ext, this repository also depends on libhwloc, libjemalloc, libuuid, libpthread, and libgomp. Running with Mnemosyne requires additional libelf, libconfig, and libitm libraries.

Before running Pronto, ensure that huge pages are turned on, or an error such as ckpt_alloc.cpp:44: GlobalAlloc::GlobalAlloc(const char*, const char*): Assertion newBlock(&region, BaseAddress, MinPoolSize) failed would occur. You may use the following command to turn on:

sudo sysctl -w vm.nr_hugepages=32768

2. Commands

First, make sure persistent memory is mounted in DAX mode at /mnt/pmem. Montage, MOD, and other native data structures in this harness will create heap files prefixed by the user name. Pronto and Mnemosyne handle heap files by their own and don't have the user name prefix. Please refer to Section 2.3 if NVM is mounted in a different path.

To test scalability (with different thread counts) of queues and maps:

./script/run_test.sh

To test queues and maps with different value sizes:

./script/run_size.sh

To test memcached on YCSB:

./script/run_memcached.sh

To test graph scalability and recovery, set up dataset and run workloads using the following commands:

./script/setup-datasets.sh
./script/run_graph.sh

To plot, run the following commands:

cd data
## data generated by run_test.sh 
Rscript mt_queue_plotting.R # plot queues_thread.pdf from queues_thread.csv
Rscript mt_map_plotting.R # plot hashtables_*_thread.pdf from maps_*_thread.csv

## data generated by run_size.sh
Rscript sz_queue_plotting.R # plot queues_size.pdf from queues_size.csv
Rscript sz_map_plotting.R # plot hashtables_*_size.pdf from maps_*_size.csv

## data generated by run_memcached.sh
Rscript threadcached_plotting.R # plot threadcached_ycsba.pdf from ycsbc_a.csv

## data generated by run_graph.sh
Rscript mt_graph_plotting.R # plot graph_*_thread.pdf from graph_thread.csv
Rscript rec_orkut_plotting.R # plot graph_recovery.pdf from graph_recovery.csv

2.1. Run Specific Queue or Map Test

Please read this subsection if you want to manually test specific queues or maps on some workloads.

To build harness for testing all the queues or maps except those with Pronto and Mnemosyne:

make

To build with Pronto full version (with background threads) and Pronto sync version (worker threads do their own logging), respectively:

make pronto-full
make pronto-sync

To compile with Mnemosyne:

make mnemosyne

Static variables such as K_SZ and V_SZ can be set while building, to adjust key and value size of workloads. See Section 3 for more details.

After building, run the following with proper arguments:

./bin/main -r <rideable> -m <test_mode> -t <thread_num> -i <duration_of_some_tests> [-v]

Running ./bin/main without an argument will print help info listing available options.

2.2. Run Specific Graph Test

If you want to manually run graph tests, do as follows:

To building for testing multithreading performance:

make

To test recovery, run the following command to build harness with default thread pinning on the harness disabled, so that it won't interfere with OpenMP threads spawned during recovery:

make graph-rec

After building, run the following with proper arguments:

./bin/main -r <rideable> -m <test_mode> -t <thread_num> -i <duration_of_some_tests> [-v]

Running ./bin/main without an argument will print help info listing available options. The performance test is GraphTest:1m:i33r33l33:c1 and GraphTest:1m:i25r25l25:c25, while the recovery test is GraphRecoveryTest:Orkut:verify and TGraphConstructionTest:Orkut.

2.3. Use Montage in Your Code

(In preparation)

2.4. Different Mount Point

If NVM is mounted in a different path, please either create a symbolic link at /mnt/pmem (recommended) or search for /mnt/pmem in this repository and replace them appeared in the following files:

ext/mnemosyne-gcc/username/mnemosyne.ini # Mnemosyne
ext/mnemosyne-gcc/usermode/library/mcore/include/config.h # Mnemosyne
ext/pronto-v1.1/src/savitar.hpp # Pronto
ext/ralloc/src/pm_config.hpp # Ralloc

script/EpochLengthSensitivity.sh # script for testing epoch lengths
script/explore_hashtable.sh # script for exploring Montage config on hash table
script/explore_memcached.sh # script for exploring Montage config on threadcached
script/run_graph.sh # script for running graph benchmarks
script/run_memcached.sh # script for running ycsb on memcached
script/run_size.sh # script for running Montage harness with different sizes
script/run_thread.sh # script for running Montage harness with different threads

src/rideables/MODHashTable.hpp # MOD nvm_malloc init path
src/rideables/MODQueue.hpp # MOD nvm_malloc init path

3. Static and Dynamic Environment Variables

3.1. Static Variables

K_SZ: Static variable to control key size (Byte) for maps. By default it's 32. It needs to be set before compilation to take effect, e.g., K_SZ=40 make. Don't pass values less than 10!

V_SZ: Static variable to control value size (Byte) for maps and queues. By default it's 24. It needs to be set before compilation to take effect, e.g., V_SZ=2048 make.

3.2. Dynamic Variables

prefill: The number of elements to be prefilled into the tested data structure. This variable will overwrite the prefill argument passed to Test constructors.

range: This decides the range of keys in map tests. This variable will also overwirte the range argument passed to Test constructors.

There are also options mentioned in ./src/persist/README.md for configuring Montage parameter, e.g., epoch length, persisting strategy, and buffering container.

3.3. Obsolete Variables

There're some obsolete variables no longer useable and are not mentioned above.

To name a few: KeySize and ValueSize. Inconsistency between them and K_SZ and V_SZ will trigger assertion. Please don't use any not mentioned above, unless you know what exactly you are doing.

qtcwt / Montage-1