This is Guanzhou's fork of Intel's Open-CAS Linux cache accelaration system.
Folder structure:
# This is the `casadm` CLI management tool.
# Added `-M` & `-N` command options and `mfwa|mfwb|mfwt` cache modes.
casadm/
|- cas_main.c
|- cas_lib.c
|- cas_lib.h
|- ...
# These are CAS kernel modules loaded into the kernel, i.e., the Linux context.
# New commands support added here.
modules/
|- cas_cache/ # This is the `cas_cache` ko, Linux control cdev also implemented here
| |- service_ui_ioctl.c
| |- layer_cache_management.c
| |- ...
|- cas_disk/ # This is the `cas_disk` core device ko
|- include/
| | cas_ioctl_codes.h
|- ...
# This is the OCF.
# MFC plugins ported to use Linux kernel headers only.
ocf/
|- ...
Everything I have added/modified are marked by [Orthus FLAG BEGIN]
and [Orthus FLAG END]
for easier future reference.
Tested on the following platforms:
- Ubuntu 18.04 LTS
Clone the repo recursively (there is a submodule - the OCF framework):
$ git clone --recursive git@github.com:josehu07/open-cas-linux-mf.git
$ cd open-cas-linux-mf
$ git submodule update --init --recursive
Make sure you have full Linux kernel headers installed:
$ sudo apt-get install linux-headers-$(uname -r)
At the top-level folder, compile by:
$ sudo ./configure
$ make
Then, install by:
$ sudo make install
Check your installation is successful:
$ sudo casadm -V
Once installed, you can set up Open-CAS to start a cache on a device, add another device as a core to the cache, and then mount the virtual /dev/casX-Y
block device to some path.
First, be sure to format the core device to desired FS type:
$ sudo mkfs.ext4 /dev/yyy
Start the cache and add a core:
$ sudo casadm -S -d /dev/xxx # Start cache in WT mode as cache_id = 1
$ sudo casadm -A -d /dev/yyy -i 1 # Add a core device to cache 1
$ sudo casadm -L # List current CAS status
Take a look at open-cas-linux-mf/ocf/src/engine/mf_monitor.c
. Edit monitor parameters and block device stat file path according to your setup.
/** Enable kernel verbose logging? */
static const bool MONITOR_VERBOSE_LOG = true;
/** For block device throughput measurement. */
static const char *CACHE_STAT_FILENAME = "/sys/block/sdc/stat";
static const char *CORE_STAT_FILENAME = "/sys/block/sdb/stat";
/** Do not attempt tuning when miss ratio is higher than X. */
static const int MISS_RATIO_TUNING_BOUND = 2000; // 20%.
/** Consider cache is stable if miss ratio within OLD_RATIO +- X. */
static const int WAIT_STABLE_THRESHOLD = 10; // 0.1%.
/** Sleep X microseconds when detecting cache stability. */
static const int WAIT_STABLE_SLEEP_INTERVAL_US = 1000000;
/** Consider workload change when miss ratio > BASE_RATIO + X. */
static const int WORKLOAD_CHANGE_THRESHOLD = 2000; // 20%.
/** `load_admit` tuning step size. */
static const int LOAD_ADMIT_TUNING_STEP = 100; // 1%.
/** Measure throughput for a `load_admit` value for X microseconds. */
static const int MEASURE_THROUGHPUT_INTERVAL_US = 5000;
/** How many chances given to not quit on `load_admit` 100%. */
static const int NOT_QUIT_ON_100_CHANCES = 10;
Then, start the monitor and switch the cache to a multi-factor cache mode, for example mfwa
:
$ sudo casadm -M -i 1 -j 1 # Start multi-factor monitor to monitor core 1-1
$ sudo casadm -Q -i 1 -c mfwa # Switch cache 1 to Multi-Factor Write-Around
Verify that the cache mode has changed:
$ sudo casadm -L
Now, you can mount the virtual CAS device to a path and starting using Open-CAS-Linux:
$ sudo mount /dev/cas1-1 /mnt/some_path
IMPORTANT: You should first switch the cache to non-mf mode and stop the monitor. Otherwise, directly unmounting or terminating the cache will make your kernel stuck. Do this by:
$ sudo casadm -Q -i 1 -c pt # Swtich cache 1 to Pass-Through mode
$ sudo casadm -N # Stop the running multi-factor monitor
Unmount the virtual device:
$ sudo umount /mnt/some_path
Then, terminate the cache:
$ sudo casadm -T -i 1 # Terminate cache 1
Make sure that no cache is now running:
$ sudo casadm -L
You will find the scripts under
open-cas-linux-mf/scripts
useful. Make sure to check their contents and set correct block device pathes on your system. They are almost certainly different from what's on my node.
Open CAS accelerates Linux applications by caching active (hot) data to a local flash device inside servers. Open CAS implements caching at the server level, utilizing local high-performance flash media as the cache drive media inside the application server as close as possible to the CPU, thus reducing storage latency as much as possible. The Open Cache Acceleration Software installs into the GNU/Linux operating system itself, as a kernel module. The nature of the integration provides a cache solution that is transparent to users and applications, and your existing storage infrastructure. No storage migration effort or application changes are required.
Open CAS is distributed on BSD-3-Clause license (see https://opensource.org/licenses/BSD-3-Clause for full license texts).
Open CAS uses Safe string library (safeclib) that is MIT licensed.
To download latest Open CAS Linux release run following commands:
wget https://github.com/Open-CAS/open-cas-linux/releases/download/v20.3/open-cas-linux-v20.03.0.0286.tar.gz
tar -xf open-cas-linux-v20.03.0.0286.tar.gz
cd open-cas-linux-v20.03.0.0286/
Alternatively, if you want recent development (unstable) version, you can clone GitHub repository:
git clone https://github.com/Open-CAS/open-cas-linux
cd open-cas-linux
git submodule update --init
To configure, build and install Open CAS Linux run following commands:
./configure
make
make install
The ./configure
performs check for dependencies, so if some of them are missing,
command will print their names in output. After installing missing dependencies
you need to run ./configure
once again - this time it should succeed.
To quickly deploy Open CAS Linux in your system please follow the instructions available here.
The complete documentation for Open CAS Linux is available in the Open CAS Linux Administration Guide.
Before running tests make sure you have a platform with at least 2 disks (one for cache and one for core). Be careful as these devices will be most likely overwritten with random data during tests. Tests can be either executed locally or on a remote platform (via ssh) specified in the dut_config.
- Go to test directory
cd test/functional
. - Install dependencies with command
pip3 install -r test-framework/requirements.txt
. - Create DUT config. See example here.
a) Set disks params. You need at least two disks, of which at least one is an SSD drive.
b) For remote execution uncomment and set theip
,user
andpassword
fields.
c) For local execution just leave these fields commented. - Run tests using command
pytest-3 --dut-config=<CONFIG>
where<CONFIG>
is path to your config file, for examplepytest-3 --dut-config="config/dut_config.yml"
.
To report a potential security vulnerability please follow the instructions here.