kahing / catfs

Cache AnyThing filesystem written in Rust

Catfs is a caching filesystem written in Rust.

Overview

Catfs allows you to have cached access to another (possibly remote) filesystem. Caching semantic is read-ahead and write-through (see Current Status). Currently it only provides a data cache and all metadata operations hit the source filesystem.

Catfs is ALPHA software. Don't use this if you value your data.

Installation

• On Linux, install via pre-built binaries. You may also need to install fuse-utils first.

• Or build from source which requires Cargo.

:~/catfs$ cargo install catfs
$ # optimized binary now in $HOME/.cargo/bin/catfs

Usage

Catfs requires extended attributes (xattr) to be enabled on the filesystem where files are cached to. Typically this means you need to have user_xattr mount option turned on.

$ catfs <from> <to> <mountpoint>

Catfs will expose files in <from> under <mountpoint>, and cache them to <to> as they are accessed. You can use --free to control how much free space <to>'s filesystem has.

To mount catfs on startup, add this to /etc/fstab:

catfs#/src/dir#/cache/dir /mnt/point    fuse    allow_other,--uid=1001,--gid=1001,--free=1%   0       0

Benchmark

Compare using catfs to cache sshfs vs sshfs only. Topology is laptop - 802.11n - router - 1Gbps wired - desktop. Laptop has SSD whereas desktop has spinning rust.

Compare running catfs with two local directories on the same filesystem with direct access. This is not a realistic use case but should give you an idea of the worst case slowdown.

Write is twice as slow as expected since we are writing twice the amount.

To run the benchmark, do:

$ sudo docker run -e SSHFS_SERVER=user@host --rm --privileged --net=host -v $PWD/target:/root/catfs/target kahing/catfs-bench
 # result is written to $PWD/target

The docker container will need to be able to ssh to user@host. Typically I arrange that by mounting the ssh socket from the host

$ sudo docker run -e SSHFS_OPTS="-o ControlPath=/root/.ssh/sockets/%r@%h_%p -o ControlMaster=auto -o StrictHostKeyChecking=no -o Cipher=arcfour user@host:/tmp" -e SSHFS_SERVER=user@host --rm --privileged --net=host -v $HOME/.ssh/sockets:/root/.ssh/sockets  -v $PWD/target:/root/catfs/target kahing/catfs-bench

License

Copyright (C) 2017 Ka-Hing Cheung

Licensed under the Apache License, Version 2.0

Current Status

Catfs is ALPHA software. Don't use this if you value your data.

Entire file is cached if it's open for read, even if nothing is actually read.

Data is written-through to the source and also cached for each write. In case of non-sequential writes, catfs detects ENOTSUP emitted by filesystems like goofys and falls back to flush the entire file on close(). Note that in the latter case even changing one byte will cause the entire file to be re-written.

References

• Catfs is designed to work with goofys
• FS-Cache provides caching for some in kernel filesystems but doesn't support other FUSE filesystems.
• Other similar fuse caching filesystems, no idea about their completeness:
  • CacheFS - written in Python, not to be confused with FS-Cache above which is in kernel
  • fuse-cache
  • gocachefs
  • mcachefs
  • pcachefs