azeroz / ssbdrv

SSB Drive - a decentralized file system over Secure Scuttlebutt

SSB Drive

This is a proof of concept for a decentralized file system for Secure Scuttlebutt (see SSB). It includes a partial but interoperable implementation of the SSB peer protocol written in Python.

With SBB Drive you can create as many file systems ("drives") as you like, share and organize files with your friends without requiring a central repository or server. When you work on your files while offline, the SSB Drive file system will merge automatically with the rest of the world once you rejoin the grid. Name conflicts are handled with "Observed Removed Sets" (OR-Sets) from CRDT.

Main usage:

$ ./ssb-drive.py [-u USERNAME] [-l] [-peer ip:port:id] [UUID]

where UUID identifies the drive to work on. Use -l to list all available drives. The -peer option selects a specific SSB pub (peer node); the default behavior is to connect to the locally running SSB program at port 8008 (e.g. patchwork or sbot).

Code status: This is a proof-of-concept and is not a well-curated Python package, it also lacks testing routines. DON'T RUN THIS CODE ON YOUR LOG: It's very hard --and often impossible-- to correct a corrupted hash chain especially if it has leaked into the global SSB system. Please read the section "Experimenting with SSB Drive" for safe ways to work locally - you have been warned.

Best is to unpack this Git repo and just run from the ssbdrv directory after installing the dependencies (see requirements.txt). In the future, when the internal interfaces of ssbdrv have stabilized, a full Python package will be provided, probably also factoring out the SSB peer-to-peer component as an independent package. We acknowledge the import of pferreir's pyssb package which had to be made more complete: The modified pyssb code is included for convenience, making this ssbdrv repo self-contained.

Doc status: Read the source, Luke. Moreover, there is a draft document on tangles for SSB in this Git repo, see 'The Tangle data structure and its use in SSB Drive'

Demo

SSB Drive behaves like a classic FTP client except that you don't have to connect to a server. The following demo shows the terminal-based user interface of this PoC.

What can be seen in the 30 second animated GIF:

Alice:  help                  // list available commands
Alice:  ls -l                 // show dir content
Bob:    ls -l                 // Bob sees same content
Bob:    put b.txt             // Bob uploads a file
Alice:  ls -l                 // Alice finds it in her directory
Alice:  cat b.txt             //  and views it
Alice:  put x.txt first.txt   // Alice races to upload first
Bob:    put y.txt first.txt   // Bob races to upload first
Bob:    ls -l                 // surprise: no race condition, but two files
Bob:    tree                  // just another view
Bob:    ls -li                // UNIX -i option: show inode (cypherlink in our case)
                              // i.e., the two files differ, can be removed individually

Pragmatics

Without UUID argument, the app first scans your SSB log and uses the most recent "root entry" it can find as the work drive -- beware that this drive could have been created by a friend. If you know the drive that you want to use (e.g. if you created several drives and/or want to be sure to work on a specific one), you should pass that drive's UUID as an argument.

You can request the creation of a new drive with the -n option and list all available drives with -l.

Because SSB is based on append-only logs, all changes ever made to a drive are preserved for as long as the log or copies of it exist. If one of your friends deletes a file from a drive that you shared, the file is still there and can be recovered: SSB Drive is "time machine-ready" in the sense that all information is available, just that this PoC does not yet implement a method to browse a drive's history (and a method to resurrect old entries).

Have fun and handle your friends' SSB drives with respect!

cft, Aug 2018

---

Example CLI sessions

The full signature of the SSB Drive app is:

$ ./ssb-drive.py [options] [UUID]
where options are:
  -h, --help        show this help message and exit
  -del              delete the given drive
  -list             list all active drives
  -new              create new drive
  -peer IP:PORT:ID  remote's ip:port:id (default is localhost:8008:default_id
  -port PORT        local port (i.e. become a server)
  -tty              run in line mode (instead of fullscreen terminal)
  -user USERNAME    username (default is ~/.ssb user, or ~/.ssb/user.USERNAME)
  -udel             undelete the given drive
UUID                ssb-drive's uuid (default is youngest drive)

Experimenting with SSB Drive and/or testing local developments

In order to perform local experiments with the SSB Drive Protocol, it is possible and advised to run with local SSB users rather than your own ID. To this end, for each user USERNAME, we keep a subdirectory with the following format:

~/.ssb/user.USERNAME

and populate it with the standard SSB data. The SSB Drive software offers an easy way to create new users as follows:

# LOCAL DEMO STEPS 1

$ ./ssb/local/config.py -list
default user:
  @AiBJDta+4boyh2USNGwIagH/wKjeruTcDX2Aj1r/haM=.ed25519
local users:

$ ./ssb/local/config.py -new Alice
** new user Alice (@C8pPydEHuGxCjFUYBLmBOGTIPkYQeZ3FnKvQTvT0MDk=.ed25519)
$ ./ssb/local/config.py -new Bob
** new user Bob (@ihS4TZa55eMjjWOC5oN+oF9GTvc23GQcGyt0xqJ1XD0=.ed25519)

$ ./ssb/local/config.py -list
default user:
  @AiBJDta+4boyh2USNGwIagH/wKjeruTcDX2Aj1r/haM=.ed25519
local users:
  @C8pPydEHuGxCjFUYBLmBOGTIPkYQeZ3FnKvQTvT0MDk=.ed25519  Alice
  @ihS4TZa55eMjjWOC5oN+oF9GTvc23GQcGyt0xqJ1XD0=.ed25519  Bob

Because peers only retrieve each other's logs if they follow each other, we have to populate the friends.json file for both, see below how this is done. Once this is established, we will (i) create a drive on Alice's side, (ii) let Bob sync with Alice's content, and (iii) start also Bob's SSB Drive client:

# LOCAL DEMO STEPS 2

$ ./ssb/local/config.py -friends Alice Bob
** friend records updated

$ ./ssb-drive.py -user Alice -new
** new drive created, uuid=9dfc8124-6a6b-5730-9c04-5eed67ac770e

# start Alice's client in one terminal window:
$ ./ssb-drive.py -user Alice -port 7007
...

# in another terminal window, let Bob sync up:
$ ./ssb-drive.py -user Bob -sync -peer localhost:7007:ID_OF_ALICE
...

# and start his SSB Drive client:
$ ./ssb-drive.py -user Bob -peer localhost:7007:ID_OF_ALICE

It is also possible to run the SSB Drive app in line mode by selecting the -tty option. Note however that this mode does not yet support peer connections i.e., you will work on the given user's log as if offline:

$ ./ssb-drive.py -user Alice -tty
Secure Scuttlebutt Drive client (v2018-08-21).  Type ? for help.
running in unencrypted mode

drv=9dfc8124-6a6b-5730-9c04-5eed67ac770e (2018-08-22 21:44:21)
cwd=/
ssb_drv> help

Documented commands (type help <topic>):
========================================
cat  cd  exit  get  help  ls  mkdir  put  pwd  rm  rmdir  stat  sync  tree

ssb_drv> tree
.
'-- dir1/
    |-- README.md
    '-- dir2/
ssb_drv> exit

---

Technical Details

The "SSB Drive Protocol" (SDP)

to be written

Todo

• "encrypted SSB Drive": adapt the tangles and let them run in the private log
• implement a time machine (browse history, enable resurrecting files)
• think about mounting other drives into a drive's name tree
• run -tty mode with asyncio and serve the peer protocol in the background

---