This crate implements a simple blob store. Clients will compute the hash digest and provide it along with the file content. The blob store will verify the digest both on write and later when retrieving the blob using the same digest. Optional authorization can be enabled in which JSON Web Tokens (JWT, RFC 7519) are received and verified against an OpenID provider's JSON Web Key Set (JWKS, RFC 7517).
- Rust stable (2021 edition)
$ cargo build
$ cargo testSome of the test cases involve making a request for the configuration and key set from an OpenID provider, and these are only enabled if the ISSUER_URI environment variable is set. Such a server is readily available via the jwt-test-server docker image (git repo). In the example below, the http://192.168.1.3 address refers to the docker host. Note that the hyper crate used by this application will not accept self-signed certificates, so starting the fake provider with http is necessary.
docker run -d -p 8108:3000 -e PROTOCOL=http -e BASE_URI=http://192.168.1.3:8108 --name jwt-test-server nlfiedler/jwt-test-server:latestRunning the tests would then look like:
env ISSUER_URI=http://192.168.1.3:8108 cargo testAn example of using this crate as a library can be found in the examples directory of the source repository, which demonstrates storing and retrieving blobs.
$ cargo run --example blobby -- put --file README.md
Compiling amemasu v0.1.0 (/Users/nfiedler/projects/amemasu)
Finished dev [unoptimized + debuginfo] target(s) in 0.37s
Running `target/debug/examples/blobby put --file README.md`
stored blob with digest: sha256-b8d6810ab4b5cd71a96c0123ec2da129f35347ded0a69650be8c71eb5c2f963alet repo = BlobStore::new(Path::new("blobs"));
let mut source = fs::File::open("blobby.txt")?;
let mut hasher = Sha1::new();
io::copy(&mut source, &mut hasher)?;
let hash = hasher.finalize();
let digest = format!("sha1-{:x}", hash);
source = fs::File::open(infile)?;
repo.store(&digest, &mut source)?;let repo = BlobStore::new(Path::new("blobs"));
let mut sink = fs::File::create("blobby.txt")?;
let digest: String = "sha1-6584933d1efb012a9f61a48b1922b86c3708a6c2".into();
let found = repo.fetch(digest, &mut sink)?;
if found {
println!("retrieved blob with digest: {}", digest);
} else {
println!("blob not found in repository");
}let repo = BlobStore::new(Path::new("blobs"));
let digest: String = "sha1-6584933d1efb012a9f61a48b1922b86c3708a6c2".into();
repo.delete(&digest)?;The binary form of amemasu will start an HTTP/S server. Depending on the configuration, it may require authorization for any /blob requests.
By default the server will bind to the local address and listen on port 3000. No authorization is required for any of the operations, unless ISSUER_URI is set.
To modify the configuration, set any of the environment variables shown in the table below. The server will look for a file named .env and evaluate it using the dotenv crate. Any name=value pairs defined in that file will result in setting the named environment variable with the given value. Comment lines, which start with #, as well as blank lines, will be ignored.
| Name | Description | Default Value |
|---|---|---|
HOST |
address on which to bind | 127.0.0.1 |
PORT |
port on which to listen | 3000 |
RUST_LOG |
logging level such as debug or info |
error (see env_logger) |
BLOB_PATH |
path to directory for storing blobs | tmp/blobs |
CERT_FILE |
path of PEM-encoded file containing public certificate | certs/cert.pem |
KEY_FILE |
path of PEM-encoded file containing private key | certs/key.pem |
ISSUER_URI |
URI of the OpenID provider | none |
AUDIENCE |
optional aud value by which to restrict access |
none |
JWKS_KEYID |
optional kid value for selecting allowed keys |
none |
Normally the server will not require any authorization to perform /blob operations. However, if the ISSUER_URI environment variable is defined, then all /blobs requests will require a JSON Web Token via the Authorization HTTP header (see RFC 6750). The token will be validated against the key set served by the provider; additionally the token payload must have a purpose claim that is either read or write. Blob operations that make modifications require write access (PUT /blobs and DELETE /blobs) while the read operation (GET /blobs) will accept either read or write access.
At this time, the only supported key encryption algorithm is RS256, meaning that a public/private key pair must be used to sign and validate the token.
In addition to validating the token against the key set, an audience value can be provided via the AUDIENCE environment variable, further restricting access to tokens that have a matching aud claim in the token payload.
By default, the server will take the first key in the key set to validate the token, but this can be restricted to a specific key by setting the JWKS_KEYID environment variable. Note that at this time, the server does not try all of the keys offered in the set, nor does it consider the kid from the token header.
The default self-signed TLS certificates in the certs directory, cert.pem and key.pem, were created using the mkcert utility, although openssl would also work. The advantage of mkcert is that it will install certificate authority certs to validate the certs created with mkcert.
In the following example, localfile is the actual path of the file to be stored in the blob repository, and digest is a hash digest in hexadecimal form, with an algorithm prefix (e.g. sha1-df5d75a08b363294bbfe80dbbbe3542f89aa8dab).
curl -F "file=@localfile;filename=digest" https://localhost:3000/blobsIn the following example, digest is a hash digest of the blob in hexadecimal form, with an algorithm prefix (e.g. sha1-df5d75a08b363294bbfe80dbbbe3542f89aa8dab).
curl -k https://localhost:3000/blobs/digestIn the following example, digest is a hash digest of the blob in hexadecimal form, with an algorithm prefix (e.g. sha1-df5d75a08b363294bbfe80dbbbe3542f89aa8dab).
curl -k -X DELETE https://localhost:3000/blobs/digest