Simple, secure, and flexible configuration management.
The cStore CLI provides a command to push config files to remote storage using $ cstore push service/dev/.env. The pushed files are replaced by a, cstore.yml file, that remembers the storage location, file encryption, and other details making restoration locally or by a service as simple as $ cstore pull -t dev.
*.env and *.json are special file types whose secrets can be tokenized, encrypted, stored separately from the configuration, and injected at runtime.
Security Best Practices
While cStore provides a simple and flexible way to store and retrieve configuration and secrets, the user has the responsibility to ensure the usage patterns and storage solution meet the oganization's security requirements.
- Understand your organizations security requirements.
- Understand what cStore does before using it.
- Never print or send cStore's
stdoutto logs. - Always use encryption when storing secrets.
- Use your organization's approved vaults for storing secrets.
- Avoid exporting secrets into the environment when possible.
- Realize many security mistakes are made by users; so, be careful!
How it Works
├── project
│ ├── components
│ ├── models
│ ├── main.go
│ ├── Dockerfile
│ ├── cstore.yml (catalog)
│ └── service
│ └── dev
│ │ └── .env (stored)
│ | └── .cstore (ghost)
│ | └── fargate.yml
│ | └── docker-compose.yml
│ │
│ └── prod
│ └── .env (stored)
│ └── .cstore (ghost)
│ └── fargate.yml
│ └── docker-compose.yml
The cstore.yml catalog and hidden .cstore ghost files reference the stored *.env files. Secrets no longer need to be checked into source control.
When the repository has been cloned or the project shared, running $ cstore pull in the same directory as the cstore.yml catalog or any of the .cstore ghost files will locate, download, and decrypt the configuration files to their respective original location restoring the project's environment configuration.
Example: cstore.yml
version: v4
context: project
files:
- path: service/dev/.env
store: aws-s3
type: env
data:
AWS_S3_BUCKET: my-bucket
AWS_STORE_KMS_KEY_ID: ""
AWS_VAULT_KMS_KEY_ID: aws/secretsmanager
tags:
- service
- dev
vaults:
access: env
secrets: aws-secrets-manager
versions: []
- path: service/prod/.env
store: aws-parameter
type: env
data:
AWS_STORE_KMS_KEY_ID: aws/ssm
AWS_VAULT_KMS_KEY_ID: aws/secretsmanager
tags:
- service
- prod
vaults:
access: env
secrets: aws-secrets-manager
versions: []Install / Upgrade
| OS | CMD | Notes |
|---|---|---|
| Mac | $ sudo curl -L -o /usr/local/bin/cstore https://github.com/turnerlabs/cstore/releases/download/v3.8.0-alpha/cstore_darwin_amd64 && sudo chmod +x /usr/local/bin/cstore |
|
| Linux | $ sudo curl -L -o /usr/local/bin/cstore https://github.com/turnerlabs/cstore/releases/download/v3.8.0-alpha/cstore_linux_386 && sudo chmod +x /usr/local/bin/cstore |
|
| Windows | C:\> mkdir %HOMEPATH%\cstore\bin & wget -O %HOMEPATH%\cstore\bin\cstore.exe https://github.com/turnerlabs/cstore/releases/download/v3.8.0-alpha/cstore_windows_amd64.exe (add %HOMEPATH%\cstore\bin to the PATH to make cstore executable from anywhere) |
install requires wget v1.20 |
AWS credential chain is used for Authentication.
$ export AWS_REGION=us-east-1
$ export AWS_PROFILE=user-profileEnsure a storage solution is available and supports the configuration file type.
During a push, tokenized secrets are removed and stored in AWS Secrets Manager.
Store Env Configs
$ cat service/dev/.env # exampleHEALTHCHECK=/ping
MONGO_URL=mongodb://{{dev/user::appuser-dev}}:{{dev/password::3lkjr4kfdro4df}}@example-server.mongodb.net:30000/example-dev
API_KEY={{dev/token::82f6f303-9e00-4a8c-be26-b9d06781d844}}
API_URL=https://dev.api.example-service.com
CONTACT=team@example-service.com
Push configs to one of the following storage solutions.
$ cstore push service/dev/.env -s aws-parameter $ cstore push service/dev/.env -s aws-s3$ cstore push service/dev/.env -s aws-secret$ cstore push service/dev/.env -s source-controlStore Json Configs
$ cat service/dev/config.json # example{
"db_url" : "mongodb://{{dev/user::app_user}}:{{dev/password::4kdnow55jdjnk3nd}}@example-server.mongodb.net:30000/example-dev",
"api_key": "{{dev/key::82f6f303-9e00-4a8c-be26-b9d06781d844}}",
"healthcheck": "/ping",
"contact": "team@example-service.com"
}$ cstore push service/dev/config.json -s aws-s3$ cstore push service/dev/config.json -s aws-secretStore Multiple Configs
$ cstore push service/dev/.env service/qa/.envAuto discover and push multiple files in service folder.
$ cstore push $(find service -name '*.env')Update Configs
$ cstore push # all configs$ cstore push service/dev/.env service/qa/.env $ cstore push -t "dev&qa" # config must have both tags$ cstore push -t "dev|qa" # config must have either tagDuring a pull, -i will retrieve and inject tokenized secrets from AWS Secrets Manager.
Restore Config Files Locally
$ cstore pull # all configs$ cstore pull service/dev/.env service/qa/.env $ cstore pull -t "dev&qa" # config must have both tags$ cstore pull -t "dev|qa" # config must have either tagFormat/Send Configs to Stdout
$ cstore pull -t dev -e # raw file contents$ cstore pull service/dev/.env -g json-object # JSON object format$ eval $( cstore pull service/dev/.env -g terminal-export ) # export environment variablesOutput Task Definition JSON Env/Secrets Formats (.env)
$ cstore pull -t dev -g task-def-env # AWS Task Definition environment$ cstore pull -t dev -g task-def-secrets --store-command refs # AWS Task Definition secretsLoading Configs in a Service
Learning Basics
| Demo | |
|---|---|
| watch | Get Configs With Secrets Injected |