avoidik / experience-with-vault

HashiCorp Vault advanced topics in one place

Experiments with Vault

Prerequisites

• installed VirtualBox
• docker & docker-machine binaries
• cfssl & cfssljson binaries
• vault & consul binaries
• curl
• fly

What is this

An ideal playground to play with HashiCorp Vault authenticaion methods, secrets backend, etc.

The following advanced topics covered in this repo:

• Docker-Machine & Docker-Containers - how to build and run them
• TLS - how to generate certificates with cfssl and Vault itself, configure TLS communication between components
• Vault Server, Consul Client, Consul Server, OpenLDAP (with phpLDAPadmin & self-service portal), DEX (with static-client & OpenLDAP) configuration
• Concourse CI Server and Workers configuration - its credential manager authenticates in Vault authomatically by using PKI certificate, user will be able to use OIDC or LDAP to authenticate
• OIDC (authenticaion) - using OpenLDAP as a central users directory, roundtrip authentication with OpenLDAP credentials over OIDC
• Identity (both Groups & Entities) - map LDAP groups to Vault identity backend
• LDAP (both authenticaion & secrets) - rotate OpenLDAP secrets and authenticate in Vault or Concourse
• PKI (both authenticaion & secrets) - configure PKI and use it for authentication purposes
• DB (secrets) - rotate database secrets
• KV (secrets) - static secrets

How to run

Just follow along from top to bottom:

1. docker-machine create dex - create test vm
2. ./ca.sh - generate all required TLS certificates
3. ./prep.sh - prepare all required docker images (build)
4. ./run.sh - run test stack on top of test vm
5. source ./vault.env - inject required environment variables (without token)
6. ./init.sh - initialize vault
7. ./unseal.sh - unseal vault
8. source ./vault.env - inject required environment variables (with token)
9. ./policy.sh - write policies
10. ./oidc_auth.sh - configure OIDC authenticaion
11. ./entity.sh - configure identity groups
12. ./ldap_auth.sh - configure LDAP authenticaion
13. ./ldap_secrets.sh - configure LDAP secrets (take a look on dynamically generated credentials for Carl)
14. ./kv.sh - configure KV secrets

How to test PKI

1. ./pki_auth.sh - configure PKI authenticaion & login with cfssl generate certificate
2. ./pki_secrets.sh - configure PKI secrets, generate TLS authenticaion certificate, and login with it (login with Vault generated PKI certificate into Vault)

How to test OIDC and LDAP

1. source ./vault.env - inject required environment variables
2. vault login -method=oidc role=oidc-reader - login with OIDC (and LDAP as identities directory)
3. vault login -method=ldap username=carl - login with LDAP (by using Carl's dynamically generated credentials)

How to start CI

1. ./run_ci.sh - start Concourse CI server (login via OpenLDAP, OIDC, or static credentials)
2. ./team.sh - set demo team in Concourse

It will not work without initializing, unsealing and configuring Vault and related authentication services (LDAP, DEX), hence ./run.sh and all associated scripts must be executed before

How to test Postgres

1. ./postgres_secrets.sh - configure database secrets (Postgres)
2. ./postgres_roles.sh - configure database roles (Postgres)

How to elevate identity entity

1. ./carl.sh - an example on how to elevate someone having access to identity backend
2. ./uncarl.sh - take elevated privileges back

How to read from Consul

1. source ./consul.env - inject required environment variables
2. consul kv get -keys -recurse vault/core/ - list consul keys
3. consul kv get vault/core/lock - read consul key

How to interact with DEX

1. ./dex.sh - view DEX configuration
2. ./login_dex.sh - retrieve JWT token from DEX
3. ./login_dex_public.sh - retrive JWT token from DEX (if Public Client configured)

How to stop

1. ./stop.sh or ./stop_ci.sh - stop test stack
2. ./unca.sh - remove all TLS certificates
3. docker-machine rm -f dex - destroy vm

Special note

Please consider reviewing configuration and processes provided in this repository if you are going to use it, e.g. hardcoded secrets, passwords, etc.

Credits

Found something interesting and want to reuse it in your own project? Please give proper credits to HashiCorp, Concourse CI, Docker-Containers maintainers, OSS authors, and me Viacheslav :-)

• HashiCorp (Consul & Vault), respective Vault Docker Container and Consul Docker Container
• Concourse CI and respective Concourse CI Container
• DEX and respective Docker Container
• LDAP Self-Service Portal and respective Docker Container
• phpLDAPadmin and respective Docker Container
• OpenLDAP and respective Docker Container
• PostgreSQL and respective Docker Container
• CFSSL