`Insufficient scope`
ashtonian opened this issue · comments
Description
Steps to reproduce
I attempted to spin up portus and push an image. The registry is setup via traefik at https://registry.mysite.com and portus is available https://portus.mysite.com.
The registry logs show 401 errors about invalid scope, I'm assuming something with oauth but I can't figure out what, any help would be appreciated.
Deployment information
Deployment method: docker
Configuration:
version: "3.7"
services:
portus:
image: opensuse/portus:2.4
# env_file:
# - ./portus.env
environment:
- PORTUS_MACHINE_FQDN_VALUE=portus.mysite.com
- PORTUS_DB_HOST=db
- PORTUS_DB_DATABASE=portus_production
- PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
- PORTUS_DB_POOL=5
- PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
- PORTUS_KEY_PATH=/certificates/portus.key
- PORTUS_PASSWORD=${PORTUS_PASSWORD}
- PORTUS_CHECK_SSL_USAGE_ENABLED='false'
- RAILS_SERVE_STATIC_FILES='true'
# ports:
# - 3000:3000
depends_on:
- db
links:
- db
volumes:
- secrets:/certificates:ro
networks:
- portus
- public
labels:
traefik.enable: "true"
traefik.docker.network: "public"
traefik.backend: "portus"
traefik.frontend.rule: "Host:portus.mysite.com"
traefik.port: "3000"
traefik.protocol: "http"
deploy:
labels:
traefik.enable: "true"
traefik.docker.network: "public"
traefik.backend: "portus"
traefik.frontend.rule: "Host:portus.mysite.com"
traefik.port: "3000"
traefik.protocol: "http"
background:
image: opensuse/portus:2.4
depends_on:
- portus
- db
environment:
# Theoretically not needed, but cconfig's been buggy on this...
- CCONFIG_PREFIX=PORTUS
- PORTUS_MACHINE_FQDN_VALUE=portus.mysite.com
# DB. The password for the database should definitely not be here. You are
# probably better off with Docker Swarm secrets.
- PORTUS_DB_HOST=db
- PORTUS_DB_DATABASE=portus_production
- PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
- PORTUS_DB_POOL=5
# Secrets. It can possibly be handled better with Swarm's secrets.
- PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
- PORTUS_KEY_PATH=/certificates/portus.key
- PORTUS_PASSWORD=${PORTUS_PASSWORD}
- PORTUS_BACKGROUND=true
links:
- db
# env_file:
# - ./portus.env
volumes:
- secrets:/certificates:ro
networks:
- portus
db:
image: library/mariadb:10.0.33
command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
# env_file:
# - ./portus.env
environment:
- MYSQL_DATABASE=portus_production
- MYSQL_ROOT_PASSWORD=${DATABASE_PASSWORD}
volumes:
- mariadb:/var/lib/mysql
networks:
- portus
registry:
image: library/registry:2
# env_file:
# - ./portus.env
environment:
# REGISTRY_HTTP_ADDR: registry.mysite.com
# Authentication
REGISTRY_AUTH_TOKEN_REALM: https://portus.mysite.com/v2/token
REGISTRY_AUTH_TOKEN_SERVICE: registry.mysite.com
REGISTRY_AUTH_TOKEN_ISSUER: portus.mysite.com
REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /secrets/portus.crt
# Portus endpoint
REGISTRY_NOTIFICATIONS_ENDPOINTS: >
- name: portus
url: https://portus.mysite.com/v2/webhooks/events
timeout: 2000ms
threshold: 5
backoff: 1s
volumes:
- registry:/var/lib/registry
- secrets:/secrets:ro
- ./config.yml:/etc/docker/registry/config.yml:ro
ports:
# - 5000:5000
- 5001:5001 # required to access debug service
links:
- portus:portus
networks:
- portus
- public
labels:
traefik.enable: "true"
traefik.docker.network: "public"
traefik.backend: "registry"
traefik.frontend.rule: "Host:registry.mysite.com"
traefik.port: "5000"
traefik.protocol: "http"
deploy:
labels:
traefik.enable: "true"
traefik.docker.network: "public"
traefik.backend: "registry"
traefik.frontend.rule: "Host:registry.mysite.com"
traefik.port: "5000"
traefik.protocol: "http"
volumes:
secrets:
driver: local
driver_opts:
type: "none"
o: "bind,rw"
device: "/mnt/workspace/portus/secrets"
mariadb:
registry:
networks:
public:
external: true
portus:
hi @ashtonian :
I have those last days worked hard on portus
, got to the same point. And I found something :
- I tried pushing images, image after image,using different tags
- The scope terminology made me think about the repositories concept, present in
quay.io
as well. Say you have one repository inside your docker registry, and you named itdkronio
. Then you would push an image to in that registry, to that repository nameddkronio
(so not in another repository), using a tag likedocker push docker.mycompany.io:5000/myinfra/dkronio:1.8.3
. Here,Portus
uses the terminologynamespace
for themyinfra
component of the later tag, and the repository name isdkronio
. - And this scope thing...It's like I don't have the right to push an image of
dkronio
to arepository
inside the freshly createdmyinfra
namespace
. I then see inPortus
, that twonamespaces
were created by default :- one associated to the registry , named like my registry's uri
docker.mycompany.io:5000/
- one associated to my user, the user I created when I signed up first in the
Portus
webapp.
- one associated to the registry , named like my registry's uri
- So suddenly I have an idea : If I have any permission, on any namespace , then that'd be on the one created for my user.
- Let the username I chose at signup be
zorro
, then, to a repository inside thezorro
namespace push an image to that one, i'll have to push my image using a tag like :
docker pull docker pull dkron/dkron
export IMAGE_ID=$(docker images|grep dkron | awk '{print $3}' )
docker tag $IMAGE_ID docker.mycompany.io:5000/zorro/dkronio:1.8.3
docker push docker.mycompany.io:5000/zorro/dkronio:1.8.3
Well try that with your own username, never the less, it worked in my case.
All in all, I think this scope thing has to be about permissions management in permissions, and I consider that because I never had any issue while pushing to a private docker registry, with any "full tag path" of the form :
docker.mycompany.io:5000/whatever/ifeellike:1.8.3`
# "whatever" and "ifeellike" are usually completely free of choice.
Btw, thank you for the traefik setup, I was so thinking about doing that next, and completely get rid of the twisted nginx conf,to endup with a clear traefik.io, and a drastically simplified,and clear nginx.conf
HI again @ashtonian :
I had more results on pushing images
Alright, here are my new results, using my latest provisioning recipe of portus :
- I use
opensuse/portus:2.4.3
(update : also tested withopensuse/portus:2.5
with same fine results) andlibrary/registry:2.6
- when i first access the web app https://portus.mycompany.io:3000/ I create a user named
garcia
- Then, in the web app, I am logged in as
garcia
, and being an admin (first user has to be an admin), I configure the registry : I tell theportus
app how to connect to my private docker registry. There is a VERY important point, and probably where the problem resides on your side. - So here is the very important point :
- When in the
portus
webapp, you configure the registry, there is anadvanced
button. See it? Well click on it, and you see another configuration parameter, namelyExternal Registry Name
this :
- When in the
- What is so important here, is that the value you give to that parameter HAS TO MATCH ONE VARIABLE IN DOCKER-COMPOSE.YML, the
REGISTRY_AUTH_TOKEN_SERVICE
variable, as of this note by @adnoh (thank yu so much @adnoh !! )
But that's not every thing : now let's push anywhere
Ok, so now that you have your problem solved, of course, we want to push images to our repo (dying to actually).
So here is on example I tested exactly as I describe below, and worked as you will understand, just as portus
is expected to behave :
- In the
portus
web app, still logged in asgarcia
, I create a team, which I namegarciasteam
. I note after that, thatportus
gives me the info that I (garcia
), am owner, of that team. Ok, just natural. - Then, still in the
portus
web app, and still logged in asgarcia
, I create a namespace attached the teamgarciasteam
, namespace I chose to namegarciashood
. - Being there, I wanna have dinner, so I just have a quick overview of all those things I just created, and I notice one thing :
- Any namespace has 3 possible choices for one settings that looks like permissions : "
locked
" (no one can push or pull from the repo), "open to team
" (only team members can pull from repo), or "public
". :
- Any namespace has 3 possible choices for one settings that looks like permissions : "
- The namespace I just created, namely
garciashood
, is locked : portus probably locks down freshly created namespaces by default. - Great, immediatly, I unlocked the
garciahood
namespace, and now let's push :- one image to the namespace created for my user,
garcia
- one image to the namespace
garciahood
I created myself, for my teamgarciasteam
- one a freely chosen namespace, here below
zorro
, which I never created when logged in theportus
web app
- one image to the namespace created for my user,
- And here are the results, which conforms to anyone's expectations :
johnbl@poste-devops-typique:~$ docker pull node
Using default tag: latest
latest: Pulling from library/node
844c33c7e6ea: Pull complete
ada5d61ae65d: Pull complete
f8427fdf4292: Pull complete
f025bafc4ab8: Pull complete
7a9577c07934: Pull complete
9b4289f800f5: Pull complete
c74d80ccdeab: Pull complete
b418965736e5: Pull complete
fb4cff8b8d55: Pull complete
Digest: sha256:a4ee833346b09f24095868f6a9d2c7781b6ac319821f912df05f71c6f5a4259c
Status: Downloaded newer image for node:latest
johnbl@poste-devops-typique:~$ export TEST_IMAGE_ID=$(docker images | grep node| awk '{print $3}')
johnbl@poste-devops-typique:~$ docker tag $TEST_IMAGE_ID oci-registry.pegasusio.io:5000/garciashood/node-latest:0.0.5
johnbl@poste-devops-typique:~$ docker tag $TEST_IMAGE_ID oci-registry.pegasusio.io:5000/garcia/node-latest:0.0.5
johnbl@poste-devops-typique:~$ docker login oci-registry.pegasusio.io:5000 --username jbl
Password:
WARNING! Your password will be stored unencrypted in /home/johnbl/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/garciashood/node-latest:0.0.5
The push refers to repository [oci-registry.pegasusio.io:5000/garciashood/node-latest]
eb6930092ccc: Pushed
e07b73aa5089: Pushed
bc9e904364b4: Pushed
553039093d83: Pushed
2e517d68c391: Pushed
5f3a5adb8e97: Pushed
73bfa217d66f: Pushed
91ecdd7165d3: Pushed
e4b20fcc48f4: Pushed
0.0.5: digest: sha256:737b3a051de3db388aac1d4ef2e7cf6b96e6dcceb3e1f700c01e8c250d7d5500 size: 2215
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/zorro/node-latest:0.0.5The push refers to repository [oci-registry.pegasusio.io:5000/zorro/node-latest]
eb6930092ccc: Preparing
e07b73aa5089: Preparing
bc9e904364b4: Preparing
553039093d83: Preparing
2e517d68c391: Preparing
5f3a5adb8e97: Waiting
73bfa217d66f: Waiting
91ecdd7165d3: Waiting
e4b20fcc48f4: Waiting
denied: requested access to the resource is denied
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/garcia/node-latest:0.0.5
The push refers to repository [oci-registry.pegasusio.io:5000/garcia/node-latest]
eb6930092ccc: Pushed
e07b73aa5089: Pushed
bc9e904364b4: Pushed
553039093d83: Pushed
2e517d68c391: Pushed
5f3a5adb8e97: Pushed
73bfa217d66f: Pushed
91ecdd7165d3: Pushed
e4b20fcc48f4: Pushed
0.0.5: digest: sha256:737b3a051de3db388aac1d4ef2e7cf6b96e6dcceb3e1f700c01e8c250d7d5500 size: 2215
johnbl@poste-devops-typique:~$
Another and most important test
- create another user
vercingetorix
, make him an admin, - create his own team
gaulois
, and a namespace for that team,alesia
, - open access to namespace
alesia
, to logged in members (of the team) only, - logged in as
garcia
, try and push an image toalesia
: which should fail. - Then logged in as
vercingetorix
, addgarcia
to thegaulois
team, withViewer
role and try again the same docker push. Should fail, because users with roleViewer
can only docker pull. So we try docker pull, and it should work. - Then logged in as
vercingetorix
, and modifygarcia
's role, as member of thegaulois
team, fromViewer
toContributor
role, and try again the same docker push. Should work this time.As well as docker pulls. - finally, i'll be in Heaven, if
portus
RESTful API (hopefully OpenAPI compliant), would be mature enough to completely automate the above described test. I'll give news about that if asked.
The Permissions rules stated in portus
' documentation :
Every Team member has a
role
, pick among the following three :
Viewer
: viewers can only pull from the repositories owned by the team.Contributor
:contributors
can both pull and push from the repositories owned by the team.Owner
: owners have the same permissions ascontributors
, but they can also manage the list of team members. Owners can: add/remove team members and edit the role of team members.(But there are no teams when we start working with
portus
, so : )Push policies
Push policies are regulated in the
user_permission.push_images.policy
option in theportus
configuration file/srv/Portus/config/config.yml
, described here. It may take one of the following values:
allow-teams
: this is the default value and it will simply apply all the rules that have been stated above on this page. That is, push policy will be regulated through team permissions. This way,Portus
administrators and team owners and contributors will be able to push to the namespaces owned by a given team.allow-personal
: this way Portus will restrict push access to only administrators of Portus. That being said, users will still have their own personal namespace at their disposal.admin-only
: when used, it will restrict push access to only Portus administrators. Users won’t even have a personal namespace. Use this option if you want to ensure that onlyPortus
administrators can submit Docker images to your private registry.Note that when either allow-personal or admin-only have been selected, then owners, contributors and viewers of a team have the same permissions on team-owned namespace: only pull access.
And About he Web UI, which is a problem in portus, well... I'll surely give news about that too. You know, i love super secret projects too :) ...
hi @ashtonian I think possible I found what's going wrong :
In your docker-compose.yml
, for the registry
, where is this :
command: ["/bin/sh", "/etc/docker/registry/init"]
?
It's important because it makes "trusted" (by the registry), the SSL/TLS https cetificate of your token issuer, namely your portus service at https://portus.mycompany.io/v2/token. The Certificate, in this recipe, is used three times :
- once to make an authority trusted (by users's computers and firefoxes) : that's the authority who signed the SSL Certtificate , lets' call it
WebCertificate.crt
of the the https://portus.mycompany.io web app. To make that authority trusted, we need the CA 's own certificate, let's call itPortusCAAuthority.crt
which has to either be signed by an already trusted CA, or self-signed (root ca). Well actually the officalportus
docker-compose recipe wants that we useWebCertificate.crt
asPortusCAAuthority.crt
. The same certificate - a second time to make an authority trusted (by the
registry
webhook client who sends the nofications to the protus web app) : that's the authority who signed the SSL Certtificate , we called itWebCertificate.crt
of the the https://portus.mycompany.io web app. Again, To make that authority trusted, we need its own certificate, again beingWebCertificate.crt
, which is self-signed. Note that if you use a certificate signed by an external authority, so not self-signed, then you will need that CA's own public SSL Certificate, exactly it's theexamples/compose/registry/init
script , mapped inside the regisry conainer to "/etc/docker/registry/init", that will need to find the signing CA's certificate, likethawte.crt
. - and a last time to make an authority trusted (by the docker clients you use to
docker push
images) : to prove identity of theregistry
service to the docker client. Well there, its the docker registry service that has the same SSL Certifcate, used as SSL Certificate (seeregistry/config.yml
). Oh but there's a part you miss perhaps here : it is that it's on the machine from which youdocker push
, that the registry's certifcate has to be trusted. So okkay, you need to do that :
# on the machine where you docker push
cp WebCertificate.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
All in all, you still have to make sure your namespaces, users, and permissions are set accordingly to the rules I detailed above, before trying to docker push
.
hi @Jean-Baptiste-Lasselle
sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.
After correctly configuring portus and registry to read the dumped traefik certificate in .key
and .crt
format and it just worked.
Here is an traefik v2 compose file with all related services for portus and registry. Closing.
version: "3.7"
services:
portus:
image: opensuse/portus:2.4.3
# env_file:
# - ./portus.env
environment:
- PORTUS_MACHINE_FQDN_VALUE=${PORTUS_DOMAIN}
- PORTUS_DB_HOST=db
- PORTUS_DB_DATABASE=portus_production
- PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
- PORTUS_DB_POOL=5
- PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
- PORTUS_KEY_PATH=/certificates/${PORTUS_DOMAIN}/privatekey.key
- PORTUS_PASSWORD=${PORTUS_PASSWORD}
- PORTUS_CHECK_SSL_USAGE_ENABLED=false
- PORTUS_SIGNUP_ENABLED=false
- RAILS_SERVE_STATIC_FILES=true
- PORTUS_GRAVATAR_ENABLED=true
- PORTUS_DELETE_ENABLED=true
- PORTUS_DELETE_CONTRIBUTORS=false
- PORTUS_DELETE_GARBAGE_COLLECTOR_ENABLED=true
- PORTUS_DELETE_GARBAGE_COLLECTOR_OLDER_THAN=30
- PORTUS_DELETE_GARBAGE_COLLECTOR_KEEP_LATEST=5
- PORTUS_OAUTH_GITHUB_ENABLED=true
- PORTUS_OAUTH_GITHUB_CLIENT_ID=${PORTUS_OAUTH_GITHUB_CLIENT_ID}
- PORTUS_OAUTH_GITHUB_CLIENT_SECRET=${PORTUS_OAUTH_GITHUB_CLIENT_SECRET}
- PORTUS_OAUTH_GITHUB_ORGANIZATION=yourteam
# - PORTUS_OAUTH_GITHUB_TEAM=''
# - PORTUS_OAUTH_GITHUB_DOMAIN=''
- PORTUS_ANONYMOUS_BROWSING_ENABLED=false
# - PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
# ports:
# - 3000:3000
depends_on:
- db
links:
- db
volumes:
- traefik_certs_raw:/certificates:ro
# - secrets:/certificates:ro
networks:
- portus
- public
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${PORTUS_DOMAIN}"
- "traefik.http.routers.portus.rule=Host(`${PORTUS_DOMAIN}`)"
- "traefik.http.routers.portus.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.portus.service=portus"
- "traefik.http.routers.portus.tls=true"
- "traefik.http.routers.portus.tls.certresolver=le"
- "traefik.http.services.portus.loadbalancer.server.port=3000"
- "traefik.http.services.portus.loadbalancer.server.scheme=http"
- "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
- "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
deploy:
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${PORTUS_DOMAIN}"
- "traefik.http.routers.portus.rule=Host(`${PORTUS_DOMAIN}`)"
- "traefik.http.routers.portus.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.portus.service=portus"
- "traefik.http.routers.portus.tls=true"
- "traefik.http.routers.portus.tls.certresolver=le"
- "traefik.http.services.portus.loadbalancer.server.port=3000"
- "traefik.http.services.portus.loadbalancer.server.scheme=http"
# - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
# - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
background:
image: opensuse/portus:2.4.3
depends_on:
- portus
- db
environment:
# Theoretically not needed, but cconfig's been buggy on this...
- CCONFIG_PREFIX=PORTUS
- PORTUS_MACHINE_FQDN_VALUE=${PORTUS_DOMAIN}
- PORTUS_DB_HOST=db
- PORTUS_DB_DATABASE=portus_production
- PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
- PORTUS_DB_POOL=5
- PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
- PORTUS_KEY_PATH=/certificates/${PORTUS_DOMAIN}/privatekey.key
- PORTUS_PASSWORD=${PORTUS_PASSWORD}
# - PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
# - PORTUS_CHECK_SSL_USAGE_ENABLED=false
- PORTUS_GRAVATAR_ENABLED=true
- PORTUS_DELETE_ENABLED=true
- PORTUS_DELETE_CONTRIBUTORS=false
- PORTUS_DELETE_GARBAGE_COLLECTOR_ENABLED=true
- PORTUS_DELETE_GARBAGE_COLLECTOR_OLDER_THAN=30
- PORTUS_DELETE_GARBAGE_COLLECTOR_KEEP_LATEST=5
- PORTUS_OAUTH_GITHUB_ENABLED=true
- PORTUS_OAUTH_GITHUB_CLIENT_ID=${PORTUS_OAUTH_GITHUB_CLIENT_ID}
- PORTUS_OAUTH_GITHUB_CLIENT_SECRET=${PORTUS_OAUTH_GITHUB_CLIENT_SECRET}
- PORTUS_OAUTH_GITHUB_ORGANIZATION=yourteam
# - PORTUS_OAUTH_GITHUB_TEAM=''
# - PORTUS_OAUTH_GITHUB_DOMAIN=''
- PORTUS_ANONYMOUS_BROWSING_ENABLED=false
- PORTUS_BACKGROUND=true
- PORTUS_BACKGROUND_REGISTRY_ENABLED=true
- PORTUS_BACKGROUND_SYNC_ENABLED=true
- PORTUS_BACKGROUND_SYNC_STRATEGY=update-delete
links:
- db
# env_file:
# - ./portus.env
volumes:
- traefik_certs_raw:/certificates:ro
networks:
- portus
db:
image: library/mariadb:10.0.33
command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
# env_file:
# - ./portus.env
environment:
- MYSQL_DATABASE=portus_production
- MYSQL_ROOT_PASSWORD=${DATABASE_PASSWORD}
volumes:
- mariadb:/var/lib/mysql
networks:
- portus
# clair: TODO:
# image: quay.io/coreos/clair
# restart: unless-stopped
# depends_on:
# - postgres
# links:
# - postgres
# - portus
# ports:
# - "6060-6061:6060-6061"
# volumes:
# - /tmp:/tmp
# - ./clair/clair.yml:/clair.yml
# command: [-config, /clair.yml]
registry:
image: library/registry:2.7.1
# env_file:
# - ./portus.env
environment:
# REGISTRY_HTTP_ADDR: ${REGISTRY_DOMAIN}
# Authentication
REGISTRY_AUTH_TOKEN_REALM: https://${PORTUS_DOMAIN}/v2/token
REGISTRY_AUTH_TOKEN_SERVICE: ${REGISTRY_DOMAIN}
REGISTRY_AUTH_TOKEN_ISSUER: ${PORTUS_DOMAIN}
REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /certificates/${PORTUS_DOMAIN}/certificate.crt
# Portus endpoint
REGISTRY_NOTIFICATIONS_ENDPOINTS: >
- name: portus
url: https://${PORTUS_DOMAIN}/v2/webhooks/events
timeout: 2000ms
threshold: 5
backoff: 1s
volumes:
- traefik_certs_raw:/certificates:ro
- registry:/var/lib/registry
- secrets:/secrets:ro
- ./config.yml:/etc/docker/registry/config.yml:ro
ports:
# - 5000:5000
- 5001:5001 # required to access debug service
links:
- portus:portus
networks:
- portus
- public
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${REGISTRY_DOMAIN}"
- "traefik.http.routers.registry.rule=Host(`${REGISTRY_DOMAIN}`)"
- "traefik.http.routers.registry.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.registry.service=registry"
- "traefik.http.routers.registry.tls=true"
- "traefik.http.routers.registry.tls.certresolver=le"
- "traefik.http.services.registry.loadbalancer.server.port=5000"
- "traefik.http.services.registry.loadbalancer.server.scheme=http"
# - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
# - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
deploy:
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${REGISTRY_DOMAIN}"
- "traefik.http.routers.registry.rule=Host(`${REGISTRY_DOMAIN}`)"
- "traefik.http.routers.registry.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.registry.service=registry"
- "traefik.http.routers.registry.tls=true"
- "traefik.http.routers.registry.tls.certresolver=le"
- "traefik.http.services.registry.loadbalancer.server.port=5000"
- "traefik.http.services.registry.loadbalancer.server.scheme=http"
# - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
# - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
traefik:
ports:
- "80:80"
- "443:443"
- "8183:8080"
image: traefik:2.1
logging: *default-logging
volumes:
- traefik_certs:/certs
networks:
- public
- dockersock
labels:
- "traefik.enable=true"
- "traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)"
- "traefik.http.routers.http_catchall.entrypoints=web"
- "traefik.http.routers.http_catchall.middlewares=https_redirect"
- "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https"
- "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
- "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
- "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
- "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
- "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
- "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
- "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
- "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
- "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)"
- "traefik.http.routers.http_catchall.entrypoints=web"
- "traefik.http.routers.http_catchall.middlewares=https_redirect,sslHeaders"
- "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https"
- "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
- "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
- "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
- "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
- "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
- "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
- "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
- "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
- "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
command:
- "--api.dashboard=true"
- "--api.insecure=true"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--providers.docker=true"
- "--certificatesResolvers.le.acme.email=youremail.com"
- "--certificatesResolvers.le.acme.storage=/certs/acme.json"
- "--certificatesResolvers.le.acme.httpChallenge.entryPoint=web"
- "--certificatesResolvers.le.acme.tlsChallenge=true"
# - "--certificatesResolvers.le.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory" # lets encrypt staging, remove after verified deployment
- "--providers.docker.endpoint=tcp://mgmt_docker-proxy:2375"
# - "--providers.docker.useBindPortIP=true"
- "--providers.docker.exposedByDefault=false"
- "--providers.docker.network=public"
# - "--providers.docker.swarmMode=true"
certDumper:
image: ldez/traefik-certs-dumper:v2.7.0
command:
- "file"
- "--version=v2"
- "--source=/certs/acme.json"
- "--domain-subdir=true"
- "--dest=/dump/"
- "--watch=true"
# - "--crt-ext=.pem"
# - "--key-ext=.pem"
volumes:
- traefik_certs:/certs
- traefik_certs_raw:/dump
volumes:
secrets:
driver: local
driver_opts:
type: "none"
o: "bind,rw"
device: "/mnt/workspace/portus/secrets"
traefik_certs_raw:
driver: local
driver_opts:
type: "none"
o: "bind,ro"
device: "/mnt/workspace/traefik_certs_raw/"
mariadb:
registry:
networks:
public:
external: true
dockersock:
external: true
portus:
hi @Jean-Baptiste-Lasselle
sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.
So it was from here that I had already seen your github username! :D SO much thank you for sharing all these infos about the certificates n traefik, I from the start planned to switch to traefik, but I didn' t want to do that before I understand completely every part of the network communication between portus background and registry.
@ashtonian that is really excellent work on certificates and traefik, you know I have to tell you about that that I crossed roads a year and a half ago with some guys at carrefour in France, working with Google, and this guy kept saying all around that traefik can't work with SSL Certificates;and is forbidden inside all infra at carrefour. So ridiculous. I wanted to say to really tell you thank you for your work on SSL and traefik with portus, valuable.
@ashtonian Hi again, I wanna mean a huuuuge thank you about the traefik cert dumper technique, it's like really great !
hi @Jean-Baptiste-Lasselle
sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.
So, just to be sure I understand everything in your config :
- The SSL certificates you use in your config, are issued by Let's Encrypt,
- and
Traefik
is configured with an ACME provider, https://docs.traefik.io/https/acme/ , am I right ? - The certificates are requested and automatically renewed by
traefik
itself, acting as would acertbot
, - And those certificates are dumped into files, thanks to https://github.com/ldez/traefik-certs-dumper , so containers have access to them through docker volume sharing, in your case the
traefik_certs_raw
volume. - Do you confirm when you run your
docker-compose.yml
, you have a./traefik_certs/
in the folder where yourdocker-compose.yml
file is ? - And If I understand well, what the cert dumper does, is reading the content of the file
/certs/acme.json
, and generate actual*.key
and*.crt/*.cert
files into the/dump
folder inside the container. Am I right there ?
@ashtonian I also really like you secret dedicated docker volume definition. I'll use that too from now on, in addition with HashiCorp Vault as Secret Manager
Hi @ashtonian, I dumped this about PORTUS_SECRET_KEY_BASE
,as a thank you for making me understand your traefik-cert-dumper
, sharing it with community.