Issue pulling images in Docker Hub automated build during testing phase.

[Jason Beverage]

I've got a private repository that I'm building on Docker Hub via the automated builds. I've provided a docker-compose.test.yml file that has worked for a very long time that has just started to fail and I'm not sure why.

My docker-compose.test.yml file looks similar to this:

version: '2'
services:
  db:
    image: mdillon/postgis:9.5-alpine
  redis:
    image: redis:5.0-alpine
  sut:
    build: .
    command: ./manage.py test
    depends_on:
      - db
      - redis

I know both of those images are very old, but they work just fine :)

The issues I've started to see seem related to authorization but I don't understand what is going on. For example, one build failed with this error:

2024-05-08T14:06:52Z Head "https://registry-1.docker.io/v2/library/redis/manifests/5.0-alpine": unauthorized: incorrect username or password

I've had another build fail with this error (my base image is based on ubuntu:22.04)

024-05-07T16:36:17Z #3 [internal] load metadata for docker.io/library/ubuntu:22.04
2024-05-07T16:36:17Z #3 ERROR: failed to authorize: rpc error: code = Unknown desc = failed to fetch oauth token: unexpected status: 401 Unauthorized
2024-05-07T16:36:17Z
2024-05-07T16:36:17Z #4 [auth] library/ubuntu:pull token for registry-1.docker.io
2024-05-07T16:36:17Z #4 DONE 0.0s
2024-05-07T16:36:17Z ------
2024-05-07T16:36:17Z > [internal] load metadata for docker.io/library/ubuntu:22.04:
2024-05-07T16:36:17Z ------
2024-05-07T16:36:17Z ERROR: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to authorize: rpc error: code = Unknown desc = failed to fetch oauth token: unexpected status: 401 Unauthorized

I did recently change my Dockerfile and it can take a bit longer than usual to build, although when build caching is working the build only takes around 5-7 minutes before the testing phase begins.

Does any of this ring any bells for anyone?

Thanks!

[Josh]

👋🏼 Hey there thanks for the ticket on this! We've discussed and found the issue here. We are rolling out some experimental improvements for a small percentage of traffic (why it's inconsistent) and found an edge case with your submission 😄. We are working on a fix for this!

[Josh]

Update on this: This should be resolved now. Can you verify on your end please?

[Jason Beverage]

I pushed my tests and everything looked good! Thanks for the quick turnaround!