Usage as a Go library with Go modules

[Gareth Rushgrove]

I'm relatively new to using Go modules, so some of this might be a little incorrect. A word of warning.

I wanted to use oras as a library, and successfully tried out the example provided in examples. I then wanted to recreate the functionality in oras which reuses the docker config to authenticate (in my case against an Azure registry. That led me to the code in the auth package.

In theory it should be enough to:

auth "github.com/deislabs/oras/pkg/auth/docker"

And then liberally copy the code from https://github.com/deislabs/oras/blob/master/cmd/oras/resolver.go#L22-L30.

In theory...

In practice this imports docker, which doesn't use tags on the repo to the go mod liking, which means you get an old version of Docker which imports logrus as with a capital letter "S" which causes the world to explode.

I think that means Oras is using Docker v17.03.2-ce, which was shipped 2 years ago https://github.com/moby/moby/releases.

go: github.com/Sirupsen/logrus@v1.4.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"                                                                                           
go: error loading module requirements

Some context for this issue sirupsen/logrus#451.

I've got this working by:

1. importing the latest version of Docker

go get github.com/docker/docker master
go mod tidy

2. Updating where the AuthConfig type comes from in https://github.com/deislabs/oras/blob/master/pkg/auth/docker/resolver.go#L8

+ "github.com/docker/cli/cli/config/types"
- "github.com/docker/docker/api/types"

I think the fix here is to:

• move over to modules from dep (this might be optional)
• Find a suitable, modern, Docker tag to depend on
• Update the types as noted above

At the moment anyone using Modules (recommended with 1.12) will run into this issue I think. It's possible this doesn't hit you if you're on macOS due to the case-incensitive filesystem, I've not tested that. The above was done on Linux.

[Josh Dolitsky]

Hey @garethr - we've only tried using this from dep-based project. I've just opened #99 to switch over to go modules, which hopefully would fix this? Will look into it in the near future unless you want to attempt :)

[Gareth Rushgrove]

I don't think swapping over to Go modules will fix the issue directly, though it may trigger the issue which would make it more pressing.

I think the answer lies in "Find a suitable, modern, Docker tag to depend on". I'm not sure what the community consensus is there so lets ask nice Docker maintainer folks: @thaJeztah, @vdemeester, @silvin-lubecki.

[Sebastiaan van Stijn]

The problem is that go modules enforce the use of SemVer; what happens on the docker/moby repository is that

a) (as you mentioned) the moby repository has not tagged releases (releases are now tagged in the downstream docker/engine repository)
b) even if releases were tagged, go mod will ignore tags that do not match the SemVer format (I'm explicitly mentioning format, because it will assume any tag that's parseable as SemVer "is SemVer"). In case of the docker/moby repositories it will therefore use the 1.13.1 tag, which is parseable as SemVer

While go mod can make sense for small repositories that provide a single package, it becomes quite problematic for repositories that provide a large number of packages (not all of which may follow the same "release cadence", so should be versioned independently).

Things are still being discussed, but I think the current solution is to define a pseudo version in the go.mod (which must be higher than 1.13, otherwise go mod may downgrade to an older version), and specify a commit in that pseudo version; see an example in the BuildKit repository; https://github.com/moby/buildkit/blob/ac18391f2b29c73e0c87cafea8cb63e0c3ac6501/go.mod#L21

[Josh Dolitsky]

@garethr it looks like this is now fixed (thank you @olegsu and @thaJeztah!)

Please verify when you have time. I used the following to test this out:

rm -rf orastest/ && mkdir orastest/ && cd orastest/
wget -O main.go https://raw.githubusercontent.com/deislabs/oras/master/examples/simple_push_pull.go
export GO111MODULE=on
go mod init
go mod vendor

[Josh Dolitsky]

I've verified the set of commands above also work on Linux, running in official Go Docker container:

docker run --rm -it --entrypoint sh -w /go/src golang

[Gareth Rushgrove]

On first importing master into my module like so:

go get -u github.com/deislabs/oras@master

I ran into several build failures, due to the change in types mentioned above.

# github.com/deislabs/oras/pkg/auth/docker
../../go/pkg/mod/github.com/deislabs/oras@v0.4.1-0.20190506165825-e903bf0b0930/pkg/auth/docker/login.go:39:50: cannot use cred (type "github.com/docker/docker/api/types".AuthConfig) as type "github.com/docker/cli/cli/config/types".AuthConfig in argument to c.primaryCredentialsStore(hostname).Store                                                                                                 
../../go/pkg/mod/github.com/deislabs/oras@v0.4.1-0.20190506165825-e903bf0b0930/pkg/auth/docker/resolver.go:27:13: cannot assign "github.com/docker/cli/cli/config/types".AuthConfig to auth (type "github.com/docker/docker/api/types".AuthConfig) in multiple assignment

This comes down to a coupling between the docker package and the docker/cli package. In order to correctly use oras in another module you need to pin the cli version like so:

go get -u github.com/docker/cli@v0.0.0-20181219132003-336b2a5cac7f 

You can see this is specified here: https://github.com/deislabs/oras/blob/master/go.mod#L14

I previously had github.com/docker/cli v0.0.0-20190429053159-3273c2e23546.

So things work, but I think it would be best to update the CLI package as well.

[Josh Dolitsky]

alright this should be fixed. @garethr, would you mind trying once again?

[Gareth Rushgrove]

@jdolitsky just checked this by updating cli and oras to the latest versions and everything works for me. Thanks!