Unable to complete go get command when GOPROXY=direct

[Gil]

I've run into an issue using this configuration
The issue is that if I push a new version of a module and try to perform go get then it doesn't find the new version unless I perform a go mod download first.

g := goproxy.Goproxy{}
dirCacher := goproxy.DirCacher(cacheDir)
g.Cacher = dirCacher
g.GoBinEnv = append(
os.Environ(),
"GOPROXY=https://proxy.golang.org,direct",
"GOPRIVATE=go.mine.com",
)
g.ProxiedSUMDBs = []string{"sum.golang.org"}
IP := ""
fmt.Printf("Listening on ...%s:%s\nCache %v\n", IP, strconv.FormatInt(port, 10), cacheDir)
fmt.Println("GoBinEnv: ")
for _,v := range g.GoBinEnv {
if strings.HasPrefix(v, "GO") {
fmt.Printf("%v\n", v)
}
}

log.Println("running...")
if err := http.ListenAndServe(fmt.Sprintf("%s:%s", IP, strconv.FormatInt(port, 10)), &g); err != nil {
log.Println(err)
}

If I change the GOPROXY from this
"GOPROXY=https://proxy.golang.org,direct",
to this
"GOPROXY=direct",

then it say's downloading but then never completes.

[Aofei Sheng]

The issue is that if I push a new version of a module and try to perform go get then it doesn't find the new version unless I perform a go mod download first.

That's because of the caching mechanism of the upstream proxy you are using.

then it say's downloading but then never completes.

What do you mean by "never"? Does your go get operation get stuck?

I just tried the simplest program below and everything works fine for me.

package main

import (
        "net/http"
        "os"

        "github.com/goproxy/goproxy"
)

func main() {
        http.ListenAndServe("localhost:8080", &goproxy.Goproxy{
                GoBinEnv: append(
                        os.Environ(),
                        "GOPROXY=direct",
                        "GOSUMDB=off",
                ),
        })
}

[Gil]

The issue is that if I push a new version of a module and try to perform go get then it doesn't find the new version unless I perform a go mod download first.

That's because of the caching mechanism of the upstream proxy you are using.

then it say's downloading but then never completes.

What do you mean by "never"? Does your go get operation get stuck?

I just tried the simplest program below and everything works fine for me.

package main

import (
        "net/http"
        "os"

        "github.com/goproxy/goproxy"
)

func main() {
        http.ListenAndServe("localhost:8080", &goproxy.Goproxy{
                GoBinEnv: append(
                        os.Environ(),
                        "GOPROXY=direct",
                        "GOSUMDB=off",
                ),
        })
}

Yes it get stuck, using what I have below but with GOPROXY set to direct.
I'll try with "GOSUMDB=off" that you have above.

[Aofei Sheng]

It should be noted that the direct usually takes time (depending on your internet network connectivity). Maybe you should try go get a smaller module, like go get rsc.io/sampler.

[Gil]

I tried this now and I'm getting odd results same code as I originally posted but using direct
"GOPROXY=direct"
Command I try: go list -m -versions github.com/blang/semver
Results:
github.com/blang/semver v1.0.0 v1.0.1 v1.0.2 v1.0.3 v1.0.4 v1.1.0 v2.0.0+incompatible v2.0.1+incompatible v2.0.2+incompatible v2.1.0+incompatible v2.2.0+incompatible

If I change to "GOPROXY=https://proxy.golang.org,direct"
Command I try: go list -m -versions github.com/blang/semver
Results:
github.com/blang/semver v1.0.0 v1.0.1 v1.0.2 v1.0.3 v1.0.4 v1.1.0 v2.0.0+incompatible v2.0.1+incompatible v2.0.2+incompatible v2.1.0+incompatible v2.2.0+incompatible v3.0.0+incompatible v3.0.1+incompatible v3.1.0+incompatible v3.2.0+incompatible v3.3.0+incompatible v3.4.0+incompatible v3.5.0+incompatible v3.5.1+incompatible

If I go to the project in Github I can see there are other tags that aren't shown either way, if I'm using direct shouldn't these show?
v3.6.0 v3.6.1 v3.7.0 v3.8.0 v4.0.0

[Aofei Sheng]

These are all correct results.

The reason your direct result is different from the proxy.golang.org is because that module's author has deleted some tags (after the proxy.golang.org caches them) that existed before.

And you will never get v3.6.0 v3.6.1 v3.7.0 v3.8.0 v4.0.0 because of this.

[Gil]

These are all correct results.

The reason your direct result is different from the proxy.golang.org is because that module's author has deleted some tags (after the proxy.golang.org caches them) that existed before.

And you will never get v3.6.0 v3.6.1 v3.7.0 v3.8.0 v4.0.0 because of this.

Ah, thank you

[Gil]

Is there a way to get more verbose logging?

When I run go mod tidy on a new project I see lines like this
go: finding module for package github.com/xxx/utils/file
go: finding module for package github.com/xxx/utils/lock
go: finding module for package github.com/xxx/utils/shutdown
...
Then it gets stuck on this line:
go: downloading github.com/xxx/utils v0.0.0-20210903160913-e621d09ede7b

Using
"GOPROXY=direct", "GOSUMDB=off",

[Gil]

Ok I was attempting this on the Terminal inside of IntelliJ and it got stuck everytime.

I attempted to just try it from a regular command line using iTerm2 and it worked. Not sure what would cause it inside of the IntelliJ terminal but all environment variables are the same.

[Aofei Sheng]

What if you set Goproxy.GoBinMaxWorkers to 1?

[Gil]

What if you set Goproxy.GoBinMaxWorkers to 1?

Not sure what happened, I can't duplicate it now after restarting IntelliJ.
Thanks anyway.