Unable to generate mocks in reflect mode

Question

Unable to generate mocks in reflect mode

nezorflame opened this issue 4 years ago · comments

Actual behavior A clear and concise description of what the bug is.

My project uses Go modules, but due to the CI restrictions we also make use of the vendor folder.
So, our modules are synced by go mod tidy && go mod vendor command.

While using mockgen in a reflect mode, I'm unable to generate mocks. Can't pinpoint exactly when this became broken as this was working fine before, but it doesn't anymore.

The error message:

❯ go generate ./...

/Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test - service - doc.go
prog.go:12:2: cannot find package "." in:
        /Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test/vendor/github.com/golang/mock/mockgen/model
prog.go:12:2: cannot find package "." in:
        /Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test/vendor/github.com/golang/mock/mockgen/model
prog.go:12:2: cannot find module providing package github.com/golang/mock/mockgen/model: working directory is not part of a module
prog.go:14:2: cannot find module providing package github.com/nezorflame/mockgen-test/internal/client: working directory is not part of a module
2020/10/22 19:41:04 Loading input failed: exit status 1
internal/service/doc.go:2: running "mockgen": exit status 1

Seems to be somewhat similar to the #423 but the proposed solution didn't work for me.

Expected behavior A clear and concise description of what you expected to
happen.

Mocks should've been generated fine.

To Reproduce Steps to reproduce the behavior

I've published a test project which can be used to reproduce the issue.

mockgen is called during the go:generate directive at internal/service/doc.go:

//go:generate echo $PWD - $GOPACKAGE - $GOFILE
//go:generate mockgen -package $GOPACKAGE -destination mocks_test.go github.com/nezorflame/mockgen-test/internal/client Producer

The interface to generate the mocks upon is pretty simple and is stored at internal/client/contract.go:

type Producer interface {
	Produce(ctx context.Context, objects []string) error
}

So, the steps are:

Download the project
(Optional) Generate the mocks without the vendor:
a. Run go mod tidy && go mod vendor to clean up the dependencies
b. Run go generate ./... once - it'll work fine, mocks_test.go file will be generated
c. Delete mocks_test.go file to reproduce the error
Run go generate ./...
The error appears

It seems that the fact that vendor contains dependencies somehow breaks the ability of mockgen to generate the mocks, since the vendor-less mode works fine.

Additional Information

gomock mode (reflect or source): reflect
gomock version or git ref: v1.4.4
golang version: v1.5.3

go env output:

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/nezorflame/Library/Caches/go-build"
GOENV="/Users/nezorflame/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="<my_gopath>/pkg/mod"
GONOPROXY="my.company.com"
GONOSUMDB="my.company.com"
GOOS="darwin"
GOPATH="<my_gopath>"
GOPRIVATE="my.company.com"
GOPROXY="https://proxy.golang.org,https://goproxy.io,https://gocenter.io,direct"
GOROOT="<my_goroot>"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="<my_goroot>/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4p/q8s4q9s50qbdl5vllqg5fs000000gp/T/go-build294373709=/tmp/go-build -gno-record-gcc-switches -fno-common"

Cole Wippern · Answer 1 · Sat Oct 24 2020 05:41:28 GMT+0800 (China Standard Time)

The error message is due to github.com/golang/mock/mockgen/model not being vendored golang/go#41936

I would assume the reason you started seeing it at an indeterminate time has to do with them slowly making modules the default behavior in new versions of golang.

How to fix this is a bit more tricky.

The golang program generated by reflection mode depends on github.com/golang/mock/mockgen/model, but in most cases nothing else would depend on this, hence it not being included in the vendor folder.

There are probably some hacky work arounds like forcing the import with an _ "github.com/golang/mock/mockgen/model", but I'm not sure what the long term fix is.

cc @codyoss

Cole Wippern · Answer 2 · Sat Oct 24 2020 05:42:29 GMT+0800 (China Standard Time)

As super hacky work around you can copy the directory github.com/golang/mock/mockgen/model into your vendor folder

Cody Oss · Answer 3 · Sat Oct 24 2020 06:02:16 GMT+0800 (China Standard Time)

I think this is related to recent changes in Go tooling. Adding --build_flags=--mod=mod is one workaround. The other would be to include an empty import of that package so it gets pulled in import _ "github.com/golang/mock/mockgen/model".

Ilya Danilkin · Answer 4 · Tue Oct 27 2020 23:19:05 GMT+0800 (China Standard Time)

Thanks!
For now I'll use the proposed workaround with the empty import (and a comment to this issue), but it'd be nice to see this resolved.
Anything I could do to help further?

Cole Wippern · Answer 5 · Fri Oct 30 2020 04:23:00 GMT+0800 (China Standard Time)

Thanks!
For now I'll use the proposed workaround with the empty import (and a comment to this issue), but it'd be nice to see this resolved.
Anything I could do to help further?

I think what needs to happen now is some discussion about how to conceptually solve this problem; any input would be welcome and appreciated.

The basic overview (IIUC @codyoss can correct me) is that during reflection mode we dynamically generate some golang source code for a new main program. This main program, the reflection program, is the heart of what makes reflection mode work.

The source code is written out by mockgen and then mockgen executes some shell commands to compile and run the reflection program.

Problem we are hitting is in the dependency management; the reflection program requires github.com/golang/mock/mockgen/model but your code does not.

When the go toolchain attempts to compile the reflection program it sees your vendor directory, but github.com/golang/mock/mockgen/model is not contained and so it fails.

So we essentially have two different build environments that require different dependencies.

Valters Vingolds · Answer 6 · Sun Apr 25 2021 16:09:24 GMT+0800 (China Standard Time)

One of the problems is that the error message is so cryptic and unergonomic, it is impossible to figure out why this is happening. And the reason why (as comments above explain) is that that the mockgen/model directory never was vendored in the first place.

My workaround (strangely enough) was to remove the whole vendor directory, and this allows mockgen to work.

Cody Oss · Answer 7 · Sat May 15 2021 06:27:42 GMT+0800 (China Standard Time)

After looking more into this issue I am not sure there is too much we can do about this. For now I will treat this as a documentation issue and add some notes in the readme.

Caskey L Dickson · Answer 8 · Fri May 28 2021 18:25:33 GMT+0800 (China Standard Time)

If documentation is the solution you could add a pointer to the readme in the error as this is a known problem and the error message printed is a dead end for the user.

Cody Oss · Answer 9 · Sat Jun 05 2021 05:04:17 GMT+0800 (China Standard Time)

@caskey That is a great idea, done!

estebane-frb · Answer 10 · Thu Sep 15 2022 06:10:48 GMT+0800 (China Standard Time)

Hi there, any chance this has changed since last year? I'm new to GoMock but ran into this. Thanks for all you do for Go and the OSS community in general!

Ahmad Helaly · Answer 11 · Sun Nov 27 2022 05:12:15 GMT+0800 (China Standard Time)

Hey I also came across this issue today, Looks like installing the mockgen in GOPATH is not enough to make this work.

The ERROR I had

prog.go:12:2: missing go.sum entry for module providing package github.com/golang/mock/mockgen/model; to add:
        go mod download github.com/golang/mock
prog.go:12:2: missing go.sum entry for module providing package github.com/golang/mock/mockgen/model; to add:
        go mod download github.com/golang/mock
prog.go:14:2: no required module provides package github.com/escalopa/go-bank/db/sqlc: go.mod file not found in current directory or any parent directory; see 'go help modules'
prog.go:12:2: no required module provides package github.com/golang/mock/mockgen/model: go.mod file not found in current directory or any parent directory; see 'go help modules'
2022/11/27 00:08:24 Loading input failed: exit status 1

SOLUTION

Install the package using the following command

go get github.com/golang/mock/

The output should be like this

go: upgraded github.com/golang/mock v1.4.4 => v1.6.0

After so ran the mockgen command and everything worked. Notice Change command below to one that matches your project

mockgen -destination db/mock/store.go github.com/escalopa/go-bank/db/sqlc Store

As a result you should see the the following files have been updated

go.mod
go.sum

This is how to worked for me, Wish this might help 😄

Ryan Olds · Answer 12 · Tue Dec 27 2022 09:22:58 GMT+0800 (China Standard Time)

Ran into this and resolve it by creating ./tools/tools.go with the contents of:

package tools

import (
	_ "github.com/golang/mock/mockgen/model"
)

Then I was able to:

go get github.com/golang/mock/mockgen/model
go mod vendor
go mod tidy

Segayi Andrew · Answer 13 · Thu Mar 30 2023 17:48:06 GMT+0800 (China Standard Time)

thnks it worked for me