Giters

golang / go

The Go programming language

Home Page:https://go.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

cmd/go: exclude vendor directories from wildcard matches?

freeformz opened this issue · comments

commented
  1. What version of Go are you using (go version)? What operating system and processor architecture are you using?

go version go1.5beta1 darwin/amd64

  1. What did you do?

using this repo/branch as a test of vendor/: https://github.com/heroku/log-shuttle/tree/vendor2

GO15VENDOREXPERIMENT=1 go install -v ./...

What did you expect to see?

GO15VENDOREXPERIMENT=1 go install -v ./...
github.com/heroku/log-shuttle/vendor/github.com/pebbe/util
github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics
github.com/heroku/log-shuttle/vendor/github.com/heroku/slog
github.com/heroku/log-shuttle/vendor/github.com/bmizerany/aws4
github.com/heroku/log-shuttle/vendor/github.com/nu7hatch/gouuid
github.com/heroku/log-shuttle
github.com/heroku/log-shuttle/cmd/log-shuttle

What did you see instead?

GO15VENDOREXPERIMENT=1 go install -v ./...
github.com/heroku/log-shuttle/vendor/github.com/nu7hatch/gouuid
github.com/heroku/log-shuttle/vendor/github.com/bmizerany/aws4
github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics
github.com/heroku/log-shuttle/vendor/github.com/heroku/slog
github.com/heroku/log-shuttle/vendor/github.com/pebbe/util
github.com/influxdb/influxdb/influxql
github.com/heroku/log-shuttle/vendor/github.com/bmizerany/aws4/dydb
github.com/boltdb/bolt
github.com/gogo/protobuf/proto
github.com/heroku/log-shuttle/vendor/github.com/pebbe/util/isatty
github.com/heroku/log-shuttle
github.com/heroku/log-shuttle/cmd/log-shuttle
github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics/cmd/metrics-bench
github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics/cmd/metrics-example
github.com/armon/go-metrics
github.com/hashicorp/go-msgpack/codec
github.com/influxdb/influxdb/meta/internal
github.com/influxdb/influxdb/toml
golang.org/x/crypto/blowfish
github.com/influxdb/influxdb/snapshot
golang.org/x/crypto/bcrypt
github.com/golang/protobuf/proto
github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics/librato
github.com/hashicorp/raft
github.com/stathat/go
github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics/stathat
github.com/influxdb/influxdb/tsdb/internal
github.com/hashicorp/raft-boltdb
github.com/influxdb/influxdb/meta
github.com/influxdb/influxdb/tsdb
github.com/influxdb/influxdb/client
github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics/influxdb
# github.com/heroku/log-shuttle/vendor/github.com/rcrowley/go-metrics/influxdb
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:19: undefined: client.ClientConfig
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:38: undefined: client.Series
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:44: undefined: client.Series
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:52: undefined: client.Series
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:60: undefined: client.Series
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:70: undefined: client.Series
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:82: undefined: client.Series
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:93: undefined: client.Series
vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go:106: client.WriteSeries undefined (type *client.Client has no field or method WriteSeries)

I expected go to skip un-needed stuff (non imported packages) in the vendor/ directory. Perhaps this is a problem with my expectations, but it was a surprise.

commented

What package imports github.com/influxdb/influxdb/influxql ?

commented

@adg it's pulled in from vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go, which is not imported by the application. Technically I guess I could remove the file from vendor/, but I still think the behavior of ./... should not consider packages in vendor/ unless they are transitively imported by packages outside.

commented

That would require special-casing vendor in package pattern matching, which is something we decided not to do (at this point anyway).

commented

When I run go test ./..., it runs the tests for all of the packages in vendor/ as well as my project. Are you suggesting that this is the desired behavior (for now), @adg?

commented

@technosophos +1 on that problem.

@adg IMO that makes the vendor experiment nearly useless (for me anyway). I'll either need to make godep much more intelligent wrt package handling and/or try to discern what to build through additional package introspection. It's also just a bad user experience IMO.

commented

@technosophos yes, that is the correct behavior as of now.

@freeformz there is an easy work around. Instead of using ./... you can do:

go install $(go list ./... | grep -v /vendor/)

or use go test if you want to run tests.

commented

On the testing front, I think it is valuable to know that your dependencies' tests pass.

commented

Yeah, I don't find any of this particularly disconcerting. I just wanted to verify that it was the intended behavior.

commented

@adg Heh, thanks. I should have thought of that. :-(

Are there any plans to special case vendor/ at all, say in the 1.6 timeframe?

commented

Possibly. That's what this issue is now tracking. We're going to gather people's experience with the 1.5 vendoring experiment before making such plans.

commented

@freeformz After looking at this issue carefully, I would say the current behavior would be my desired behavior. While I understand Godep copies over the entire repository to vendor, I don't think that is what I would encourage. I think copying over each package as needed by the vendor tool is where the analysis work needs to be. At least that is what vendor will do.

commented

@kardianos The issue really has nothing to do with Godep, even though Godep's current behavior triggers it.

commented

A couple reasons I would prefer go test ./... not include things in vendor:

When I choose to take on a dependency, I've already verified it serves my need. That includes verifying automated tests are run by that dependency's overall project, running them myself, or similar. With things in vendor likely changing infrequently relative to my project it doesn't feel right to run the same tests for unchanged things every time. This is the same logic I use for feeling comfortable using, say, net/http in my project without running its tests every time.

Second, It's possible a dependency's tests will never build, work, or be relevant in my particular environment. This could be due to the way the tests are written, differences in developing vs deploying platform, etc. I develop projects on my OS X laptop that use libcontainer that would probably be good examples for this.

commented

@dpiddy Tests that are specific to an environment shouldn't run on a different environment anyway. If they try to run file a bug against the package. I would also argue when copying packages over the vendor tool user should decide if they want to bring over the test files or not

The case where my reasoning falls down is where people choose to use a vcs features like sub-modules.

(Not directly trying to argue here, just state other sides.)

commented

@kardianos good point, tools could do that.

Re the original issue of go install ./... installing things from vendor, that sounds really confusing and surprising. "Why did $GOPATH/bin/x change unexpectedly? Oh, I ran go install ./... in some completely unrelated project that vendors an old version of x three dependencies deep."

Vendoring tools could also skip main packages I suppose.

commented

@dpiddy Of note, I've had at least one person explicitly want to vendor "main" packages. What the kardianos/vendor tool does it to categorize packages by a "status":

Missing (packages referenced but not found)
Standard (in standard library)
Local (package is under "project" and not rewritten or in vendor dir)
External (package is outside of "project" but referenced)
Internal (package is rewritten under "internal" but still a vendor package)
Unused (package is rewritten under "internal" or in vendor folder but unused)
Program (package is a "main" package but rewritten or in vendor folder)
Vendor (package is in vendor folder)

These are two overlapping issues I hope to address soon (location and status), but that's the general idea. Then my tool can work on package status rather then just paths. For instance, if vendor added an install command, you could run "vendor install -status local".

I'm not convinced that adding a "status" concept to the go tool would ever be a desirable thing. "go" already supports "std". The syntax "go install ./...!local" could be interesting I think.

commented

This has come up repeatedly, but it seems like a bad idea.
Wildcards are wildcards.
Vendoring is vendoring.
They're completely separate things.
One should not influence the other.
If there is junk in a vendor/ subtree,
I think that's an argument for deleting, not for ignoring.

commented

I missed Andrew's comment earlier about using this issue to track feedback. That's fine, but if so it should not be marked as milestone Go 1.5. Reopening and reclassifying.

commented

@rsc Thanks for re-opening. I hope other's provide additional feedback ... My general thoughts on your last comments are: I "technically" agree with you, but it makes for a poor user experience.

commented

Having go test ./... run dependencies' tests seems attractive to me in theory, but I just tried it in practice and it was a pain in the ass. Mostly because it means you need to pull in the transitive closure of their dependencies, which is large and which godep doesn't seem to support well.

commented

@jacobsa Sure, but don't confuse the issue of "go test ./..." with a tool, godep. There (will be) many tools. There will be some that analyze dependencies well.

commented

You're right, sorry. In theory the current tool support isn't so important, but nevertheless it still is a pragmatic issue today (since as far as I can tell godep is the only tool that comes close to supporting the expected format today).

commented

My expected behavior from other languages would be that all wildcard matching would by default apply, but the user should have the ability to easily exclude a folder in a CLI flag or dot file, like gitignore.

git add . takes into account what I've ignored even though it essentially wildcards the whole subtree.

go test ./... --ignore ./vendor wouldn't include packages in vendor.

If not done in the Go tool itself, try to make the "right" values part of the public API so it can be easily extended.

My 2c, Thanks!

commented

I'd like to reopen the discussion on naming the vendor folder, _vendor as
many tools already ignore this pattern.
On 28 Jul 2015 8:45 pm, "Jamie Stackhouse" notifications@github.com wrote:

My expected behavior from other languages would be that all wildcard
matching would by default apply, but the user should have the ability to
easily exclude a folder in a CLI flag or dot file, like gitignore.

git add . takes into account what I've ignored even though it essentially
wildcards the whole subtree.

go test ./... --ignore ./vendor wouldn't include packages in vendor.

If not done in the Go tool itself, try to make the "right" values part of
the public API so it can be easily extended.

My 2c, Thanks!

—
Reply to this email directly or view it on GitHub
#11659 (comment).

commented

@jacobsa Please look into github.com/kardianos/govendor . It now supports the vendor folder as well. It also flattens any nested dependencies and only picks packages that you use.

commented

@kardianos: Thanks, I quite like the tool. It also imports tests that I don't want; I'll file a bug.

commented

github.com/kardianos/govendor can now ignore test files and arbitrary build tags. If you don't want to run tests, then we don't vendor them.

commented

btw.. for those impatient, this helps run tests and excluding vendor/..

GO15VENDOREXPERIMENT=1 go test $(GO15VENDOREXPERIMENT=1 go list ./... | grep -v '/vendor/')

go list does adhere to the env and excludes the vendor/ directory.. pretty inconsistent / busted implementation :/

commented

@pkieltyka go list ./... actually does include the vendor directories. It is grep that is removing them:

-v, --invert-match
Selected lines are those not matching any of the specified patterns.

commented

Thanks for teaching me about grep, but the point is that it shouldn't include the vendor directories on go test ./...

On Aug 23, 2015, at 6:41 PM, Nathan Youngman notifications@github.com wrote:

@pkieltyka go list ./... actually does include the vendor directories. It is grep that is removing them:

-v, --invert-match
Selected lines are those not matching any of the specified patterns.

—
Reply to this email directly or view it on GitHub.

commented

The documentation for go list says:

Usage:

go list [-e] [-f format] [-json] [build flags] [packages]

List lists the packages named by the import paths, one per line.

The default output shows the package import path

Running go list all will list all Go packages that are inside /vendor/ folders for use with GO15VENDOREXPERIMENT=1.

However, github.com/foo/bar/vendor/github.com/foo/baz is not a valid import path. You can't import it, you can't build it, you can't install it.

Similarly, running go build github.com/foo/bar/... or go generate github.com/foo/bar/... used to be completely normal and produced expected results in Go 1.4, but as of Go 1.5 it will do unexpected things and fail for repos that have a /vendor/ directory. Setting or not setting GO15VENDOREXPERIMENT has no effect here.

Import path matching and general importing of Go packages uses the following rule for ignoring certain paths:

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

Source: http://golang.org/cmd/go/#hdr-Description_of_package_lists.

In order to resolve this problem for vendored Go packages inside /vendor/ folder (if that decision is made), I see two approaches:

  1. Modify the rule to ignore directories and file names that begin with ".", "_" and directories named "testdata" as well as "vendor".
    • This would have to be done by the go tool as well as every tool in the wild that implements this rule. Tools would need to be updated for Go 1.5 and for GO15VENDOREXPERIMENT even if those people are not interested in using GO15VENDOREXPERIMENT=1 themselves.
  2. Use the existing rules to hide contents of /vendor/ folder. For example, by renaming it to /_vendor/.
    • Existing tools will continue to work on non-vendored packages just fine, but will ignore contents of vendored packages. Optionally, support for GO15VENDOREXPERIMENT=1 could be added; those tools can then treat /_vendor/ specially.
commented
commented

I don't want to add more special cases to cmd/go.

I agree. I would prefer changing the newly added (and still experimental) GO15VENDOREXPERIMENT to use the existing rules of go tool, for example by using the folder name /_vendor/.

I think between the two, it's only acceptable option, given that GO15VENDOREXPERIMENT is still an experiment and it's meant to be opt-in. Using a folder /vendor/ means it's not opt-in for tools that are unaware it's actually a special folder when GO15VENDOREXPERIMENT=1 is set.

Normally when you're working on one package, you don't need to test
everything, only the actually affected packages. Using ./... is pretty
overkill.

Most of my personal git repositories contain 1-6~ Go packages. Depending on the exact project, they may be direct subfolders in the git repo root, or they may be a few levels deep. So, when I'm in the repo root of a random project github.com/shurcooL/randompkg, I really appreciate the ability to do things that affect all Go packages inside that folder (since they're all related to the project I'm working on), rather than having to remember individual Go packages and run commands on them individually (waste of my time, plus risk of forgetting some Go package).

In fact, the ... pattern matching is one of my favorite aspects of the go command and I use/rely on it daily, upwards of hundreds of times. I don't think I abuse it, I think I use it to a great benefit. Without it, I would feel like I'm back in C++ land where I have to write and maintain Makefiles that explicitly list subfolders in a given project.

commented

Well, I think this is a pretty obvious one if developer experience matters

On Aug 23, 2015, at 9:09 PM, Dmitri Shuralyov notifications@github.com wrote:

I don't want to add more special cases to cmd/go.

I agree. I would prefer changing the newly added (and still experimental) GO15VENDOREXPERIMENT to use the existing rules of go tool, for example by using the folder name /_vendor/.

I think between the two, it's only acceptable option, given that GO15VENDOREXPERIMENT is still an experiment and it's meant to be opt-in. Using a folder /vendor/ means it's not opt-in for tools that are unaware it's actually a special folder when GO15VENDOREXPERIMENT=1 is set.

Normally when you're working on one package, you don't need to test
everything, only the actually affected packages. Using ./... is pretty
overkill.

Most of my personal repositories contain 1-6~ Go packages. Depending on the exact project, they may be direct subfolders in the git repo root, or they may be a few levels deep. So, when I'm in the repo root of a random project github.com/shurcooL/randomfoo, I really appreciate the ability to do things that affect all Go packages inside that folder (since they're all related to the project I'm working on), rather than having to remember individual Go packages and run commands on them individually (waste of my time, plus risk of forgetting some Go package).

In fact, the ... pattern matching is one of my favorite aspects of the go command and I use/rely on it daily, upwards of hundreds of times. I don't think I abuse it, I think I use it to a great benefit. Without it, I would feel like I'm back in C++ land where I have to write Makefiles that explicitly list subfolders in a given project.

—
Reply to this email directly or view it on GitHub.

commented

+1 to @shurcooL's thoughtful comments. In particular, I use ./... all of the time for testing the implementation details of a command and how they work together.

commented
commented

I would second @davecheney suggesting bringing the discussion back up around renaming the vendor directory to _vendor which @shurcooL also mentions above. Like others I believe the dependency problem should be solved utilizing the conventions already established if possible and _vendor seems like a plausible way to do that.

commented

@jackspirou: Agreed, that's the most elegant solution I've heard so far.

commented

Testing

On the testing front, I think it is valuable to know that your dependencies' tests pass. @adg

Agreed. Since I began using GO15VENDOREXPERIMENT=1, I began noticing failing tests in dependencies I hadn't checked before. It's something I should either look into or reevaluate which dependencies I'm using.

Mind you, it could still be possible to test vendor/ dependencies independently of go test ./.... Though in the little testing I did with a hypothetical _vendor folder, I did run into various issues.

I think copying over each package as needed by the vendor tool is where the analysis work needs to be. @kardianos

This makes a lot of sense to me. If you want to vendor _test.go files, it may be necessary to vendor their assertion and test packages as well (like go get -t). If not, just delete the _test files or have the vendor tool provide that option.

Installation

The original issue is with regards to installation, not testing. Using ./... with go install may be asking for trouble, but I do find it useful:

cd x/tools
go install cmd/...

I guess the @kardianos approach is to have the vendor tool or person vendoring a library be aware of main packages and not copy them if not needed.

Deciding to use vendored tests and main packages

So which is better:

  • Making the decision once when vendoring files as to which files (tests/cmds) are included?
  • Deferring the decision to every test/install command (whatever the default may be)?
commented

@shurcooL I see this as opt-in per project, not per user. If a project opts-in, then all the users that use it need to opt-in when developing it.

@nathany Good summary. One additional point I'd like to make is how vendor code is handled. Many people like to think of vendor code per repository, tools are easier to build using this assumption. I like to think of vendor code per package. The difference is extremely important. When thought of as per package it becomes natural to exclude main packages unless they are requested. It is also a natural extension to exclude files based on if they are test files or other build tags. But if you think of vendor code versions and copying as per repository choosing the packages you want out of the mix becomes much more problematic.

commented
commented

Renaming the vendor directory to _vendor will mean that 'go test _vendor/...' won't work anymore.

I believe that is the goal, people using vendoring do not want to test their vendored dependencies

commented
commented

I believe that is the goal, people using vendoring do not want to test their vendored dependencies

What about a situation when there is a bugfix applied to a vendored package that cannot be taken from upstream (for whatever reason). It seems to be useful to run tests for vendored dependencies in this case.

commented

I do not want tests / main packages in vendor/ compiled or executed when ./... is specified, but any packages contained inside of vendor searched and used as necessary. My main motivation for this is so that building / testing can be done simply at the repo level (go install -v ./... & go test -v ./...) w/o having to specify the local packages to build / test via Makefiles or similar means.

I do not want to have to rely on a specific vendoring tool to accomplish this either. I don't think go should require a specific vendoring tool. (PS: I applaud govendor / @kardianos for implementing so much.)

If I want to run the tests for my vendored dependencies (which I sometimes do), I do that by checking out a version, reading their docs and running their tests in whatever manner is necessary. Sometimes it's not as simple as go test -v ./..., but I think we should strive to make it that easy. ALWAYS.

Renaming vendor/ to _vendor/ I believe accomplished this and IMO is the path to least surprise for users.

commented

What if support was provided for both vendor/ and _vendor/?

Since there seems to be use cases for both could support be built for both? This way anyone can choose how they would like to treat testing vendored dependencies for a specific project. This might mean that the future GO15VENDOREXPERIMENT functionality would work with both _vendor/ and vendor/.

If vendor/ is present go test ./... will test vendored packages and if _vendor/ is present go test ./... will skip over third party vendored packages. If both _vendor/ and vendor/ is present then an error is thrown. Perhaps a command could also be included that makes a go project's vendored packages testable by renaming _vendor/ to vendor/ (not that renaming it by hand is difficult, just a thought for automation purposes).

I think allowing both vendor/ and _vendor/ would be a simple solution that leverages established go conventions. Giving options to different projects might be useful if the value gained from testing third party vendored packages varies from project to project.

While I say "simple" above, I do understand giving this kind of flexibility to users may have unintended consequences. Go, being known as very opinionated, usually seems to have one way of solving a particular problem. Maybe allowing vendor/ and _vendor/ is too much flexibility, or maybe it is just the right amount?

commented

What if support was provided for both vendor/ and _vendor/?

No thanks. Then I and all other users don't need to read the following wall of text to understand the difference. Having two nearly-identical ways doing things is not in the Go spirit.

commented

FWIW, I prefer just vendor/

commented

Removing the ability to test your dependencies entirely is not the same as having the ability to not test them.

As a CI user I would want to be able run a minimal set of tests for pull requests that won't hit production, to give me a good feeling that this PR doesn't break existing functionality.

However, before doing a build that would hit a prod server I want the ability to do a full regression of all the sub packages. To make sure that we aren't deploying a version of a dependency that has failing tests.

I also would like to be able to this while being able to keeping the command simple, and not having to explicitly list each dependency, as having to enter our CI tool to update the build scripts on every change would not be ideal.

I would be fine / great with making it a requirement that if you wanted to skip vendored tests that you must use go list to feed the go test list of packages. go test ./... would be the method of running all tests, including the vendored ones.

commented

@itsjamie fair points. this would work too if ./vendor was just ignored in ./...

go test ./... && cd ./vendor && go test ./...

solves both cases

commented

this would work too if ./vendor was just ignored in ./...

If ./vendor is ignored why not just rename it to _vendor to work with other tools and maintain current conventions as @davecheney and @shurcooL have mentioned above?

Then your example would be go test ./... && cd ./_vendor && go test ./... which does not break current conventions.

commented

@itsjamie

I'm not sure how that removes the ability to test dependencies. It just makes them 2 explicit operations, which could be chained together when it's appropriate to do so:

$ go test -v ./... && cd _vendor && go test -v ./...
commented

All sounds good to me.

commented

If you guys decide on _vendor, then I'll support it. I just prefer ./vendor aesthetically, I realize it doesn't make sense beyond that. But aesthetics do matter :)

commented

I also think that go test ./... et al. should ignore the vendor folder by default.

Dependencies should not be treated like they are my own project. Installing my project should not install all dependencies. Testing my project should not test all dependencies.

Before, my dependencies were in $GOPATH, and I did not install with go install or test them with go test, golint them, etc.

All of a sudden, instead of using the good old go test ./... etc, I have to use a wrapper script or a clumsy subcommand with go list ./... | grep -v /vendor/, update all my CI scripts, etc.?

commented

Including ./vendor/ in wildcard commands like go list ./... significantly increases the duration of the command when there are a lot of dependencies (obviously).

While I agree with @rsc that wildcards are wildcards and vendoring is vendoring, including dependencies in a wildcard seems like a significant unintended consequence.

I like the aesthetics of using ./vendor/ as well, but think that the functionality of ./_vendor is what people coming from previous vendoring projects, like godep, are expecting.

I guess my suggestion is that if $GO15VENDOREXPERIMENT=1 then ./vendor should be treated specially and excluded from wildcards.

commented

Honest question: Do people plan to vendor packages by hand?

If they do, these questions have merit. If they plan to use a tool, let the tool do the work after the developer decides what they want to test and what they want to not test. Some will want to run dependency tests especially when if multi-platform. Others may decide they don't want the tests. Then the default behavior falls out.

Do people really not want to make their tools do this work?

commented

I think people do plan to use tools to prepare the vendor directory. For ex, I've written a super crappy one to copy packages from $GOPATH to ./vendor so I can get by: https://github.com/pressly/gv ..which I trigger through a Makefile like in https://github.com/pressly/imgry.

.. the challenge is, once a tool prepares the vendor directory, how will that tool fix go test ./... ? Try to design one and offer some suggestions

On Aug 26, 2015, at 6:34 PM, Daniel Theophanes notifications@github.com wrote:

Honest question: Do people plan to vendor packages by hand?

If they do, these questions have merit. If they plan to use a tool, let the tool do the work after the developer decides what they want to test and what they want to not test. Some will want to run dependency tests especially when if multi-platform. Others may decide they don't want the tests. Then the default behavior falls out.

Do people really not want to make their tools do this work?

—
Reply to this email directly or view it on GitHub.

commented

@pkieltyka Maybe I suck at marketing... I've made one that can remove tests here: github.com/kardianos/govendor . Not trying to promote it here, but once you have a tool that works well, adding a test filter isn't too hard. It only copies over needed or declared packages so go install ./... also works because main packages aren't copied over by default either. I'm sure other tools could do similar things.

commented

One other thing to keep in mind. As much as we want everyone to use Go 1.5
immediately, experience has shown that there are many go programs out there
being built with older versions on a if it ain't broke, don't touch it,
basis.

For those older versions, and more importantly library writers, who want to
adopt the versioning experiment in their code, using _vendor would allow
those library writers to try this feature without being overwhelmed with
support requests from users on older versions.

Thanks

Dave

On Thu, 27 Aug 2015 09:21 Daniel Theophanes notifications@github.com
wrote:

@pkieltyka https://github.com/pkieltyka Maybe I suck at marketing...
I've made one that can remove tests here: github.com/kardianos/govendor .
Not trying to promote it here, but once you have a tool that works well,
adding a test filter isn't too hard. It only copies over needed or declared
packages so go install ./... also works because main packages aren't
copied over by default either. I'm sure other tools could do similar things.

—
Reply to this email directly or view it on GitHub
#11659 (comment).

commented

@davecheney I understand you are saying using "_vendor", which existing go tool will ignore, will result in fewer support requests to package authors. But the same is true for the go 1.5 tool as well in the context of this issue because go test ./... and go install ./... will also do the same thing as older versions. So while I think I understand what you are saying, I don't follow your logic.

It would appear you think that tools shouldn't exclude test files should the author request it. Is that true? Could you explain why?

commented

@kardianos

But the same is true for the go 1.5 tool as well in the context of this issue because go test ./... and go install ./... will also do the same thing as older versions. So while I think I understand what you are saying, I don't follow your logic.

I don't believe this is true, Go 1.4 will see .../vendor/... as a valid path and attempt to build, test and install code it find inside it. If the path were .../_vendor/... it would be ignored by Go 1.4.2. Am I misunderstanding how Go 1.4 works ?

It would appear you think that tools shouldn't exclude test files should the author request it. Is that true? Could you explain why?

I am sorry that I gave that impression. I am saying the opposite.

commented

@kardianos I don't want to limit Go users into having to use a specific tool in order to properly vendor things, unless that tool also ships as part of the go toolchain. FWIW: I'd be perfectly happy if govendor was that tool, although it doesn't provide for all of my needs.

commented

@freeformz I'm not suggesting using a single tool. I'm suggesting other tools work a little bit harder.

commented

Any chance we could see a change to _vendor in 1.5.1? Would a CL for that be helpful?

commented

@dpiddy, we have an extremely high bar for changes in point releases (1.n.m): security bugs, or bugs with no possible workaround, and only if they're a new regression (not something we've had for the past few major releases). Tweaking an experimental feature meets none of the criteria.

commented
commented

@minux It also doesn't scale well to have to list every package by hand in a Makefile (for example) so that make test will test all non-vendored packages. That introduces a human element that will certainly be forgotten. Obviously this can be scripted, but it is very ugly. Here are some lines of a Makefile I have for a go1.5 vendored project:

ALL_DIRS=$(shell find . \( -path ./vendor -o -path ./.git \) -prune -o -type d -print)
GO_PKGS=$(foreach pkg, $(shell go list ./...), $(if $(findstring /vendor/, $(pkg)), , $(pkg)))

lint:
    @golint ./... | grep -v '^vendor\/' || true
    @go vet ./... 2>&1 | grep -v '^vendor\/' | grep -v '^exit\ status\ 1'

test:
    @for pkg in $(GO_PKGS); do \
        cmd="go test -v $$pkg"; \
        eval $$cmd; \
        if test $$? -ne 0; then \
            exit 1; \
        fi; \
    done
commented

This isn't just about testing. Here are my thoughts to date....

When vendor/ is the name of the vendoring directory and a package spec of ./... matches everything in that directory as well...

The issues I have with this are:

  1. Most people I've talked to don't expect the package spec ./... to match stuff in vendor/. This indicates to me that it's counter intuitive. I will likely have to do a lot more education about packages / paths / etc. This isn't really a bad thing, but most people I talk to are already confused and/or frustrated by the way $GOPATH works. I don't want to add to that confusion.
  2. It's impossible to test only the packages that I'm writing with a repo level ./.... I care about testing the packages I'm writing more regularly than testing the stuff I've vendored. I'll test the stuff I vendor when I originally vendor it and/or when I upgrade the vendored deps. IMO/E more people follow this workflow than the test everything all the time workflow. I'm worried that this will end up leading people to encode test Makefile or similar targets to avoid having to remember / type the go test $(go list ./... | grep -v vendor/) stuff. I thought we wanted to avoid that?
  3. Additionally some package's tests require additional setup which I may not have on my local machine / environment.
  4. I will require additional meta data about what to install when someone pushes code to me. Right now I can assume that go install ./... will install all of the packages in the pushed repo. Godeps handles this case just fine by recording the package spec(s) used during godep save, but I can tell you most people just use ./...
  5. Similarly for projects with multiple main packages for commands, it will require additional documentation where it's now possible to simply say go get github.com/my/proj/.... This is probably pretty minor though.
  6. As per @davecheney's comment: Not everyone is using Go 1.5. vendor/ will cause issues for users of Go versions < 1.5.

Some, but not all of the above stuff can be handled by vendoring tools. IMO if we're going to cause this much disruption then Go itself should provide a vendoring tool such as govendor/godeps/something new/whatever as part of the standard tooling (i.e. go vendor).

commented

I second @freeformz's points FWIW..

commented

@minux

And If you just want to run all the tests, I think knowing that the vendored packages also works correctly is also desirable.

If I add dependency X that is only a sub-package of a larger package Y, the whole thing ends up in the vendored folder, even if I only need the subpackage and its dependencies. If I test the whole project with go test ./..., it tests the whole package Y. I don't need to test all of Y, and worse, it will fail if I did not add its recursive dependencies, which I don't need for package X.

You may ask why I add the all of Y if I only need X. The reason is that I use git submodules to add the repo, and go get behaves the same way.

commented

I share all of @freeformz's concerns. I think it is telling that at least godep has always used an ignored _workspace path, even when using godep save -r which made saved dependencies valid import paths as the vendoring experiment does.

@bradfitz that is what I figured. I fear if it takes a major release to change the experiment's behavior enough momentum (whether because it's good or not) will be built with the current behavior that it will be unlikely to ever change.

commented

My main concern, before I can start to think about others, is that of the import paths. As I understand example.com/foo/bar/vendor/example.com/foo/baz is not a valid import path. example.com/foo/baz is a valid import path of a Go package that happens to be vendored inside example.com/foo/bar.

Are other people okay with the idea that go list all will print out a bunch of invalid import paths (in addition to valid ones)? How should other tools deal with these invalid import paths?

I thought the point of vendoring was to put a copy of the original package (e.g., example.com/foo/baz) inside your VCS tree, but not change it in any way. No import path rewriting required - the vendored package still has the same original import path. You should be able to vendor/not vendor any package as you wish. It seems to do that but also change import paths of the packages being vendored.

commented

I've been dealing with the repercussions of the vendoring experiment for a few weeks now. I like having fine-grained control over my dependencies, and it’s awesome to be able to commit the vendor source along with my project source without having to change all the vendored import paths.

I think I voice the same concern as most of the non-googlers here, which is that I didn’t expect to have golint, go vet, and go test blow up in my face when I moved everything to the vendor directory. It’s frustrating to have to contort my workflow to accommodate external code that didn’t previously require it.

However, I wouldn’t have known about many of these problems with the third-party libraries had I not moved them into the vendor directory. It’s been a pain to get everything to play nice, and I understand that not everyone has the luxury of being able to fix other-people’s-code, but the new methodology incentivizes quality code… which is one reason I believe Go is as successful as it is today.

If I compare the two solutions from a long-term perspective, I see two very different outcomes. Allowing people to ignore bad code results in a complex solution whereby some code is whitelisted, some is blacklisted, the lists require regular maintenance, and you have no guarantee of a minimum level of quality. The other solution is to develop good code, fork bad code, and take the penalty of due diligence up front.

I’m not saying either solution is right or wrong, but since this issue is still open for feedback, I’m of the opinion that the short-term pain of broken libraries will eventually subside as the authors feel the pain and/or new authors emerge. I believe that the long-term result will be a better foundation from which to build future solutions.

commented

I was asked to provide my opinion here, so here it is.

I agree with @rsc's comment in #11659 (comment).

Further, it seems renaming the directory to _vendor would make it harder to operate on the vendored packages. For example, it's useful (at least to me) to be able to run go list ./vendor/... and see everything that's been vendored.

Personally, I think the current behavior is ok. It hasn't really been a problem for me (though I understand why some find it inconvenient). For example, when I run interactive commands to test or vet packages, I type go test or go test ./foo ./bar or maybe go test ./foo/... since I generally want to test only one or two packages at a time. Our automated testing system tests all our packages, using a command like go test $(go list ./...|grep -v vendor). This would be inconvenient to type by hand, but there is more than one way to make it convenient. Complicating the behavior of the go command probably isn't the best way.

commented

Thanks for all of your input. I think everything has been covered here that could be covered.

I'm satisfied with the outcome, although I still think it ignores usability a bit, so I'm going to close this issue (since I opened it).

commented
commented

That doesn't help with the issue where go list all outputs invalid import paths (sorry, I'm repeating myself).

commented
commented

But I said go list all specifically, not just go list.

I have multiple tools that use the output of go list all to navigate and open Go packages. For example, if I want to go to github.com/russross/blackfriday, I expect there to be just one. With vendoring as is, there may be 10, 20 or 30+ matches for blackfriday in various vendor folders.

All tools that prefer to preserve previous behavior of not including vendored package copies (e.g., like they did not include the vendored packages in Godeps/_workspace subfolders) need to be updated to perform go list all and then try to filter out vendored copies. Even if GO15VENDOREXPERIMENT is not enabled.

commented

@shurcooL In hopes of being helpful, you might try to migrate your tools to use something else. For instance you could try http://godoc.org/github.com/kardianos/govendor/context create a new context then call ctx.Status(), then filter all the returned items by their "status" which includes if they are in a vendor directory or not. If that doesn't quite fit your need, but is close, feel free to open an issue.

commented

I have been doing research and setting up a continuous integration & testing pipeline for Go during most of the day, on different tools, and this is an issue that really bothers me.

I love Go for it's minimalist and clean approach. In line with that, I use the clean architecture approach (or something close to it) for most of my projects. This means I will usually have four subpackages, with the main package in the root folder. Sometimes, there are some small additional tools in the cmd folder as well. I was under the impression that this is a configuration that most projects use.

I now have to use a dirty workaround to get CI to build and test properly, all due to the fact that ./.. includes folders in vendor. This is counter-intuitive and not well thought-out at all. Which is contrary to what I've come to expect and appreciate about Go. The default case should not require any workaround.

There should be a coherent approach where all go tools exclude vendor when using ./... if the GO15VENDOREXPERIMENT environment variable is set to 1. That would be consistent. As someone mentioned before, you can still do "cd vendor && go test ./..." if you want to include the subpackages. But you will have to be explicit about it, which is fine since it constitutes a non-default workflow imo.

If the go tool deals with vendoring using the environment flag (and by default in Go 1.6), then it should also deal with the resulting consequences and solve them in a coherent way. Let's keep it simple, folks.

For now, I'm forced to switch back to godep to keep a clean non-hacky script in my repository.

commented

I think there are two cases:

  1. A package in a vendor tree that is actually used (directly or
    indirectly) by the code outside the vendor tree. In this case I think you
    would want to test the package, just as you test all your other
    dependencies. If code you depend on is broken, you want to find out in the
    most precise test possible, not wait until a test of the main program fails
    mysteriously.
  2. A package in a vendor tree that is not actually used (directly or
    indirectly) by the code outside the vendor tree. In this case, obviously no
    one cares and the test need not run. But in this case I don't understand
    why tools are bringing that package into the vendor tree in the first
    place. I would argue that, for many reasons, dead (unused) code should not
    be copied into a vendor tree. Code that is in your vendor tree has to be
    maintained, updated, and so on. If it's not needed, it should just be
    omitted.

I think it's a mistake to skip the test in (1). And I think (2) indicates a
problem in the vendoring process.

The counter-argument to this is that you can't omit individual packages
when you're using git submodules. But when you're using git submodules I
think you're tying yourself to the target repository quite a bit more and
so it may still be appropriate to test the entire thing.

commented

@rsc I disagree even with your first point. If you vendor your dependencies, you will usually pin a specific version, so it is nonsense to run the tests each and every time you push a commit and your CI kicks in. You should test your dependencies once, manually, each time you update them. That's completely besides the point of automatic testing of your own commits and pull requests.

commented

Agree with @awishformore, similar to what I said in #11659 (comment).

How many projects that use net/http run those tests every time? Would that be reasonable? I don't think so, because presumably you've verified that the Go release you're using is good and you know net/http isn't changing out from under you.

I feel things in vendor/ can be treated similarly: on add/change, verify it works for you, then be sure the vendored code doesn't change.

commented

As others have noted, there is an easy workaround if you just want to test
everything but vendor:

go test $(go list ./... | grep -v /vendor/)

Depending on your configuration, it may be appropriate or worthwhile to use
that in your CI system. My point is that it may also be appropriate or
worthwhile to include those tests, so it is not appropriate for ./... to
categorically exclude certain directories. Different people use vendor
differently. Some people do make local changes, or do want to include it in
"go vet ./..." or other commands.

I also still think that for the specific case of testing, if we could solve
test caching then that would help considerably.

commented

@rsc Please don't qualify it as an "easy" workaround. It's a dirty workaround for something that should work out-of-the-box. Why? Because someone decided that the Go tool should deal with vendoring, and since it does, the default way should correspond to the most intuitive and common approach. The workaround should be required if you do something out of the ordinary, like testing dependencies on each build or, like you mention, customizing dependencies. In that case, a workaround (i.e. cd vendor && go test ./...) seems perfectly fine. I don't understand why you argue to make the default more annoying in favour of some edge cases. One of the most wonderful things about Go is that defaults usually work in intuitive ways and hardly ever annoy me.

commented

You should test your dependencies once, manually, each time you update them.

It's more convenient and simpler (for both the user and for the go tool's implementation) to test them every time you run your own tests. I'd love to know, concretely, why some folks are so resistant to this idea. Is it because running more tests is slower? (If so, see @rsc's last comment above.)

It's a dirty workaround for something that should work out-of-the-box.

From my point of view, it does work out of the box. That's because I like to run the tests for my dependencies. Of course, anyone is free to prefer otherwise, but that preference doesn't necessarily imply that the go tool's design is wrong.

commented

That's because I like to run the tests for my dependencies.

In case it's helpful, here's an easy workaround if you want to test dependencies:

go test $(go list -f '{{join .Deps "\n"}}' ./...)
commented

@kr build time is certainly a factor with current Go versions. I also gave my reasons in #11659 (comment) and other comments.

Perhaps my mental model of what vendor/ is is just wrong. I consider it to be an extension of the stdlib for my project. If that is true, should I be running tests for everything I've used from the stdlib every time I test my project?

commented

Of course I'm forgetting that an easy way to exclude vendored tests is to do just that: exclude them when vendoring things. At least godep and govendor already support this. Same for unwanted main packages.

Curious to see how things like go install github.com/foo/bar/... play out once 1.6 is released.

commented

The fact that everyone is proposing workarounds just points to the fact that this should be fixed. Renaming it to _vendor just works. If you want to test or install stuff from your vendor directory, you can by just cd'ing into that directory.... but if you don't want to, then you can go whatever ./... just like you do today, and it'll work just like it does today. Making it "vendor" breaks things. I can't imagine anyone wanting that to be the default.

Keep go simple. Make the vendor directory _vendor so that tools skip it. I shouldn't have to use some annoyingly arcane command to just build and test my code.

commented

"vendor" is not being renamed. That ship has long sailed.

I think everything that can be said on this issue has been said in the many comments above. I think it's time we move this to other forums (for general discussion) or new bugs (for specific new bugs that aren't just a dup of this one). Ultimately this bug is a policy/behavior question and not actually a bug report.