No such file or directory: '.git/info/exclude'

[Amin Yahyaabadi]

I updated my bazel-compile-commands-extractor recently. When I build my code inside Gitlab, I get the following error. This doesn't happen locally. Do you know what is going on? a+ should open the file if it doesn't, but it fails for some reason.

INFO: Running command line: bazel-bin/refresh_compile_commands '--config=release_lto' '--compilation_mode=opt'
Traceback (most recent call last):
  File "/home/sai/.cache/bazel/_bazel_aminya/f8e760a4c7b015c873892d25be2d4f05/execroot/test/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/test/refresh_compile_commands.py", line 1062, in <module>
    _ensure_gitignore_entries_exist()
  File "/home/sai/.cache/bazel/_bazel_aminya/f8e760a4c7b015c873892d25be2d4f05/execroot/test/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/test/refresh_compile_commands.py", line 1027, in _ensure_gitignore_entries_exist
    with open(hidden_gitignore_path, 'a+') as gitignore:
FileNotFoundError: [Errno 2] No such file or directory: '.git/info/exclude'

I have fixed it by adding the following to the CI scripts, but not sure why the error is happening:

      mkdir -p ./.git/info/
      touch ./.git/info/exclude

[Chris Sauer]

Hello again, @aminya! Thanks for writing in to report--let's track this one down together.

First off, can I ask you to confirm that you're using the latest version of this tool? Near the start of March, we added support for git worktrees in response to requests, and older versions of the tool would show an error like this if gitlab ci were using work trees. (More details on the change history if you want them: #103)

If you're already using the latest:
I think we're getting the FileNotFoundError because the parent directories don't exist in gitlab's environment.

Could you check that assumption by seeing if things still work without the touch line? [If not, let's think about next steps. If so, could you poke around a bit more, seeing, e.g., if the .git directory exists, checking it's not a link, etc?]

If that assumption is correct, that's still surprising, since we'd only get to that line if git reported that we were in a repository (via git rev-parse --git-common-dir). Any special details we should know about how gitlab clones things to sets up CI--so we can figure out how to handle that case? [Very worst case we could just fail gracefully with a warning, but it'd be much better to understand and fix than just plaster over the issue.]

Thanks,
Chris

[Chris Sauer]

Hey @aminya! Checking back in on this. Any updates?

[Amin Yahyaabadi]

Hey @cpsauer, sorry for the delay, I was on vacation.

Thanks for the suggestion and detailed answer! I tested the pipeline without touch ./.git/info/exclude, and it worked without any issues. So the problem seems to be that the .git/info folder doesn't exist for some reason when Gitlab fetches the repository.

[Chris Sauer]

heh, np, same it turns out! I just got back from my first trip to Puerto Rico. Where'd you go, man?
Could I still ask you to do the other poking around from the earlier comment, to try to figure out what's up? (i.e. check you're using the latest, investigate the .git dir (or like, tree it), rev-parse, etc?

[Amin Yahyaabadi]

Nice! I was in Montreal.

Yes, I am using the latest version. I'll let you know the result for .git dir.

[Amin Yahyaabadi]

Here is the folder structure when using tree ./.git:

./.git/
├── FETCH_HEAD
├── HEAD
├── config
├── index
├── lfs
│   └── tmp
├── logs
│   ├── HEAD
│   └── refs
│       └── remotes
│           └── origin
│               └── compile-commands
├── objects
│   ├── info
│   └── pack
│       ├── pack-6b1b58b3838cd23613fb207e28a627bcf3d8321e.idx
│       └── pack-6b1b58b3838cd23613fb207e28a627bcf3d8321e.pack
├── refs
│   ├── heads
│   ├── pipelines
│   │   └── 834356483
│   ├── remotes
│   │   └── origin
│   │       └── compile-commands
│   └── tags
│       ├── v1.0.0
│       ├── v1.0.1
│       ├── v1.0.2
│       └── v1.0.3
└── shallow
15 directories, 15 files

I also checked that git rev-parse --git-common-dir correctly returns .git

[Chris Sauer]

Super helpful. Thanks, @aminya. Exactly what I was hoping for. Gives confidence in making the right fix.

I'm still not sure why .git is missing the info subdir on gitlab (as with you, I'm seeing it in new and cloned repos) but I can see evidence online that some (old) git versions don't create it. Strange that gitlab would be using those old versions, but hey, it's worth supporting.

I'll write a commit that fixes in the next few minutes. (A backlink should auto-appear.) Could I ask you to test, closing this down if it works for you? (And if it doesn't, comment back here and/or toss up a PR.)

[Chris Sauer]

@aminya?

[Amin Yahyaabadi]

I tested the new version. It works as expected. Thank you!

[Chris Sauer]

yay! Thanks for testing. Delighted to hear it.

Thanks for writing in and working with me to fix it--so we can resolve it for you and for all the others that might come along in the future.

If, on future things, you're ever pretty confident around a fix and want to just toss up a PR you know resolves it to save some round trips, that's awesome. Would love to get you on the contributor board. And if you're not sure, well, always happy to help scope like this.

-CS

[Amin Yahyaabadi]

Thanks very much!