config value 'safe.directory' was not found (Rugged::ConfigError) error after upgrade to 0.11.1

Question

config value 'safe.directory' was not found (Rugged::ConfigError) error after upgrade to 0.11.1

gingerlime opened this issue a year ago · comments

We're seeing a weird error, which seems to happen only on our CI but not on local development:

bundler: failed to load command: pronto (/usr/local/bundle/bin/pronto)
/usr/local/bundle/gems/pronto-0.11.1/lib/pronto/cli.rb:65:in `discover': config value 'safe.directory' was not found (Rugged::ConfigError)
	from /usr/local/bundle/gems/pronto-0.11.1/lib/pronto/cli.rb:65:in `run'
	from /usr/local/bundle/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
	from /usr/local/bundle/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
	from /usr/local/bundle/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
	from /usr/local/bundle/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
	from /usr/local/bundle/gems/pronto-0.11.1/bin/pronto:6:in `<top (required)>'
	from /usr/local/bundle/bin/pronto:23:in `load'
	from /usr/local/bundle/bin/pronto:23:in `<top (required)>'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/cli/exec.rb:58:in `load'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/cli/exec.rb:58:in `kernel_load'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/cli/exec.rb:23:in `run'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/cli.rb:491:in `exec'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/cli.rb:34:in `dispatch'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/cli.rb:28:in `start'
	from /usr/local/bundle/gems/bundler-2.4.2/exe/bundle:45:in `block in <top (required)>'
	from /usr/local/bundle/gems/bundler-2.4.2/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	from /usr/local/bundle/gems/bundler-2.4.2/exe/bundle:33:in `<top (required)>'
	from /app/bin/bundle:113:in `load'
	from /app/bin/bundle:113:in `<main>'

Our CI (Semaphore) seems to use some kind of a shallow clone of our github repository, but we fetch all the code as far as I can tell. Not sure what else might be the difference. File permissions seem fine. It's consistently failing on Semaphore but works on our local dev environment somehow.

Our semaphore yml looks like this (redacted with the key commands)

            # See https://docs.semaphoreci.com/ci-cd-environment/toolbox-reference/#shallow-clone
            - checkout
            - git fetch -q --unshallow
            - git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'
            - git fetch -q origin master
            - ...
            - docker-compose exec specs bundle exec pronto run

It works 100% consistently with pronto 0.11.0

Any suggestions? :)

Ashish Kulkarni · Answer 1 · Sun Jan 08 2023 14:40:37 GMT+0800 (China Standard Time)

In 0.11.0, the version of rugged used was more constrained. Maybe there's an incompatibility with the later versions with your CI -- can you try pinning it via gem 'rugged', '< 1.1.0' and see if it helps?

gingerlime · Answer 2 · Tue Jan 10 2023 11:11:24 GMT+0800 (China Standard Time)

Thank you @ashkulz. Yes, if we pin rugged down it works again. Is this the best solution however?

Ashish Kulkarni · Answer 3 · Tue Jan 10 2023 13:32:06 GMT+0800 (China Standard Time)

@gingerlime the error is occurring at this line. Can you make a reproducible test case scenario or at least narrow which version of rugged is causing the issue?

gingerlime · Answer 4 · Tue Jan 10 2023 14:00:47 GMT+0800 (China Standard Time)

@ashkulz yeah, I saw the line where it fails, but I'm not sure how to reproduce it. It seems something internal to git and how rugged is accessing git data. I'm entirely unfamiliar with this unfortunately. As I mentioned it only happens on Semaphore. I can ask the Semaphore support team however. Maybe they have an idea.

Ashish Kulkarni · Answer 5 · Tue Jan 10 2023 14:23:23 GMT+0800 (China Standard Time)

@gingerlime doing a quick google search tells that the permissions on the git directory in SemaphoreCI are incorrect -- e.g. see this discussion. I think you should pin the rugged version till they fix it, I'm not really sure I can do anything here 🤷‍♂️

gingerlime · Answer 6 · Tue Jan 10 2023 14:30:49 GMT+0800 (China Standard Time)

Thanks @ashkulz yeah, I found this discussion as well, but I don't see any permission issues on Semaphore. I tried to debug it and it looked ok to me. I contacted their support, so let's see what they say. But I can imagine them saying "it's Rugged. The new version broke things", so it's something in-between the two :) I understand also pronto is in-between here, but I hope we can find a solution together. I'd be happy to test whatever I can and also hope that Semaphore is able to help. I appreciate you taking the time to look into it.

Ashish Kulkarni · Answer 7 · Tue Jan 10 2023 14:37:38 GMT+0800 (China Standard Time)

The best way is to pin rugged (during CI?) till their support responds with a solution. I've not got reports from anyone else, so I think it is something specific to their setup 🤷‍♂️

gingerlime · Answer 8 · Tue Jan 10 2023 14:47:17 GMT+0800 (China Standard Time)

It might be specific to Semaphore, so that's definitely good to know. Let's see what they say and take it from there :) thanks again for your patience and assistance.