samrocketman / gitlab-mirrors

Using gitlab-mirrors 0.5.1 with GitLab CE 7.14.1 on Ubuntu 14.04.01 installed GitLab from the repo.

I have gitlab-mirrors setting to not force updates, but still it happens:

#Enable force fetching and pushing.  Will overwrite references if upstream
#forced pushed.  Applies to git projects only.
force_update=false

Looking at the cron.log (modified to hide actual names):

To git@my-gitlab.example.com:my-scope/my-project.git
 + 11d30b8...da80e38 my_branch -> my_branch (forced update)

In my setup I have a project at Bitbucket and I mirror that repository to my local GitLab instance using deployment key. Then I multiple people work against the GitLab instance that is sync'd to Bitbucket manually every now and then. This is not optimal situation, but we have to work this way for reasons beyond our control.

Now the problematic thing is that when ever I push changes to my branch at GitLab, mirrorer will do a force push on next scheduled mirror and resets the branch HEAD at GitLab to what ever is the HEAD at Bitbucket. As a side effect this will turn new commits into dangling commits. Additionally with GitLab (might be git feature) these commits becomes inaccessible using git fetch or pull (though you can find them with GitLab web interface if you know the commit IDs).

Update the refspec and remove the + from it. That means editing .git/config to update the refspec of your cloned repository. I'll have to update the default behavior of adding mirrors when force is disabled.

To learn more see git help push and view the docs about refspec.

I did some additional tests and my modified my gitlab-mirrors clone under repositories so that I removed the + from config (/home/gitmirror/repositories/blaa/myproject/config), but it does not fix the situation.

When testing with a clone from on my laptop everything works as expected. When having + in .git/config push refspec and doing push, I can push even when my branch is behind its remote counterpart. When removing the +, push fails since the tip of my branch is behind its remote counterpart. This works as expected based on git documentation.

When using gitlab-mirrors this does not seem to work as expected. When I remove the +, it does the push even when the remote's tip of the branch is behind the gitlab's tip of the same branch.

Git version at the server is 1.9.1 and on my laptop it is 2.1.4.

Just tested by upgrading git to 2.5.0 at the server, but it had no effect.

Found now the reason and way to fix, but now sure if this is "the right way" to fix it.

Basically if config states mirror = true, then refspecs are ignored. Got hint of this by manually trying to pass refspecs to git push and I got error stating refspecs not allowed with --mirror.

Once I removed mirror = true from gitlab origin, then push started to fail as expected when removing the + from push config refspec. In other words, removing the + is not enough, also mirror = true needs to be removed and then push will fail when origin is behind gitlab.

I could write a patch doing the needed adjustments in add_mirror.sh and send a pull request if you agree this is right thing to do, though I'm not sure if there are any problematic side-effects from not mirroring. You can find me with nick "mikoni" from freenode and I'm hanging at #gitlab if you wish to have a chat.

Reading closer git push man page, --mirror should respect refspec from the config, but for some reason having mirror = true in the config changes this.

Git push man page description seems to be contradicting a bit with rest of the documentation and --mirror or "mirror = true" in config implies the + in refspec and cannot be overridden. So actually those push refspec lines in add_mirror.sh are unnecessary and dropping the + to prevent force push has no effect. Mirroring does always force push.

Thanks for your research. Feel free to open a pull request and we can comment in it on how best to support the force option.

After thinking a lot about it. We can resolve this by pushing by refspec instead of depending on the git config.

e.g.

if ${force_enabled}; then
  forcespec='+'
else
  forcespec=''
fi
git push gitlab "${forcespec}refs/heads/*:refs/heads/*" "${forcespec}refs/tags/*:refs/tags/*"

We would have to remove the mirror option from existing gitlab git remotes for mirrors. e.g.

git config --unset-all remote.gitlab.mirror

Hybrid option might be best

A hybrid option between an added setting and adhering to remote settings might be best.

Right now I'm thinking on whether or not it should be an option for repository creation. Or an option during push. The advantage of the former is per repository force setting. It's possible to have a hybrid and make use of both methods at once.

if ${force_enabled}; then
  forcespec='+'
else
  forcespec=''
fi
git config --get-all remote.gitlab.push | while read refspec; do
  git push gitlab "${forcespec}${refspec#+}"
done

The hybrid option could check for git config --get gitlabmirrors.localonlyrefspec option and determine whether or not it uses the dynamic forcespec or just makes use of normal git push gitlab.

That would look something like this...

if ${force_enabled}; then
  forcespec='+'
else
  forcespec=''
fi
if [ "$(git config --get gitlabmirrors.localonlyrefspec)" = 'true' ]; then
  git push gitlab
else
  git config --get-all remote.gitlab.push | while read refspec; do
    git push gitlab "${forcespec}${refspec#+}"
  done
fi

I don't quite have decent settings and option terminology fleshed out but I think that would solve the issue. Maybe call the option perrepoforce i.e. per repo force option.

git config gitlabmirrors.perrepoforce
./add_mirror.sh --no-per-repo-force --enable-force

That would disable per repo force pushing and enable force pushing for that repository. That means that repository would always force push regardless of the global config.sh setting. The option could also be called --disable-force. Since force pushing is the default behavior currently I think it should stay the default behavior with the option to disable it.

I think this is now so outdated that this can be closed. Also, I realized after thinking different scenarios that what I was asking is wrong. So this issue is pretty much invalid.

I forgot to return to this issue and close it. Sorry.

force_push setting ignored

Hybrid option might be best