re-base patches to another base branch

[Terin Stock]

I previously created a base branch from a Git tag, on which to base my patches. The software has made a new major release, and I've created a new base branch. Is there a correct way to, uh, "re-base" my patches to this new base branch?

[mackyle]

Recommendation

Before making any changes, create a TopGit tag using the tg tag
command. Something like tg tag <new-topgit-tag-name> --all.
That will allow you to more easily recover from any accidents
and make any needed "maintenance" updates to the old patches,
if necessary, much easier.

As you may have noticed, there is no tg rebase. Although git rebase
does have --keep-empty --preserve-merges --merge options, reading
the description of --preserve-merges will probably convice you
that it could be problematic. Combine that with the fact that
different versions of Git have had various corner-case problems
with rebase and that TopGit merge histories can be somewhat complicated
and it's unlikely that there will ever be a tg rebase command
that uses git rebase.

The tg create --base and tg update --base pair of commands can
be helpful if you have created a "base release" TopGit branch to
base all your patches off of. But I gather from your question above
you do not have such a branch.

Two questions are relevant here:

1. Is the "major new release" a fast-forward of the previous Git tag
   you based your patches on?

2. Are your patches strictly linear? In other words each patch
   depends on exactly one previous patch and it's only the very
   first patch that depends on the original Git tag.

If the answer to (1) is "yes," then you may safely manually edit
each .topdeps file that mentions the original branch you based
your patches off of and change it to mention the new release (but
it has to be a branch name, it cannot be a tag name). When you
next run tg update all the changes from the new release will then
be merged into your patches.

You could also do tg create --base on the old Git tag and update
the .topdeps file(s) (there will only be one that needs to be
updated if your answer to (2) is "yes") to point to that and then
use tg update --base to update your new "base release" TopGit
branch to the new tag. That will ultimately save you some time when
you next move to a new release, but it will accomplish the same
thing in that the next time you run tg update all the changes from
the new release will be merged into your patches.

This is not a rebase, it attempts to merge the changes from the old
release to the new release into your patches. Rebase, merge, rebase
with the -m option -- which one produces the best results via the
fewest merge conflicts really depends on the sources in question.

If your answer to (1) is "no" doing this (basically fast-forwarding
your base and merging in the updates) will probably not work for
you or at least will be somewhat more painful. (tg update --base
does have a --force option to allow such a thing, but some of the
resulting conflict resolution might get rather complicated involving
files that have nothing to do with your patches and you would need
to carefully examine the final results to make sure that each of
your resulting patches really does what it's supposed to do.)

If you don't mind discarding the change history of each of your
TopGit patches (i.e. tg log would only show one commit on each
of your TopGit branches after this), you can use the tg export
command to get a set of patches and then tg import to apply them
to the "major new release". This will work best if your answer to
(2) is "yes".

If the answer to both (1) and (2) is "no" you can use the
tg export --collapse mode and then use tg import -s <name> to
import one patch at a time onto the "major new release" adjusting
.topdeps files as you go to recreate your non-linear TopGit patches.

It's conceivable that TopGit might grow a tg cleanrebase command
at some point that's a combination of the internal tg export --collapse
logic and git rebase --preserve-merges command. The small number
of commits/merges produced by the "collapse" logic ought to be
workable with a git "rebase" operation. There are complications
(TopGit patch sets need not have exactly one base upon which they
build but Git's "rebase" command only takes a single --onto
argument) so don't hold your breath waiting for such a command. ;)

For me personally, the things I keep patch sets for always have a
"yes" answer to (1). (Even if that's only because I've been importing
exploded release tarballs into Git and making it that way in some
cases.) For some of the patch sets the answer to (2) is most
definitely "no". But all of the patch sets are now based off of a
TopGit branch created using tg create --base and I simply update
them (after first creating a tg tag so I can go back) using
tg update --base to move them to the next release.

[Terin Stock]

For the record, the answers here were:

1. No
2. Yes

Unfortunately, I created the topics off of master first, before realizing what I had done. So the "major new release" is a simplification: in reality it's going backwards.

I resolved this with a variation of the suggested tg export to export all of the topics, delete the topic and staging branches, ensuring I was based off of the correct base branch, and created the patch series one at a time, the recreated the staging branch.

From here on out, the base branches should be fast-forwards only, so it will hopefully be less of a problem. I should be able to manually update .topdep to reflect the new base. (It would be nice if tg depend understood how to do this, but I can live without it.)

As a happy StGit user, the differences here are initially confusing, but I think I'm starting to understand it.

[mackyle]

As far as I know, StGit is still around.

I think I see a feature request lurking in there somewhere... :)

To confirm, you're looking for something like this:

tg depend replace foo bar

Where a dependency in the .topdeps file named "foo" gets replaced
by one named "bar"? The command could check and complain if it's
not a fast-forward and perhaps handle more than one TopGit branch
at a time. But is that basically the feature request?

To verify your original description, you started with something
like this:

git clone .../project && cd project
tg create t/branch1 master
tg create t/branch2 t/branch1

Giving you an output from tg summary --rdeps --heads like so:

t/branch2
  t/branch1
    master

Then, having determined you didn't want to be based off of master
after all did something similar to this I suspect:

git branch patch-basis v1.0
tg create t/branch1 patch-basis
tg create t/branch2 t/branch1

Thus giving you this:

t/branch2
  t/branch1
    patch-basis

Alternatively you could do this:

tg create --base t/patch-basis v1.0
tg create t/branch1 t/patch-basis
tg create t/branch2 t/branch1

resulting in this:

t/branch2
  t/branch1
    t/patch-basis

To update the patch basis for the first two you do something like
this:

git checkout patch-basis
git merge --ff-only v2.0 # --ff-only isn't strictly necessary
tg update t/branch2

To update the patch basis for the last one you do this:

tg update --base t/patch-basis v2.0
tg update t/branch2

When you generate patches via tg export you still only get two
patches for the last one because the t/patch-basis TopGit branch
doesn't have any changes of its own.

The difference between having a non-TopGit branch as the basis
(either master or patch-basis above) versus having a "[BASE]"
TopGit branch as the basis (t/patch-basis) only comes into play
if you are collaborating on the patches with someone else and pushing
your TopGit branches somewhere.

While it is true that tg push (and git push if properly configured)
will push the non-TopGit basis branches (unless you add the
--tgish-only option), when a collaborator does the initial clone
and then uses tg remote --populate to set up their local branches,
the non-TopGit branches will be left entirely alone. They will
have to be set up by hand. Any future fetches that grab updates
to the basis branch will also need to have those manually incorporated.

On the other hand if you use a TopGit "[BASE]" branch, it will
automatically be set up by tg remote --populate and at tg update
time updates to it that were "fetched" will be automatically
incorporated.

That's the primary difference. If you're not planning to push your
TopGit branches somewhere for multiple collaborators to work on it
won't make any difference.

You can always change your mind later and do something like this
to switch to a TopGit "[BASE]" branch:

tg create --base t/patch-basis v1.0
# manually edit each .topdeps file that mentions
# "patch-basis" and replace it with "t/patch-basis"
# Or use the vaporware command :)
tg depend replace patch-basis t/patch-basis

[Terin Stock]

As far as I know, StGit is still around.

In this case, temporarily maintaining patches on third-party software while we work to upstream, while having a staging branch that brings all the patches together. TopGit felt more like the right tool for the job.

To confirm, you're looking for something like this:

tg depend replace foo bar

Where a dependency in the .topdeps file named "foo" gets replaced
by one named "bar"?

This is correct.

The other behavior described seemed to be mostly related to my incorrect usage of TopGit the first time.

I can create a new topic [pun intended], if you think this is off-topic here, but I'm curious what you recommend for peer reviewing of changes to the patches.

[Terin Stock]

Closing this as I no longer am maintaining a fork of an upstream, so no longer utilizing TopGit as described here.