17 Feb 2009 My git-svn workflow
A while back Mark expressed interest to me in using git and git-svn for version control on his own machine, against a remote Subversion repository. However, when I followed up with him recently, he admitted that in the small amount of time he’d spent looking at it, he hadn’t really got his head around how it was all going to hang together.
I can relate. I found Git to have a steep learning curve. It took me a while – and some assistance (thanks Tom) – to figure out the magical incantations to accomplish what I wanted. But now that I know them, I’ve found Git very useful for local version control.
I was going to walk Mark through my rough git-svn workflow, figuring it’d either get him started or scare him off for good 😉 Then I decided I might as well share it with the world.
I’ve covered my motivations for using git for local version control in a previous post, so I won’t repeat them here. Nor am I going to provide a tutorial on git and git-svn – there’s plenty of those out there already. Instead, I’m going to run through an example taken from my day-to-day work to try and show how I use git and git-svn in real-life.
The Basics
- The first step is something I normally only do very occasionally: create a local Git clone of my remote Subversion repository. So say that I want to create a git clone of the ‘remote_maintenance’ project on the Shine Subversion repository:
git svn clone svn+ssh://subversion.shinetech.com/home/svn/remote_maintenance/trunk remote_maintenance
This checks out the contents of the ‘trunk’ branch into a local directory called ‘remote_maintenance’. This local checkout is known as a ‘working tree’.
The most significant thing I can say about ‘git svn clone’ is that it will take a while if your SVN repository has a big history, as git will create an entire replica of this history. This might sound like a drag but can be very useful later if you’re working offline.
- Having cloned the repository, I merrily jump into the code in the working tree and start modifying, adding, deleting and moving files. Note that if I want to move a file and have git track a move, I use ‘git mv’. For example, to rename the ‘README’ file to ‘README.txt’, I’d do:
bent:remote_maintenance bent$ git mv README README.txt
I could just move it using ‘mv’, but git wouldn’t be able to track the change.
- When I’m curious to know what my changes to the working tree have been since my last commit, I use ‘git status’. For example, having renamed a file and added a new file, I’d get the following output:
If I want to look at my changes in more detail, I run:
bent:remote_maintenance bent$ git status
# On branch master
# Changes to be committed:
# (use "git reset HEAD ..." to unstage)
#
# renamed: README -> README.txt
#
# Untracked files:
# (use "git add ..." to include in what will be committed)
#
# LICENCE.txt
git diff
Which gives me a line-by-line diff.
- When I’m ready to commit my changes to git, I first run:
git add .
This will add any new files to the local index. I’ll cover what this means in more detail in the next step. After I’ve done run this command, running ‘git status’ again will yield the following:
bent:remote_maintenance bent$ git status
# On branch master
# Changes to be committed:
# (use "git reset HEAD ..." to unstage)
#
# new file: LICENCE.txt
# renamed: README -> README.txt
#
- Next, I commit all my changes by running:
git commit -a -m 'Miscellaneous changes'
The ‘-a’ means that git will automatically add any changed files to the local index before doing the commit. For git newbies, git differs from Subversion in that shepherding changes into a git repository is a two-stage process – first you add a changed file to a local index, then you commit the changes to the repository itself. ‘git commit -a’ is a convenient way of combining these two steps into one.Unfortunately, if I have added brand new files to my working tree, ‘git commit -a’ won’t pick them up. Thus the need to explicitly add them to the repository using ‘git add’ in the previous step. - Having done my commit, I can return to step 2 and repeat as often as I want: make changes to my working tree, add them to the local index, and commit them.
Branching
- So what if I want to branch? Well firstly, let’s examine the history of my commits by using ‘git log’:
Miscellaneous changes
bent:remote_maintenance bent$ git log
commit 76b1ebf06586843e990a29423df39f2def2492b5
Author: Ben Teese
Date: Tue Jan 6 12:04:49 2009 +1100commit 21873ad305a06b97504c0c2270549a9a20238596
Author: Ben Teese
Date: Wed Jan 7 16:09:34 2009 +1100Made it that ground server binds TCP server to provided IP address only, not all network interface
commit 69b5b7421bc989538689711198510702f129d8f6
Author: Ben Teese
Date: Wed Jan 7 14:06:05 2009 +1100Made it that transfer service TCP server only binds to IP address provided. Added logging.
… - To see what branches we’ve got at the moment, we use ‘git branch’:
bent:remote_maintenance bent$ git branch
* master
We see that we currently only have one branch called ‘master’ – the default one that you get when you create a new repository. - Say that I want to try an alternate approach to the change I just committed. To do that, I’d create a create a local branch from the previous commit and check it out. I can do this in one step using:
git checkout -b new_branch 21873ad305a06b97504c0c2270549a9a20238596
The long string is a SHA that represents the commit we want to branch from. There are other, shorter, ways to refer to commits, but I find it just as easy to copy the SHA from the output of ‘git log’ directly into the ‘git checkout’ command. - Now let’s use ‘git branch’ to see what branches we’ve now got:
bent:remote_maintenance bent$ git branch
master
* new_branch
We see that out new branch appears and that it has an asterisk next to it. This means it is the currently checked-out branch. - Now, if I do a ‘git log’:
commit 21873ad305a06b97504c0c2270549a9a20238596
Author: Ben Teese
Date: Wed Jan 7 16:09:34 2009 +1100Made it that ground server binds TCP server to provided IP address only, not all network interface
commit 69b5b7421bc989538689711198510702f129d8f6
Author: Ben Teese
Date: Wed Jan 7 14:06:05 2009 +1100Made it that transfer service TCP server only binds to IP address provided. Added logging.
…
I see that my new branch only goes as far as the previous commit. You could now commit changes to this branch, and they wouldn’t appear in the ‘master’ branch. - We’ll switch back to the ‘master’ branch to continue with this demo:
bent:remote_maintenance bent$ git checkout master
Switched to branch "master"
Synchronizing with Subversion
- When I want to synchronize my local git branch with the remote Subversion repository, I run:
git svn rebase
It’ll spit out something like this:
A rebase temporarily winds back the commits that you’ve made on the branch since the last time you rebased, applies the commits from Subversion to the branch, then reapplies your commits to the branch. The great thing about this is that git-svn keeps track of the last rebase you did, so you never have to tell it that you only want to rebase from a particular point. This avoids some of the problems that Subversion has when you repeatedly merge from one branch to another.
M test/unit/csdb_inventory_content_test.rb
M test/unit/resource_entity_test.rb
M test/unit/ground_server_test.rb
r3743 = 64471395cf084217f7eab91e07abb03297492c83 (git-svn)
M test/unit/csdb_inventory_content_test.rb
M test/unit/resource_entity_test.rb
M test/unit/ground_server_test.rb
r3744 = 1828baa8e7b91cd8861dfaaecc26eb66b0f9264e (git-svn)
First, rewinding head to replay your work on top of it...
Applying: Made it that transfer service TCP server only binds to IP address provided. Added logging.
Applying: Miscellaneous changes
/Users/bent/NetBeansProjects/remote_maintenance/.git/rebase-apply/patch:31: trailing whitespace.
- But what if somebody has changed a file that I’ve changed and we have a merge conflict? Git will report the problem as it is trying to reapply the local commits:
Auto-merged test/unit/command/reader_test.rb
CONFLICT (content): Merge conflict in test/unit/command/reader_test.rb
Failed to merge in the changes.
Patch failed at 0002.When you have resolved this problem run “git rebase –continue”.
If you would prefer to skip this patch, instead run “git rebase –skip”.
To restore the original branch and stop rebasing run “git rebase –abort”. - To resolve the merge conflict, I open up test/unit/command/reader_test.rb, resolve the merge conflicts, then run:
git add test/unit/command/reader_test.rb
- I repeat step the previous step for any other file that reported a conflict during the re-application of that particular commit. When I’m done, I run:
git rebase --continue
- Additional merge conflicts may occur when git reapplies later commits, in which case I repeat steps 3 and 4.
- Now I have a look at my log on the current branch:
Made it that ground server binds TCP server to provided IP address only, not all network interface
bent:remote_maintenance bent$ git log
commit 21873ad305a06b97504c0c2270549a9a20238596
Author: Ben Teese
Date: Wed Jan 7 16:09:34 2009 +1100commit 69b5b7421bc989538689711198510702f129d8f6
Author: Ben Teese
Date: Wed Jan 7 14:06:05 2009 +1100Made it that transfer service TCP server only binds to IP address provided. Added logging.
commit 1828baa8e7b91cd8861dfaaecc26eb66b0f9264e
Author: danielw
Date: Tue Jan 6 22:13:34 2009 +0000converted the first letter of the test names to be lower case
git-svn-id: svn+ssh://svn.shinetech.com/home/svn/remote_maintenance/trunk@3744 25b05753-7f2e-0410-
We can see that the changes in my local git repository have been applied on top of the latest change from Subversion.
Squashing Changes Together
- Now I want to squash together the first 2 of the changes I’ve made to my local git respository into a single commit. To do this, I run:
bent:remote_maintenance bent$ git rebase -i git-svn
My local terminal editor will appear with the following:
# Rebase 1828baa..76b1ebf onto 1828baa
pick 69b5b74 Made it that transfer service TCP server only binds to IP address provided. Added logging.
pick 21873ad Made it that ground server binds TCP server to provided IP address only, not all network interfaces.
pick 76b1ebf Added dvd stub that responds correctly to ProtocolRequest from iPhone.
#
# Commands:
# p, pick = use commit
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
# - To squash the second commit into the first, I edit the text as follows:
# Rebase 1828baa..76b1ebf onto 1828baa
pick 69b5b74 Made it that transfer service TCP server only binds to IP address provided. Added logging.
squash 21873ad Made it that ground server binds TCP server to provided IP address only, not all network interfaces.
pick 76b1ebf Miscellaneous changes
#
# Commands:
# p, pick = use commit
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#
and save and exit the editor. - git immediately pops up another editor that allows me to merge the comments for these two commits that I have said I want to squash together:
Made it that transfer service TCP server only binds to IP address provided. Added logging.
# This is a combination of two commits.
# The first commit's message is:# This is the 2nd commit message:
Made it that ground server binds TCP server to provided IP address only, not all network interfaces.
# Please enter the commit message for your changes. Lines starting
# with ‘#’ will be ignored, and an empty message aborts the commit.
#
# Committer: Ben Teese
#
# Not currently on any branch.
# Changes to be committed:
# (use “git reset HEAD …” to unstage)
#
# new file: lib/dvd_stub.rb
# modified: lib/ground_server.rb
# modified: lib/rmu_stub.rb
# - Which I do accordingly:
Made it that TCP servers only uses IP address provided. This ensures it only binds to the network inte
rface of that IP address, not all network interfaces. Added logging.
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# Committer: Ben Teese
#
# Not currently on any branch.
# Changes to be committed:
# (use "git reset HEAD ..." to unstage)
#
# new file: lib/dvd_stub.rb
# modified: lib/ground_server.rb
# modified: lib/rmu_stub.rb
#
and save and exit. Git will then squash the two commits together. - If we do a ‘git log’, we see that git has merged the two commits into one, complete with a new message:
Miscellaneous changes
bent:remote_maintenance bent$ git log
commit 6a29acd801f280360699ac5278fcb63bb9c8744a
Author: Ben Teese
Date: Tue Jan 6 12:04:49 2009 +1100commit 1fb107af4c765d2220d6674d61edca3fb13f5dc2
Author: Ben Teese
Date: Wed Jan 7 14:06:05 2009 +1100Made it that TCP servers only uses IP address provided. This ensures it only binds to the network
commit 1828baa8e7b91cd8861dfaaecc26eb66b0f9264e
Author: danielw
Date: Tue Jan 6 22:13:34 2009 +0000converted the first letter of the test names to be lower case
git-svn-id: svn+ssh://svn.shinetech.com/home/svn/remote_maintenance/trunk@3744 25b05753-7f2e-0410-
…
Commiting to Subversion
Having squashed some of our commits together, commiting to Subversion is relatively straightforward:
bent:remote_maintenance bent$ git svn dcommit
Committing to svn+ssh://svn.shinetech.com/home/svn/remote_maintenance/trunk ...
A lib/dvd_stub.rb
M lib/ground_server.rb
M lib/rmu_stub.rb
Committed r3745
M lib/ground_server.rb
M lib/rmu_stub.rb
A lib/dvd_stub.rb
r3745 = fd5fbaf8ad9a71ad997f61e39fc2f3446c47b848 (git-svn)
...
This will create one commit in Subversion for each git commit.
Keeping Branches Synchronized
So what if you’ve made changes in one branch and you want to get them into another branch? The key piece of advice I can give is to use the Subversion repository as the transport mechanism. Once you’ve finished making changes in one branch, check them into the Subversion repository. Then switch to the other branch and rebase from the Subversion repository.
But what if you want to move code from one branch to another, without commiting to Subversion? Well, I have never done a direct git merge from one branch to another, and the ‘Caveats’ section of the git-svn man page recommends against it if you’re using git-svn. To be perfectly honest, it’s never been an issue for me. I tend to work on distinct features in separate branches so there is little code overlap. If there is some code overlap, I just have to fix the merge conflicts when I rebase from Subversion. However, given that I made both sets of changes, this usually isn’t too difficult.
Conclusion
From this demo you’ll see that my day-to-day usage of git and git-svn essentially boils down to 10 commands. I’ve listed them below, along with a quick summary of what they do:
- git status :See an overview of the files I’ve changed
- git diff: See a diff of the changes I’ve made
- git add: Add new or merged files to the index.
- git commit -a: Add changed files to the index and commit
- git log: See the history of commits I’ve made.
- git branch: See what branch is currently checked out.
- git checkout -b: Checkout a new branch from a particular point.
- git rebase -i: Squash together a bunch of git commits.
- git svn rebase: Get the changes in the Subversion repository.
- git svn dcommit: Send git commits to the Subversion repository.
Once I figured these out, I never looked back. I hope that you find them as useful as I did when using git for local version control with a remote Subversion repository.
rip747
Posted at 12:15h, 02 Aprildo you know of anyway to apply a git format-patch to an svn repo?
Joe Atzberger
Posted at 15:43h, 08 AugustGenerate the patch with –no-prefix and apply as normal:
git format-patch –no-prefix master # or whatever target
….
patch -p0 <0001-whatever.patch
Of course that's not the same thing as getting the patch via the repo w/ a git-svn ID, but since you are using format-patch, you obviously intend to use the patch file(s) output.
Ben Teese
Posted at 12:46h, 02 AprilI’m not an expert on git-format-patch, but essentially it sounds like it produces diffs, albeit in something resembling the UNIX mailbox format. Diffs can be applied to anything, so you could just apply them to a checkout of your SVN repo, then check the changes into the repo.
Wayne
Posted at 09:01h, 19 NovemberSteep learning curve, but very useful indeed. I found myself using it once when I was about to try something extremely fiddly and experimental in my own pet project. It feels secure and logical to have local stable snapshots of the project, which you can fall back upon during development. I also found that it saved me alot of time when exploring multiple different approaches in solving a problem. If you found a bug which you suspect is caused by a recent change, you would also be able to revert back to a previous build with minimal hassle.
Tough but really worth learning. Thanks Ben 🙂
joez
Posted at 17:55h, 14 Januaryexcellent, see also http://trac.parrot.org/parrot/wiki/git-svn-tutorial
Ben
Posted at 19:35h, 07 JuneFYI, git CAN recognise a rename/move, even if you just do it. You change a class name, change the name of the file, and move it somewhere completely different, and it will ‘just know’.
You may not notice this at first because it only does the magic when you have added the ‘delete’ and ‘untracked’ files to the index.
Try it though, it’s pretty amazing, and one thing I really hate about other SCMs (they all want to be in charge of my file structure!)
Ben Teese
Posted at 19:31h, 10 June@Ben Yeah in recent months I’ve been noticing git automatically pick up some amazing stuff when it comes to moving and copying files. For example, the other day I opened a Word document that happened to be in a git repository, made some modifications to it and saved it under a new name in the same directory. When I did a ‘git status’, git flagged that the new file was a copied and modified version of the old file – which was basically true. I’ve never seen an SCM pick that sort of thing up before. I’m not sure if this is something that has gotten better with recent releases or was always there, but it sure makes life easier to have that sort of automatic tracking in place.
Pingback:Using Git for Local Version Control | Shine Technologies Blog
Posted at 10:10h, 16 May[…] See My git-svn workflow for further […]
Pingback:Using Git for Local Version Control | The Shine Technologies Blog
Posted at 06:10h, 24 August[…] As suggested by one of the commenters, a better way to do this is to use git-svn rebase -i See My git-svn workflow for further […]
Pingback:Git-svn workflows « I Geek Code
Posted at 05:19h, 27 July[…] My git-svn workflow via ShineTech.com has a good overview of the entire workflow. […]