Moving from subversion to git with gitolite

The following notes provide details on the conversion of several repositories from subversion to git making use of two tools to help with the process:

  1. gitolite which provides a framework to help managing a central git repository. From https://github.com/sitaramc/gitolite
  2. svn2git to convert existing repositories from subversion to git. From: https://github.com/nirvdrum/svn2git

This is the second time I performed such conversions and it was a lot easier the second time around due to greater familiarity with git and the tools being used to perform the conversion. These notes should make it even easier the next time I need to do this.

Setting up gitolite

Gitolite is a management framework for git. This is installed onto the server and will be used to manage ACLs, available repositories, configuration and hooks etc.

1. Create the git group and user account

There is a single account that will be used for all access to the git repositories and this needs to be created on the server.

~#> adduser --system --shell /bin/bash --group --disabled-password --home /srv/git git
 Adding system user `git' (UID 125) ...
 Adding new group `git' (GID 125) ...
 Adding new user `git' (UID 125) with group `git' ...
 Creating home directory `/srv/git' ...
 ~#>

2. Change to the git user

We want to install gitolite into the new git account, so we need to su into it.

~#> su - git
 $

3. Download the gitolite code

In this case I have downloaded the code into the src directory in the gitolite account.

$ pwd
 /srv/git
 $ mkdir src
 $ cd src
 $ git clone git://github.com/sitaramc/gitolite
 Initialized empty Git repository in /srv/git/src/gitolite/.git/
 remote: Counting objects: 7695, done.
 remote: Compressing objects: 100% (2528/2528), done.
 remote: Total 7695 (delta 5273), reused 7410 (delta 5025)
 Receiving objects: 100% (7695/7695), 1.79 MiB | 426 KiB/s, done.
 Resolving deltas: 100% (5273/5273), done.
 $

4. Install into ~/bin:

And now install gitolite. There are a few options provided in the gitolite documentation, in this case it is using the recommended method:

$ mkdir ~/bin
 $ ./gitolite/install -ln
 $ ls ~/bin
 gitolite
 $

5. Acquire the initial ssh administrator key

The ssh public key of the initial administrator user is now needed. I copied this from my main workstation:

$ cd ~
$ scp johnsmith@nynaeve:.ssh/id_rsa.pub johnsmith.pub
johnsmith@nynaeve's password: 
id_rsa.pub                                                     100%  397     0.4KB/s   00:00    
$

6. Setup the initial administration repository

Now we are ready to create the gitolite-admin and test repositories.

$ export PATH=$PATH:/srv/git/bin
$  gitolite setup -pk johnsmith.pub
Initialized empty Git repository in /srv/git/repositories/gitolite-admin.git/
Initialized empty Git repository in /srv/git/repositories/testing.git/
WARNING: /srv/git/.ssh/authorized_keys missing; creating a new one
$

7. Edit .gitolite.rc

I made two changes here so that I could manage the repository configuration and hooks from the gitolite admin repository.

First to allow manage of configuration keys I modified GIT_CONFIG_KEYS. Note that this allows any config keys which may not be desirable in all situations. Read the gitolite documentation for more information

GIT_CONFIG_KEYS             =>  '.*',

The next change was to add LOCAL_CODE to set the local that gitolite would look for additional code, such as hooks. This was set to:
# Local code

LOCAL_CODE                  =>  "$ENV{HOME}/.gitolite/local",

This allows hooks to be committed to local/hooks/common directory in the gitolite-admin repository and have them actually take effect.

Managing gitolite

All management of gitolite is now performed via the gitolite-admin repository. This can be checked out using the account associated with the administrator ssh public key used during setup.

~/work%> git clone git@servername:gitolite-aCloning into gitolite-admin...
remote: Counting objects: 54, done.
remote: Compressing objects: 100% (40/40), done.
remote: Total 54 (delta 10), reused 0 (delta 0)
Receiving objects: 100% (54/54), 10.16 KiB, done.
Resolving deltas: 100% (10/10), done.
~/work%>

To allow multiple public keys per user there are a few methods. I prefer the one file with different names per key.

~/work%> cd gitolite-admin/keydir
~/work/gitolite-admin/keydir%> git mv johnsmith.pub johnsmith@hosta.pub
~/work/gitolite-admin/keydir%> git add johnsmith@hostb.pub
~/work/gitolite-admin/keydir%> git commit -a

For the emailer I created local/hooks/common/post-receive and local/sender.cfg in the gitolite-admin repository and for configuration I placed the config keys in conf/gitolite.conf:

~/work/gitolite-admin%> ls
conf/  keydir/  local/
~/work/gitolite-admin%> ls local
hooks/  sender.cfg
~/work/gitolite-admin%> ls local/hooks/common
post-receive*
~/work/gitolite-admin%> cat conf/gitolite.conf
# Default hook settings
repo @all
    config hooks.mailinglist = johnsmith@example.org
    config hooks.emailprefix = "[%GL_REPO] "
    config hooks.maxdiffsize = 5000
    config hooks.project = "%GL_REPO"
    config hooks.link = "http://example.org/git/%GL_REPO/commit/?id=%s"
    config hooks.hostname = "example.org"

repo gitolite-admin
    RW+     =  johnsmith

repo testing
    RW+     =  johnsmith

These settings would depend on the emailer being used but give the basic idea on what is required.

Converting from subversion to git

The conversion process is done using svn2git and is reasonable simple. You should look at the svn2git documentation as it is reasonably straight foward and provides all the various options you are likely to need.

1. Install svn2git

sudo apt-get install git-core git-svn ruby rubygems
umask 022
sudo gem install svn2git

2. Create ~/.svn2git/authors

This will map subversion user names to git usernames. Alternatively you can pass the name of the authors files as a command line argument.

It just contains lines of the form:

johnsmith = John Smith <johnsmitch@example.com>

3. Convert your repository

Make sure you have either saved the subversion password, by directly accessing the repository with subversion, or that you use the password command line option.

There are different command line options to use depending on the layout of the subversion repository. Check the documentation for all the example.

To convert from a standard layout:

svn2git http://svn.example.com/path/to/repo

To convert from a repository that has no structure, just all the files in the root of the repository:

svn2git http://svn.example.com/path/to/repo --rootistrunk

To convert from a repository that has a project name at the top level, then the standard subversion structure underneath, but has no branches:

svn2git http://svn.example.com/path/to/repo --trunk projecta/trunk --tags projecta/tags --nobranches

Use the –verbose command line option if you need to see what is happening.

Use the –rebase if you have already converted the repository but new changes have been made and you want to get those additional changes. This is also useful if the subversion server times out and disconnects you during conversion (I had this happen when converting a very large repository hosted on a windows server when I needed to retry 30 to 40 times before managing to get the entire repository converted.)

Pushing a new repository to gitolite

1. Add the repository

In the gitolite-admin admin repository edit conf/gitolite.conf and define the new repository. This can be as simple as adding the following:

repo reponame
    RW+     =  johnsmith

Then commit the change and push this change to the server. This will create a new empty repository on the server.

2. Push up the existing repository

Change into your existing repository (such as the one just converted from subversion), then add your git server as a remote server and push up the new code:

git remote add origin git@server.example.com:reponame
git push origin master
git push --all
git push --tags

The push –all is only needed if additional branches (other than master) are present in the repository. The push –tags is only needed if there are tags in the repository.

3. Verify that it worked

Finally do a new clone of the repository and verify that the contents are correct:

mkdir ~/test
cd ~test
git clone git@server.example.com:reponame