Git in Debian

From Cateee.net

Jump to: navigation, search
This article is outdated! Actually you will find more informations (from experts) from vcs-pkg.org


At the Debconf7 in Edinburgh, madduck and manoj had two talks about VCS for maintainig debian packages. In this article I want to summarize and put in a better (wikified) way the use of git to maintain the debian packages.


Using a SCM/VCS (Source Code Management / Version Control System) can significantly help package maintainer tasks. In the following article I explain one effective method to use git for packing in Debian. git is a distributed VCS, so it has a very good support for branching and merging, and this capability is central to this HOWTO.


Contents

Introduction

The maintainer task that could use git are:

  • import upstream releases or/and developement version
  • import upstream patches
  • managing the debian directory
  • debianize upstream files (i.e. change paths, Makefile)
  • fix bugs, improve package (i.e. add features, add manpages)
  • export (send) fix and improvement patches to upstream
  • backporting security fixes and important update to stable

Outside the scope of this article are the following use of git:

  • debian package as upstream:
    • the upstream already contain the debian dir, debianisation, the fixes and improvements
    • developement is different, so normal use git documentation is better for standard usage
  • very complex packages:
    • i.e. packages that have a lot of developers, some parallel developement branches (experimental, unstable,...)
    • in such case, it is needed more coordination and the distributed part of git, so you need more understanding on git. See git documentation and i.e. XStrikeForce/git-usage

git package and documentation

To use git in debian you should install one package: git-core, which includes normal git commands and an huge collection of manpages and few introductory documents. You can browse the manpages in file:///usr/share/doc/git-doc/index.html, which are in the package git-doc (or in upstream site http://www.kernel.org/pub/software/scm/git/docs/ ).

Note:
It exists an other git package (and it is also older and has the package name git): the GIT GNU Interactive Tools, a file browser/viewer and process viewer/killer. In this documentation git means always the revision control system named git.

The git commands are always written with the prefix "git-" or "git ", and every command has a man page "git-command"(1).

How to set up bash to use autocompletition?

Additionally there are few graphical client for git: git-gui, gitk or qgit.

There exists also a ggit-buildpackage, which is similar to the other -buildpackages: it contain some shortcut to create and use git. Personally I think it is too much like other -buildpackages, i.e. to much oriented as the old VCS, without effectively using branches.

On the index page of git documentation [file:///usr/share/doc/git-doc/index.html local]/kernel.org there is a nice tutorial [file:///usr/share/doc/git-doc/tutorial.html local]/kernel.org and a very complete user guide [file:///usr/share/doc/git-doc/user-manual.html local]/kernel.org.


Repository structure (branches and tags)

git distinguishes branch and tag. In this document we use the standard interpretation:

  • branches are moving targets (they point to the last version of the package, of a fix,...)
  • tags define a specific and fix version (upstream version, debian release).

We split the Debian packages into the following branches:

  • source branches
    1. upstream: The upstream branch (vanilla sources), ev. with upstream patches
    2. debian-dir: debian/ directory [1] (Note: it can be handled also outside base repository)
    3. debianisation: debian specific changes to upstream (i.e. policy conformance as for paths, etc), which are not intended to be backported to upstream
    4. many fix-xxx: fix or improvement of upstream sources, one xxx-branch for every fixes and improvements. [3]
    5. ev. derivation branches (what prefix/suffix?)
  • integration branches
    1. sloppy: test branch
    2. master (integration): final integration package, ready for final tests and upload [2].
    3. ev. master-dist (integration-dist: eventually (i.e. proposed-stable, security, experimental)

Additionally, instead of removing unneeded branches, we could use old-fix-xxx branch for abandoned fixes and for fix merged in upstream,

Notes:
[1] -- debian-dir can be handled externally (i.e. in a common repository for all packages)
[2] -- we and git-buildpackage use master (the name of the default branche) and maduck/manoj uses integration as final branch.
[3] -- alternativelly you can use only one (or few) fix- branch

The debian-dir, debianisation and the fix-branche are always derived from last upstream branch; sloppy and master are derived from debian-dir (ev debianisation)


We define also some tag which target a specific release:

  • upstream/1.0: the upstream version
  • debian/1.0-1: the debian version
madduck use also debian-dir/1.0-1 tag. Why?


The workflow

At the beginning you should populate the branches.


The workflow for new debian release

  • Normal work is done on sloppy branch and (preliminary) tested
  • we import the changes in the source branches
  • then there is two possibilities to update master (and sloppy):
    • delete old master, and recreate it from a source branch and merge all other relevant branches
    • use current master and merge the changed source branches
  • test, fix and release

Because master contain only merges, we don't lose information (but conflict fixes)

The worklow for new upstream

  • new upstream is imported in upstream branch (check for new and removed files)
  • the other source brancehs are rebased to new upstream
  • master and sloppy are recreated
  • we continue as for new debian releases

Graphical workflow 

            .   Base                    .   New deb rel (2 create)    .   New deb rel (2 merge)    .   New upstream (rebase)
            .                           .                             .                            . 
upstream   ->   u-----U1----------      .   u-----U1----------        .   u-----U1----------       .   u-----U1---U2-------
            .         |\                .         |\                  .         |\                 .              |\   
debian-dir ->         |  a--b--c        .         |  a--b--c--d       .         |  a--b--c---d     .              |  a'--b'--c'
            .          \        \       .          \           \      .          \        \   \    .              \           \ 
fix-branch ->            m---n   |      .            m---n---o  |     .            m---n---|-o |   .                m'----n'   |
            .                  \ |      .                     \ |     .                  \ |  \|   .                        \  |
master     ->                    D1     .                       D2    .                    D1--D2  .                         D1'


Creating the initial git repository

The first time you should create the (local) git repository and split source and patches into the appropriate branches.

Prepare the clean upstream source: 

 tar xzf project-1.0.tar.gz  # extract project
 mv project-1.0 project
 ##  ev. dfsg clean, i.e. removing problematic files and recreating the -dfsg.tar.gz archive
 cd project

Usually creating a new upstrem branche consists in: creating a new repository, commiting all the file in the repository and tagging the version: 

 git init                                           # Creates an empty git repository
 git add .                                          # Add file contents to the changeset to be committed next
 git commit -m 'upstream version 1.0'               # Record changes to the repository
 git tag -a -m 'tagging upstream 1.0' upstream/1.0  # Create a tag object
 git branch -m master upstream                      # Move (rename) branch from default 'master' to 'upstream'
see git-init(1), git-add(1), git-commit(1), git-tag(1), git-branch(1)
Note: madduck signs the tag with GPG (in git tag susbtitute -a with -s

The second step is to create the branches. For the source branches, usually you should create by duplicating the upstream branch. It doesn't cost disk space, and anyway git is very fast, also if you will not use upstream files. 

 git checkout -b debian-dir upstream
 git checkout -b debianisation upstream
see git-checkout(1)

Now we need to populate these (debian-dir, debianisation) and new (fix-XXX) branches, but these are usual task, so they are explained in next section. The rest of this section will give you some suggestions on how to split patches from an existing package.

For every initial branch, you should have a separate patch. Depending on how you worked, you may already have all of the important patches, but if you should complete or create such patches, you have two methods:

  • split the debian .diff file into more file. Usually is a simple copy/paste. It is the only method that permit you to split multiple patches of a single file (i.e. in the main Makefile you can split the debianisation.diff (DESTDIR) and the fix-make-clean.diff).
  • doing diff for every changed file
  • or a mix of the two methods

Handling sources

To modify sources, (i.e. appling patched from the initial import), you should do: 

 git checkout -b debian-dir upstream         # Checkout from upstream branche and create a new branch
 patch -p1 < ../debian-dir.diff              # Apply the patch of last debian package
 git add .                                   # Add file contents to the changeset to be committed next
 git commit -m 'import debian dir, as 1.0-1' # Record changes to the repository
See: git-checkout(1), git-add(1), git-commit(1)

and repeat this for every patch/branch.

Note:

  • If you have apply a patch to an existing branches, simply modify the git checkout with: 
   git checkout debian-dir
  • If you only modify existing files, you can skip git add command and specifiing the modified files into the git commit command: 
   git commit -m 'import debian dir, as 1.0-1' debian/control debian/rules debian/copyright
  • Instead of patch you can modify files with you own editor.

As you can immagine, instead of patch you can modify files with you prefered editor, without changing the git commands.

Examples on common tasks

To add a fix branch (single file): 

git checkout -b fix-manpage upstream   # Checkout and create a new branch
# Fix the manpage, i.e. 'project.1'
git commit -m 'fix manpage' project.1  # Record changes to the repository

To add a mew file to an existing branch === 

git checkout debian-dir              # Checkout and switch to a branch
echo 5 > debian/compat               # Create the new file
git add debian/compat                # Add file contents to the changeset to be committed next 
git commit -m 'created debian/compat with value 5' debian/compat  # Record changes to the repository

Changing a branch: 

git checkout debian-dir                                 # Checkout and switch to a branch
echo 6 > debian/compat
git commit -m 'update debian/compat to 6' debian/compat  # Record changes to the repository

Using sloppy branch

If you need some tests, usually it is better to use a sloppy branch, try to pack, correct, test, correct, and only at the end to divide the changes into the source brances. In this manner you don't forget to remove unneded workaround, disgnostic codes, etc.

To create and destroy sloppy branch (note you can use also more testing branches in parallel). The sloppy branch should be based on master (integration) branch, so that you can test the complete package. In the next section we explain ho to create the master branch. 

 git checkout -b sloppy master    # Create a new branch
 ...
 git branch git branch -d sloppy   # Delete a branch if is is already merged
 # or
 git branch git branch -D sloppy   # Delete a branch anyway.
See: git-checkout(1), git-branch(1)

Check the commits SHA1 code in sloppy branch 

 git log --pretty=oneline sloppy
Note: you can use one of the following format oneline, short, medium, full, fuller, email, raw and format:<string>

Go to the right branch (eventually create it) and reapply the changes from a sloppy commit

alternatelly, you can take the relevant part and only later commit the changes: 

 git checkout debian-dir
 ...
 git-cherry-pick -n SHA1-commit-id
 git-cherry-pick -n SHA1-commit-id3
 ...
 git commit -m 'Correct (to the minimun) the package dependencies' debian/control
See git-cherry-pickick(1)

Integration

Until now you worked with separate branches for every fix. Before to pack, you should integrate all fix and debianisation patches. It is very simple: 

 git checkout master
 git merge debian-dir
 git merge debianisation
 git merge fix-xxx
 ...
See: git-checkout(1), git-merge(1)

The first time, you should create the master branch with: 

 git checkout -b master upstream

but subsequent merge you should still work with existing master, so not to lose the already done manual merges.

Resolving a merge

Sometimes the automatic merge will fail, so you should manually merge files: 

 git status   # it will show the files and the part of file that failed the automatic merge 
 ... # edit such file (you will see both part of failed merge
 git commit Makefile  # note that default failed merge commit message should be good enough

Check the git user manual to have more information on the topic, but usually git is smart enough that nearly all of your merge will be done automatically, and the eventually few manual merge are normally very trivial tasks.

Building package

After you have integrated the master-branch, simply: 

 git-buildpackage

If the package is good and:

  • it pass usual tests (i.e. lintian, linda, piuparts, debuild-pbuilder)
  • the .diff.gz and the .deb files don't include unwanted stuff
  • it is ok

than you should tag the source: 

 git status   # verify that all is already commited
 # ev.  git commit -m Debian package 1.0-2' debian/changelog
 git tag -a -m 'Debian package 1.0-2' debian/1.0-2

upload the package amd eventually update the public git repository.

New upstream

When you have a new upstream version, you should update the upstream branches (and tag the new sources), and rebase the other source branches: 

 git checkout upstream
 ... # Check the new code, apply patch or untar new version
 git add .
 git commit -m 'new upstream 1.1'
 git tag -a -m 'Upstream 1.1' upstream/1.1
 git rebase upstream debianisation
 git rebase upstream fix-..
 # ...
See: git-checkout(1), git-add(1), git-commit(1), git-tag(1), git-rebase(1)

The integration branches are rebased by reintegrating the source branches.

Checking, reporting and getting patches

Show the branches tree and the current working branche 

git branches   # show the branches tree and the current working branche

Show the status of current tree: 

git status

Details of current tree changes (diff) 

git diff   # show the diff between current tree and last committed tree

It will check the changes between the current working tree and the last commit (on current branche):

The git diff command will generate for you a unified patch, ready to be sent to upstream. 

git diff upstream..fix-manpage  # show the diff of a branches (ready to be sent to upstream)

Other usefull command

Moving and removing files:

  • git add file-name add a specific file (see git-add(1))
  • git mv file-name move a specific file without loosing history and merges (see git-mv(1))
  • git rm file-name add a specific file (see git-rm(1))

Managing commit errors:

  • git revert reverts a specific commit (see git-revert(1))
  • git reset resets last commits, removing from history (see git-reset(1))

Public git repository

You can publish your repository on a public git server, so that all user can see and help the package, and it will help in case of NMU, MIA, ...

As prerequisite, you need an alioth account.

Log into your account with ssh and: 

 mkdir public_git       # create the git.debian.org/~your-name directory
 chmod a+xr public_git  # give access to git daemon

On your working machine, prepare the public repository (i.e. only the .git directory), and copy to the remote site. 

 git clone --bare ~/proj proj.git    # create the exportable directory
 touch proj.git/git-daemon-export-ok # tell git daemon that it is a public repo
 scp -r proj.git/ alioth.debian.org:public_git/

Now you should have a working public repository. You can test with: 

 cd /tmp
 git clone git://git.debian.org/~your-name/proj.git proj
 cd proj
 git branch -a
 git checkout -b master origin/master
 git branch -a
 git pull git://git.debian.org/~your-name/proj.git debian-dir

To push branches into the public git server, do the folowing command for every branch: 

 git push ssh://alioth.debian.org/var/lib/gforge/chroot/home/users/your-name/public_git/proj.git debian-dir
See git-push(1)

References

Other usefull command

External Links

Retrieved from "http://www.cateee.net/wikipage/Git_in_Debian"
Personal tools