Subversion: The Definitive Guide

All version control systems have to solve the same fundamental problem: how will the system allow users to share information, but prevent them from accidentally stepping on each other's feet? It's all too easy for users to accidentally overwrite each other's changes in the repository.

Consider this scenario: suppose we have two co-workers, Harry and Sally. They each decide to edit the same repository file at the same time. If Harry saves his changes to the repository first, then it's possible that (a few moments later) Sally could accidentally overwrite them with her own new version of the file. While Harry's version of the file won't be lost forever (because the system remembers every change), any changes Harry made won't be present in Sally's newer version of the file, because she never saw Harry's changes to begin with. Harry's work is still effectively lost—or at least missing from the latest version of the file—and probably by accident. This is definitely a situation we want to avoid!

Figure 2.2. The Problem to Avoid

Many version control systems use a lock-modify-unlock model to address this problem, which is a very simple solution. In such a system, the repository allows only one person to change a file at a time. First Harry must "lock" the file before he can begin making changes to it. Locking a file is a lot like borrowing a book from the library; if Harry has locked a file, then Sally cannot make any changes to it. If she tries to lock the file, the repository will deny the request. All she can do is read the file, and wait for Harry to finish his changes and release his lock. After Harry unlocks the file, his turn is over, and now Sally can take her turn by locking and editing.

Figure 2.3. The Lock-Modify-Unlock Solution

The problem with the lock-modify-unlock model is that it's a bit restrictive, and often becomes a roadblock for users:

Subversion, CVS, and other version control systems use a copy-modify-merge model as an alternative to locking. In this model, each user's client reads the repository and creates a personal working copy of the file or project. Users then work in parallel, modifying their private copies. Finally, the private copies are merged together into a new, final version. The version control system often assists with the merging, but ultimately a human being is responsible for making it happen correctly.

Here's an example. Say that Harry and Sally each create working copies of the same project, copied from the repository. They work concurrently, and make changes to the same file "A" within their copies. Sally saves her changes to the repository first. When Harry attempts to save his changes later, the repository informs him that his file A is out-of-date. In other words, that file A in the repository has somehow changed since he last copied it. So Harry asks his client to merge any new changes from the repository into his working copy of file A. Chances are that Sally's changes don't overlap with his own; so once he has both sets of changes integrated, he saves his working copy back to the repository.

Figure 2.4. The Copy-Modify-Merge Solution

Figure 2.5. …Copy-Modify-Merge Continued

But what if Sally's changes do overlap with Harry's changes? What then? This situation is called a conflict, and it's usually not much of a problem. When Harry asks his client to merge the latest repository changes into his working copy, his copy of file A is somehow flagged as being in a state of conflict: he'll be able to see both sets of conflicting changes, and manually choose between them. Note that software can't automatically resolve conflicts; only humans are capable of understanding and making the necessary intelligent choices. Once Harry has manually resolved the overlapping changes (perhaps by discussing the conflict with Sally!), he can safely save the merged file back to the repository.

The copy-modify-merge model may sound a bit chaotic, but in practice, it runs extremely smoothly. Users can work in parallel, never waiting for one another. When they work on the same files, it turns out that most of their concurrent changes don't overlap at all; conflicts are infrequent. And the amount of time it takes to resolve conflicts is far less than the time lost by a locking system.

In the end, it all comes down to one critical factor: user communication. When users communicate poorly, both syntactic and semantic conflicts increase. No system can force users to communicate perfectly, and no system can detect semantic conflicts. So there's no point in being lulled into a false promise that a locking system will somehow prevent conflicts; in practice, locking seems to inhibit productivity more than anything else.

You've already read about working copies; now we'll demonstrate how the Subversion client creates and uses them.

A Subversion working copy is an ordinary directory tree on your local system, containing a collection of files. You can edit these files however you wish, and if they're source code files, you can compile your program from them in the usual way. Your working copy is your own private work area: Subversion will never incorporate other people's changes, nor make your own changes available to others, until you explicitly tell it to do so.

After you've made some changes to the files in your working copy and verified that they work properly, Subversion provides you with commands to "publish" your changes to the other people working with you on your project (by writing to the repository). If other people publish their own changes, Subversion provides you with commands to merge those changes into your working directory (by reading from the repository).

A working copy also contains some extra files, created and maintained by Subversion, to help it carry out these commands. In particular, each directory in your working copy contains a subdirectory named .svn, also known as the working copy administrative directory. The files in each administrative directory help Subversion recognize which files contain unpublished changes, and which files are out-of-date with respect to others' work.

A typical Subversion repository often holds the files (or source code) for several projects; usually, each project is a subdirectory in the repository's filesystem tree. In this arrangement, a user's working copy will usually correspond to a particular subtree of the repository.

For example, suppose you have a repository that contains two software projects.

Figure 2.6. The Repository's Filesystem

In other words, the repository's root directory has two subdirectories: paint and calc.

To get a working copy, you must check out some subtree of the repository. (The term "check out" may sound like it has something to do with locking or reserving resources, but it doesn't; it simply creates a private copy of the project for you.) For example, If you check out /calc, you will get a working copy like this:

$ svn checkout http://svn.example.com/repos/calc
A  calc
A  calc/Makefile
A  calc/integer.c
A  calc/button.c

$ ls -a calc
Makefile  integer.c  button.c  .svn/

The list of letter A's indicates that Subversion is adding a number of items to your working copy. You now have a personal copy of the repository's /calc directory, with one additional entry—.svn—which holds the extra information needed by Subversion, as mentioned earlier.

$ svn checkout file:///path/to/repos
…
$ svn checkout file://localhost/path/to/repos
…

$ svn checkout file:///X:/path/to/repos
…
$ svn checkout file:///X|/path/to/repos
…

Suppose you make changes to button.c. Since the .svn directory remembers the file's modification date and original contents, Subversion can tell that you've changed the file. However, Subversion does not make your changes public until you explicitly tell it to. The act of publishing your changes is more commonly known as committing (or checking in) changes to the repository.

To publish your changes to others, you can use Subversion's commit command:

$ svn commit button.c
Sending button.c
Transmitting file data..
Committed revision 57.

Now your changes to button.c have been committed to the repository; if another user checks out a working copy of /calc, they will see your changes in the latest version of the file.

Suppose you have a collaborator, Sally, who checked out a working copy of /calc at the same time you did. When you commit your change to button.c, Sally's working copy is left unchanged; Subversion only modifies working copies at the user's request.

To bring her project up to date, Sally can ask Subversion to update her working copy, by using the Subversion update command. This will incorporate your changes into her working copy, as well as any others that have been committed since she checked it out.

$ pwd
/home/sally/calc

$ ls -a 
.svn/ Makefile integer.c button.c

$ svn update
U button.c

The output from the svn update command indicates that Subversion updated the contents of button.c. Note that Sally didn't need to specify which files to update; Subversion uses the information in the .svn directory, and further information in the repository, to decide which files need to be brought up to date.

A svn commit operation can publish changes to any number of files and directories as a single atomic transaction. In your working copy, you can change files' contents, create, delete, rename and copy files and directories, and then commit the complete set of changes as a unit.

In the repository, each commit is treated as an atomic transaction: either all the commit's changes take place, or none of them take place. Subversion tries to retain this atomicity in the face of program crashes, system crashes, network problems, and other users' actions.

Each time the repository accepts a commit, this creates a new state of the filesystem tree, called a revision. Each revision is assigned a unique natural number, one greater than the number of the previous revision. The initial revision of a freshly created repository is numbered zero, and consists of nothing but an empty root directory.

A nice way to visualize the repository is as a series of trees. Imagine an array of revision numbers, starting at 0, stretching from left to right. Each revision number has a filesystem tree hanging below it, and each tree is a “snapshot” of the way the repository looked after each commit.

Figure 2.7. The Repository

It's important to note that working copies do not always correspond to any single revision in the repository; they may contain files from several different revisions. For example, suppose you check out a working copy from a repository whose most recent revision is 4:

calc/Makefile:4
     integer.c:4
     button.c:4

At the moment, this working directory corresponds exactly to revision 4 in the repository. However, suppose you make a change to button.c, and commit that change. Assuming no other commits have taken place, your commit will create revision 5 of the repository, and your working copy will now look like this:

calc/Makefile:4
     integer.c:4
     button.c:5

Suppose that, at this point, Sally commits a change to integer.c, creating revision 6. If you use svn update to bring your working copy up to date, then it will look like this:

calc/Makefile:6
     integer.c:6
     button.c:6

Sally's changes to integer.c will appear in your working copy, and your change will still be present in button.c. In this example, the text of Makefile is identical in revisions 4, 5, and 6, but Subversion will mark your working copy of Makefile with revision 6 to indicate that it is still current. So, after you do a clean update at the top of your working copy, it will generally correspond to exactly one revision in the repository.

For each file in a working directory, Subversion records two essential pieces of information in the .svn/ administrative area:

Given this information, by talking to the repository, Subversion can tell which of the following four states a working file is in:

This may sound like a lot to keep track of, but the svn status command will show you the state of any item in your working copy. For more information on that command, see Section , “svn status”.

As a general principle, Subversion tries to be as flexible as possible. One special kind of flexibility is the ability to have a working copy containing mixed revision numbers.

At first, it may not be entirely clear why this sort of flexibility is considered a feature, and not a liability. After completing a commit to the repository, the freshly committed files and directories are at more recent working revision than the rest of the working copy. It looks like a bit of a mess. As demonstrated earlier, the working copy can always be brought to a single working revision by running svn update. Why would someone deliberately want a mixture of working revisions?

Assuming your project is sufficiently complex, you'll discover that it's sometimes nice to forcibly “backdate” portions of your working copy to an earlier revision; you'll learn how to do that in Chapter 3. Perhaps you'd like to test an earlier version of a sub-module, contained in a subdirectory, or perhaps you'd like to examine a number of previous versions of a file in the context of the latest tree.

However you make use of mixed-revisions in your working copy, there are limitations to this flexibility.

First, you cannot commit the deletion of a file or directory which isn't fully up-to-date. If a newer version of the item exists in the repository, your attempt to delete will be rejected, to prevent you from accidentally destroying changes you've not yet seen.

Second, you cannot commit a metadata change to a directory unless it's fully up-to-date. You'll learn about attaching “properties” to items in Chapter 6. A directory's working revision defines a specific set of entries and properties, and thus committing a property change to an out-of-date directory may destroy properties you've not yet seen.

When you create a new Subversion repository, it begins its life at revision zero and each successive commit increases the revision number by one. After your commit completes, the Subversion client informs you of the new revision number:

$ svn commit --message "Corrected number of cheese slices."
Sending        sandwich.txt
Transmitting file data .
Committed revision 3.
      

If at any point in the future you want to refer to that revision (We'll see how and why we might want to do that later in this chapter), you can refer to it as “3”.

The Subversion client understands a number of revision keywords. These keywords can be used instead of integer arguments to the --revision switch, and are resolved into specific revision numbers by Subversion:

Here are some examples of revision keywords in action (don't worry if the commands don't make sense yet; we'll be explaining these commands as we go through the chapter):

$ svn diff --revision PREV:COMMITTED foo.c
# shows the last change committed to foo.c

$ svn log --revision HEAD
# shows log message for the latest repository commit

$ svn diff --revision HEAD
# compares your working file (with local mods) to the latest version
# in the repository.

$ svn diff --revision BASE:HEAD foo.c
# compares your “pristine” foo.c (no local mods) with the 
# latest version in the repository

$ svn log --revision BASE:HEAD
# shows all commit logs since you last updated

$ svn update --revision PREV foo.c
# rewinds the last change on foo.c.
# (foo.c's working revision is decreased.)
      

These keywords allow you to perform many common (and helpful) operations without having to look up specific revision numbers or remember the exact revision of your working copy.

Anywhere that you specify a revision number or revision keyword, you can also specify a date by specifying the date inside curly braces “{}”. You can even access a range of changes in the repository using both dates and revisions together!

Subversion accepts an incredible number of date formats —just remember to use quotes around any date that contains spaces. Here are just a few of the formats that Subversion accepts:

$ svn checkout --revision {2002-02-17}
$ svn checkout --revision {2/17/02}
$ svn checkout --revision {"17 Feb"}
$ svn checkout --revision {"17 Feb 2002"}
$ svn checkout --revision {"17 Feb 2002 15:30"}
$ svn checkout --revision {"17 Feb 2002 15:30:12 GMT"}
$ svn checkout --revision {"10 days ago"} 
$ svn checkout --revision {"last week"} 
$ svn checkout --revision {"yesterday"} 
…
      

When you specify a date as a revision, Subversion finds the most recent revision of the repository as of that date:

$ svn log --revision {11/28/2002}
------------------------------------------------------------------------
rev 12:  ira | 2002-11-27 12:31:51 -0600 (Wed, 27 Nov 2002) | 6 lines
…
      

You can also use a range of dates. Subversion will find all revisions between both dates, inclusive:

$ svn log --revision {2002-11-20}:{2002-11-29}
…
      

As we pointed out, you can also mix dates and revisions:

$svn log -r {11/20/02}:4040
      

When working on a project with a team, you'll want to update your working copy: that is, receive any changes from other developers on the project. You'll use svn update to bring your working copy into sync with the latest revision in the repository.

$ svn update
U  ./foo.c
U  ./bar.c
Updated to revision 2.
      

In this case, someone else checked in modifications to both foo.c and bar.c since the last time you updated, and Subversion has updated your working copy to include those changes.

Let's examine the output of svn update a bit more. When the server sends changes to your working copy, a letter code is displayed next to each item to let you know what actions Subversion performed to bring your working copy up-to-date:

Now you can get to work and make changes in your working copy. It's usually most convenient to decide on a particular change (or set of changes) to make, such as writing a new feature, fixing a bug, etc. The Subversion commands that you will use here are svn add, svn delete, svn copy, and svn move. However, if you are merely editing a file (or files) that is already in Subversion, you may not need to use any of these commands until you commit. Changes you can make to your working copy:

To make file changes, use your text editor, word processor, graphics program, or whatever tool you would normally use. Subversion handles binary files just as easily as it handles text files—and just as efficiently too.

Here is an overview of the four Subversion subcommands that you'll use most often to make tree changes (we'll cover svn import and svn mkdir later).

Once you've finished making changes, you need to commit them to the repository, but before you do so, it's usually a good idea to take a look at exactly what you've changed. By examining your changes before you commit, you cannot only make a more accurate log message, but you may discover that you've inadvertently changed a file, and this gives you a chance to revert those changes before committing. Additionally, this is a good opportunity to review and scrutinize changes before publishing them. You can see exactly what changes you've made by using svn status, svn diff, and svn revert. You will usually use the first two commands to find out what files have changed in your working copy, and then perhaps the third to revert some (or all) of those changes.

Subversion has been optimized to help you with this task, and is able to do many things without communicating with the repository. In particular, your working copy contains a secret cached “pristine” copy of each version controlled file within the .svn area. Because of this, Subversion can quickly show you how your working files have changed, or even allow you to undo your changes without contacting the repository.

$ svn status
  L    ./abc.c               # svn has a lock in its .svn directory for abc.c
M      ./bar.c               # the content in bar.c has local modifications
 M     ./baz.c               # baz.c has property but no content modifications
?      ./foo.o               # svn doesn't manage foo.o
!      ./some_dir            # svn manages this, but it's either missing or incomplete
~      ./qux                 # versioned as dir, but is file, or vice versa
A  +   ./moved_dir           # added with history of where it came from
M  +   ./moved_dir/README    # added with history and has local modifications
D      ./stuff/fish.c        # this file is scheduled for deletion
A      ./stuff/loot/bloo.h   # this file is scheduled for addition
C      ./stuff/loot/lump.c   # this file has conflicts from an update
    S  ./stuff/squawk        # this file or dir has been switched to a branch
        

$ svn status stuff/fish.c
D      stuff/fish.c
        

$ svn status --verbose
M               44        23    sally     ./README
                44        30    sally     ./INSTALL
M               44        20    harry     ./bar.c
                44        18    ira       ./stuff
                44        35    harry     ./stuff/trout.c
D               44        19    ira       ./stuff/fish.c
                44        21    sally     ./stuff/things
A                0         ?     ?        ./stuff/things/bloo.h
                44        36    harry     ./stuff/things/gloo.c
        

$ svn status --show-updates --verbose
M      *        44        23    sally     ./README
M               44        20    harry     ./bar.c
       *        44        35    harry     ./stuff/trout.c
D               44        19    ira       ./stuff/fish.c
A                0         ?     ?        ./stuff/things/bloo.h
        

$ svn diff
Index: ./bar.c
===================================================================
--- ./bar.c
+++ ./bar.c	Mon Jul 15 17:58:18 2002
@@ -1,7 +1,12 @@
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include <stdio.h>

int main(void) {
-  printf("Sixty-four slices of American Cheese...\n");
+  printf("Sixty-five slices of American Cheese...\n");
return 0;
}

Index: ./README
===================================================================
--- ./README
+++ ./README	Mon Jul 15 17:58:18 2002
@@ -193,3 +193,4 @@ 
+Note to self:  pick up laundry.

Index: ./stuff/fish.c
===================================================================
--- ./stuff/fish.c
+++ ./stuff/fish.c  Mon Jul 15 17:58:18 2002
-Welcome to the file known as 'fish'.
-Information on fish will be here soon.

Index: ./stuff/things/bloo.h
===================================================================
--- ./stuff/things/bloo.h
+++ ./stuff/things/bloo.h  Mon Jul 15 17:58:18 2002
+Here is a new file to describe
+things about bloo.
        

$ svn diff > patchfile
        

$ svn revert README
Reverted ./README
        

$ svn status foo
?      foo

$ svn add foo
A         foo

$ svn revert foo
Reverted foo

$ svn status foo
?      foo
        

$ svn status README 
       README

$ svn delete README 
D         README

$ svn revert README
Reverted README

$ svn status README
       README
        

We've already seen how svn status -u can predict conflicts. Suppose you run svn update and some interesting things occur:

$ svn update
U  ./INSTALL
G  ./README
C  ./bar.c
      

The U and G codes are no cause for concern; those files cleanly absorbed changes from the repository. The files marked with U contained no local changes but were Updated with changes from the repository. The G stands for merGed, which means that the file had local changes to begin with, but the changes coming from the repository didn't overlap in any way.

But the C stands for conflict. This means that the changes from the server overlapped with your own, and now you have to manually choose between them.

Whenever a conflict occurs, your Subversion client does the three things:

For example, Sally makes changes to the file sandwich.txt in the repository. Harry has just changed the file in his working copy and checked it in. Sally updates her working copy before checking in and she gets a conflict:

$ svn update
C  sandwich.txt
Updated to revision 2.
$ ls -1
sandwich.txt                       
sandwich.txt.r2        
sandwich.txt.r1
sandwich.txt.mine
$
      

At this point, Subversion will not allow you to commit the file sandwich.txt until the three temporary files are removed.

$ svn commit --message "Add a few more things"
subversion/libsvn_client/commit.c:654: (apr_err=155015, src_err=0)
svn: A conflict in the working copy obstructs the current operation
svn: Commit failed (details follow):
subversion/libsvn_client/commit_util.c:180: (apr_err=155015, src_err=0)
svn: Aborting commit: '/home/sally/svn-work/sandwich.txt' remains in conflict.
      

If you get a conflict, you need to do one of three things:

Once you've resolved the conflict, you need to let Subversion know by running svn resolved. This removes the three temporary files and Subversion no longer considers the file to be in a state of conflict.^[4]

$ svn resolved sandwich.txt
Resolved conflicted state of sandwich.txt
      

$ cat sandwich.txt
Top piece of bread
Mayonnaise
Lettuce
Tomato
Provolone
<<<<<<< .mine
Salami
Mortadella
Prosciutto
=======
Sauerkraut
Grilled Chicken
>>>>>>> .r2
Creole Mustard
Bottom piece of bread
        

<<<<<<< .mine
Salami
Mortadella
Prosciutto
=======
        

=======
Sauerkraut
Grilled Chicken
>>>>>>> .r2
        

Top piece of bread
Mayonnaise
Lettuce
Tomato
Provolone
Salami
Mortadella
Prosciutto
Creole Mustard
Bottom piece of bread
        

$ svn resolved sandwich.txt
$ svn commit -m "Go ahead and use my sandwich, discarding Sally's edits."
        

$ svn update
C  sandwich.txt
Updated to revision 2.
$ ls sandwich.*
sandwich.txt  sandwich.txt.mine  sandwich.txt.r2  sandwich.txt.r1
$ cp sandwich.txt.r2 sandwich.txt
$ svn resolved
$ svn commit -m "Go ahead and use Sally's sandwich, discarding my edits."
        

$ svn revert sandwich.txt
Reverted sandwich.txt
$ ls sandwich.*
sandwich.txt
        

Now you're ready to check in your changes. Note that svn resolved, unlike most of the other commands we've dealt with in this chapter, requires an argument. In any case, you want to be careful and only run svn resolved when you're certain that you've fixed the conflict in your file—once the temporary files are removed, Subversion will let you commit the file even if it still contains conflict markers.

Finally! Your edits are finished, you've merged all changes from the server, and you're ready to commit your changes to the repository.

The svn commit command sends all of your changes to the repository. When you commit a change, you need to supply a log message, describing your change. Your log message will be attached to the new revision you create. If your log message is brief, you may wish to supply it on the command line using the --message (or -m) option:

$ svn commit --message "Corrected number of cheese slices."
Sending        sandwich.txt
Transmitting file data .
Committed revision 3.
      

However, if you've been composing your log message as you work, you may want to tell Subversion to get the message from a file by passing the filename with the --file switch:

svn commit --file logmsg 
Sending        sandwich
Transmitting file data .
Committed revision 4.
      

If you fail to specify either the --message or --file switch, then Subversion will automatically launch your favorite editor (as defined in the environment variable $EDITOR) for composing a log message.

$ svn commit
Waiting for Emacs...Done

Log message unchanged or not specified
a)bort, c)ontinue, e)dit
a
$
        

The repository doesn't know or care if your changes make any sense as a whole; it only checks to make sure that nobody else has changed any of the same files that you did when you weren't looking. If somebody has done that, the entire commit will fail with a message informing you that one or more of your files is out-of-date:

$ svn commit --message "Add another rule"
Sending        rules.txt
subversion/libsvn_client/commit.c:655: (apr_err=160028, src_err=0)
svn: Transaction is out of date
svn: Commit failed (details follow):
subversion/libsvn_repos/commit.c:112: (apr_err=160028, src_err=0)
svn: out of date: `rules.txt' in txn `g'
$

At this point, you need to run svn update, deal with any merges or conflicts that result, and attempt your commit again.

That covers the basic work cycle for using Subversion. There are many other features in Subversion that you can use to manage your repository and working copy, but you can get by quite easily using only the commands that we've discussed so far in this chapter.

To find out information about the history of a file or directory, use the svn log command. svn log will provide you with a record of who made changes to a file or directory, at what revision it changed, the time and date of that revision, and, if it was provided, the log message that accompanied the commit.

$ svn log
------------------------------------------------------------------------
rev 3:  sally | Mon, 15 Jul 2002 18:03:46 -0500 | 1 line

Added include lines and corrected # of cheese slices.
------------------------------------------------------------------------
rev 2:  harry | Mon, 15 Jul 2002 17:47:57 -0500 | 1 line

Added main() methods.
------------------------------------------------------------------------
rev 1:  sally | Mon, 15 Jul 2002 17:40:08 -0500 | 2 lines

Initial import
------------------------------------------------------------------------
      

Note that the log messages are printed in reverse chronological order by default. If you wish to see a different range of revisions in a particular order, or just a single revision, pass the --revision (-r) switch:

$ svn log --revision 5:19    # shows logs 5 through 19 in chronological order

$ svn log -r 19:5            # shows logs 5 through 19 in reverse order

$ svn log -r 8               # shows log for revision 8
      

You can also examine the log history of a single file or directory. For example:

$ svn log foo.c
…
$ svn log http://foo.com/svn/trunk/code/foo.c
…
      

These will display log messages only for those revisions in which the working file (or URL) changed.

If you want even more information about a file or directory, svn log also takes a --verbose (-v) switch. Because Subversion allows you to move and copy files and directories, it is important to be able to track path changes in the filesystem, so in verbose mode, svn log will include a list of changed-paths in a revision in its output:

$ svn log -r 8 -v
------------------------------------------------------------------------
rev 8:  sally | 2002-07-14 08:15:29 -0500 | 1 line
Changed paths:
U /trunk/code/foo.c
U /trunk/code/bar.h
A /trunk/code/doc/README

Frozzled the sub-space winch.

------------------------------------------------------------------------
      

$ svn log -r 2
------------------------------------------------------------------------
$
        

We've already seen svn diff before—it displays file differences in unified diff format; it was used to show the local modifications made to our working copy before committing to the repository.

In fact, it turns out that there are three distinct uses of svn diff:

$ svn diff
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth open
+Chew with your mouth closed
+Listen when others are speaking
$
        

$ svn diff --revision 3 rules.txt 
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth open
+Chew with your mouth closed
+Listen when others are speaking
$
        

$ svn diff --revision 2:3 rules.txt 
Index: rules.txt
===================================================================
--- rules.txt	(revision 2)
+++ rules.txt	(revision 3)
@@ -1,4 +1,4 @@
 Be kind to others
-Freedom = Chocolate Ice Cream
+Freedom = Responsibility
 Everything in moderation
 Chew with your mouth closed 
$
        

$ svn diff --revision 4:5 http://svn.red-bean.com/repos/example/trunk/text/rules.txt
…
$
        

If you want to examine an earlier version of a file and not necessarily the differences between two files, you can use svn cat:

$ svn cat --revision 2 rules.txt 
Be kind to others
Freedom = Chocolate Ice Cream
Everything in moderation
Chew with your mouth closed
$
      

You can also redirect the output directly into a file:

$ svn cat --revision 2 rules.txt > rules.txt.v2
$
      

You're probably wondering why we don't just use svn update --revision to update the file to the older revision. There are a few reasons why we might prefer to use svn cat.

First, you may want to see the differences between two revisions of a file using an external diff program (perhaps a graphical one, or perhaps your file is in such a format that the output of unified diff is nonsensical). In this case, you'll need to grab a copy of the old revision, redirect it to a file, and pass both that and the file in your working copy to your external diff program.

Sometimes it's easier to look at an older version of a file in its entirety as opposed to just the differences between it and another revision.

The svn list command shows you what files are in a repository directory without actually downloading the files to your local machine:

$ svn list http://svn.collab.net/repos/svn
README
branches/
clients/
tags/
trunk/
      

If you want a more detailed listing, pass the --verbose (-v) flag to get output like this.

$ svn list --verbose http://svn.collab.net/repos/svn
_    2755    harry     1331 Jul 28 02:07 README
_    2773    sally        0 Jul 29 15:07 branches/
_    2769    sally        0 Jul 29 12:07 clients/
_    2698    harry        0 Jul 24 18:07 tags/
_    2785    sally        0 Jul 29 19:07 trunk/
      

The columns tell you if a file has any properties (“P” if it does, “_” if it doesn't), the revision it was last updated at, the user who last updated it, the size if it is a file, the date it was last updated, and the item's name.

In addition to all of the above commands, you can use svn update and svn checkout with the --revision switch to take an entire working copy “back in time” ^[6]:

$ svn checkout --revision 1729 # Checks out a new working copy at r1729
…
$ svn update --revision 1729 # Updates an existing working copy to r1729
…
      

When Subversion modifies your working copy (or any information within .svn), it tries to do so as safely as possible. Before changing anything, it writes its intentions to a logfile, executes the commands in the logfile, then removes the logfile (This is similar in design to a journaled filesystem). If a Subversion operation is interrupted (If you hit Control-C, or if the machine crashes, for example), the logfiles remain on disk. By re-executing the logfiles, Subversion can complete the previously started operation, and your working copy can get itself back into a consistent state.

And this is exactly what svn cleanup does: it searches your working copy and runs any leftover logs, removing locks in the process. If Subversion ever tells you that some part of your working copy is “locked”, then this is the command that you should run. Also, svn status will display an L next to locked items:

$ svn status
  L    ./somedir
M      ./somedir/foo.c 

$ svn cleanup
$ svn status
M      ./somedir/foo.c
      

The import command is a quick way to move an unversioned tree of files into a repository.

$ svnadmin create /usr/local/svn/newrepos
$ svn import mytree file:///usr/local/svn/newrepos/fooproject
Adding  mytree/foo.c
Adding  mytree/bar.c
Adding  mytree/subdir
Adding  mytree/subdir/quux.h
Transmitting file data....
Committed revision 1.
      

The above example places the contents of directory mytree under the directory fooproject in the repository:

/fooproject/foo.c
/fooproject/bar.c
/fooproject/subdir
/fooproject/subdir/quux.h
      

Now we've covered most of the Subversion client commands. Notable exceptions are those dealing with branching and merging (see Chapter 4, Branching and Merging) and properties (see Section , “Properties”). However, you may want to take a moment to skim through Chapter 8, Subversion Complete Reference to get an idea of all the many different commands that Subversion has—and how you can use them to make your work easier.

Creating a branch is very simple—you make a copy of the project in the repository using the svn copy command. Subversion is not only able to copy single files, but whole directories as well. In this case, you want to make a copy of the /trunk/calc directory. Where should the new copy live? Wherever you wish—it's a matter of project policy. Let's say that your team has a policy of creating branches in the /branches/calc area of the repository, and you want to name your branch "my-calc-branch". You'll want to create a new directory, /branches/calc/my-calc-branch, which starts as a copy of /trunk/calc.

There are two different ways to make a copy. We'll demonstrate the messy way first, just to make the concept clear. To begin, check out a working copy of the root (/) of the repository:

$ svn checkout http://svn.example.com/repos bigwc
A  bigwc/branches/
A  bigwc/branches/calc
A  bigwc/branches/paint
A  bigwc/trunk/
A  bigwc/trunk/calc
A  bigwc/trunk/calc/Makefile
A  bigwc/trunk/calc/integer.c
A  bigwc/trunk/calc/button.c
A  bigwc/trunk/paint
A  bigwc/trunk/paint/Makefile
A  bigwc/trunk/paint/canvas.c
A  bigwc/trunk/paint/brush.c
Checked out revision 340.

Making a copy is now simply a matter of passing two working-copy paths to the svn copy command:

$ cd bigwc
$ svn copy trunk/calc branches/calc/my-calc-branch
$ svn status
A  +   branches/calc/my-calc-branch

In this case, the svn copy command recursively copies the trunk/calc working directory to a new working directory, branches/calc/my-calc-branch. As you can see from the svn status command, the new directory is now scheduled for addition to the repository. But also notice the + sign next to the letter A. This indicates that the scheduled addition is a copy of something, not something new. When you commit your changes, Subversion will create /branches/calc/my-calc-branch in the repository by copying /trunk/calc, rather than resending all of the working copy data over the network:

$ svn commit -m "Creating a private branch of /trunk/calc."
Adding      branches/calc/my-calc-branch
Committed revision 341.

And now the easier method of creating a branch, which we should have told you about in first place: svn copy is able to operate on two URLs.

$ svn copy http://svn.example.com/repos/trunk/calc \
           http://svn.example.com/repos/branches/calc/my-calc-branch \
      -m "Creating a private branch of /trunk/calc"

Committed revision 341.

There's really no difference between these two methods. Both procedures create a new directory in revision 341, and the new directory is a copy of /trunk/calc. Notice that the second method, however, performs an immediate commit. It's an easier procedure, because it doesn't require you to check out a large mirror of the repository. In fact, this technique doesn't even require you to have a working copy at all.

Figure 4.3. Repository With New Copy

Now that you've created a new branch of the project, you can check out a new working copy to start using it:

$ svn checkout http://svn.example.com/repos/branches/calc/my-calc-branch
A  my-calc-branch/Makefile
A  my-calc-branch/integer.c
A  my-calc-branch/button.c
Checked out revision 341.

There's nothing special about this working copy; it simply mirrors a different location of the repository. When you commit changes, however, Sally won't ever see them when she updates. Her working copy is of /trunk/calc.

Let's pretend that a week goes by, and the following commits happen:

There are now two independent lines of development happening on integer.c:

Figure 4.4. The Branching of One File's History

Things get interesting when you look at the history of changes made to your copy of integer.c:

$ pwd
/home/user/my-calc-branch

$ svn log integer.c
------------------------------------------------------------------------
rev 343:  user | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines

* integer.c:  frozzled the wazjub.

------------------------------------------------------------------------
rev 303:  sally | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines

* integer.c:  changed a docstring.

------------------------------------------------------------------------
rev 98:  sally | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines

* integer.c:  adding this file to the project.

------------------------------------------------------------------------

Notice that Subversion is tracing the history of your integer.c all the way back through time, traversing the point where it was copied. (Remember that your branch was created in revision 341.) Now look what happens when Sally runs the same command on her copy of the file:

$ pwd
/home/sally/calc

$ svn log integer.c
------------------------------------------------------------------------
rev 344:  sally | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines

* integer.c:  fix a bunch of spelling errors.

------------------------------------------------------------------------
rev 303:  sally | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines

* integer.c:  changed a docstring.

------------------------------------------------------------------------
rev 98:  sally | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines

* integer.c:  adding this file to the project.

------------------------------------------------------------------------

Sally sees her own revision 344 change, but not the change you made in revision 343. As far as Subversion is concerned, these two commits affected different files in different repository locations. However, Subversion does show that the two files share a common history. Before the branch-copy was made in revision 341, they used to be the same file. That's why you and Sally both see revisions 303 and 98.

There are two important lessons that you should remember from this section.

In the previous section, we mentioned that both you and Sally made changes to integer.c on different branches. If you look at Sally's log message for revision 344, you can see that she fixed some spelling errors. No doubt, your copy of the same file still has the same spelling errors. It's likely that your future changes to this file will be affecting the same areas that have the spelling errors, so you're in for some potential conflicts when you merge your branch someday. It's better, then, to receive Sally's change now, before you start working too heavily in the same places.

It's time to use the svn merge command. This command, it turns out, is a very close cousin to the svn diff command (which you read about in Chapter 3). Both commands are able to compare any two objects in the repository and describe the differences. For example, you can ask svn diff to show you the exact change made by Sally in revision 344:

$ svn diff -r 343:344 http://svn.example.com/repos/trunk/calc

Index: integer.c
===================================================================
--- integer.c	(revision 343)
+++ integer.c	(revision 344)
@@ -147,7 +147,7 @@
     case 6:  sprintf(info->operating_system, "HPFS (OS/2 or NT)"); break;
     case 7:  sprintf(info->operating_system, "Macintosh"); break;
     case 8:  sprintf(info->operating_system, "Z-System"); break;
-    case 9:  sprintf(info->operating_system, "CPM"); break;
+    case 9:  sprintf(info->operating_system, "CP/M"); break;
     case 10:  sprintf(info->operating_system, "TOPS-20"); break;
     case 11:  sprintf(info->operating_system, "NTFS (Windows NT)"); break;
     case 12:  sprintf(info->operating_system, "QDOS"); break;
@@ -164,7 +164,7 @@
     low = (unsigned short) read_byte(gzfile);  /* read LSB */
     high = (unsigned short) read_byte(gzfile); /* read MSB */
     high = high << 8;  /* interpret MSB correctly */
-    total = low + high; /* add them togethe for correct total */
+    total = low + high; /* add them together for correct total */
 
     info->extra_header = (unsigned char *) my_malloc(total);
     fread(info->extra_header, total, 1, gzfile);
@@ -241,7 +241,7 @@
      Store the offset with ftell() ! */
 
   if ((info->data_offset = ftell(gzfile))== -1) {
-    printf("error: ftell() retturned -1.\n");
+    printf("error: ftell() returned -1.\n");
     exit(1);
   }
 
@@ -249,7 +249,7 @@
   printf("I believe start of compressed data is %u\n", info->data_offset);
   #endif
   
-  /* Set postion eight bytes from the end of the file. */
+  /* Set position eight bytes from the end of the file. */
 
   if (fseek(gzfile, -8, SEEK_END)) {
     printf("error: fseek() returned non-zero\n");

The svn merge is almost exactly the same. Instead of printing the differences to your terminal, however, it applies them directly to your working copy as local modifications:

$ svn merge -r 343:344 http://svn.example.com/repos/trunk/calc
U  integer.c

$ svn status
M  integer.c

The output of svn merge shows that your copy of integer.c was patched. It now contains Sally's change — it has been "copied" from the trunk to your working copy of your private branch, and now exists as a local modification. At this point, it's up to you to review the local modification and make sure it works correctly.

In another scenario, it's possible that things may not have gone so well, and that integer.c may have entered a conflicted state. You might need to resolve the conflict using standard procedures (see Chapter 3), or if you decide that the merge was a bad idea altogether, simply give up and svn revert the local change.

But assuming the you've reviewed the merged change, you can svn commit the change as usual. At that point, the change has been merged into your repository branch. In version control terminology, this act of copying changes between branches is commonly called porting changes.

$ svn diff -r 343:344 http://svn.example.com/repos/trunk/calc > patchfile
$ patch -p0  < patchfile
Patching file integer.c using Plan A...
Hunk #1 succeeded at 147.
Hunk #2 succeeded at 164.
Hunk #3 succeeded at 241.
Hunk #4 succeeded at 249.
done

A word of warning: while svn diff and svn merge are very similar in concept, they do have different syntax in many cases. Be sure to read about them in Chapter 8 for details, or ask svn help. For example, svn merge requires a working-copy path as a target, i.e. a place where it should apply the tree-changes. If the target isn't specified, it assumes you are trying to perform one of the following common operations:

If you are merging a directory and haven't specified a target path, svn merge assumes the first case above and tries to apply the changes into your current directory. If you are merging a file, and that file (or a file by the same name) exists in your current working directory, svn merge assumes the second case and tries to apply the changes to a local file with the same name.

If you want changes applied somewhere else, you'll need to say so:

$ svn merge -r 343:344 http://svn.example.com/repos/trunk/calc my-calc-branch
U   my-calc-branch/integer.c

Merging changes sounds simple enough, but in practice it can become a headache. The problem is that if you repeatedly merge changes from one branch to another, you may accidentally merge the same change twice. When this happens, sometimes things will work fine. When patching a file, Subversion typically notices if the file already has the change, and does nothing. But if the already-existing change has been modified in any way, you'll get a conflict. Ideally, Subversion would automatically prevent applying changes more than once.

This is a problem that plagues many version control systems, including both CVS and Subversion. For now, the only way to avoid this problem in Subversion is to carefully keep track of which changes have been merged, and which haven't. When you create a branch directory, you need to track what revision it was created in — make a note somewhere to yourself. When you merge a revision (or range of revisions) into your working copy, you'll need to remember them as well. If you forget any of this information, you can rediscover it by examining the output of svn log -v branch-dir. But the point is that each subsequent merge needs to be carefully constructed by hand, making sure that previously-merged revisions aren't re-merged again.

Of course, Subversion has plans to solve this problem sometime after release 1.0. All of this merging information can be tracked in property metadata (see Section , “Properties”), and thus Subversion will someday be able to automatically avoid repeated merges.

To complete our running example, we'll move forward in time. Several days have passed, and many changes have happened on both the trunk and your private branch. Suppose that you've finished working on your private branch; the feature or bugfix is finally complete, and now you want to merge all of your branch changes back into the trunk for others to enjoy.

So how do we use the svn merge in this scenario? Remember that this command compares two trees, and applies the differences to a working copy. So to receive the changes, you need to have a working copy of the trunk. We'll assume that you still have your original one lying around (fully updated), or that you checked out a new working copy of /trunk/calc.

But what about the two trees to compare? At first glance, the answer may seem obvious: just compare the latest trunk tree with your latest branch tree. But beware — this assumption is wrong, and has burned many a new user! If you think about it, comparing the latest trunk and branch trees will not merely describe the set of changes you made to your branch. Such a comparison shows too many changes: it would not only show the addition of your branch changes, but also the removal of trunk changes that never happened on your branch.

To express only changes that happened on your branch, you need to compare two trees: the initial state of your branch, and the final state of your branch. Using svn log on your branch, you can see that your branch was created in revision 341. And the final state of your branch is simply a matter of using the HEAD revision.

Here's the final merging procedure, then:

$ cd trunk/calc

$ svn merge -r 341:HEAD http://svn.example.com/repos/branches/calc/my-calc-branch
U   integer.c
U   button.c
U   Makefile

$ svn status
M   integer.c
M   button.c
M   Makefile

[examine the diffs, compile, test, etc.]

$ svn commit -m "Merged all my-calc-branch changes into the trunk."
…

Once again, svn copy comes to the rescue. If you want to create a snapshot of /trunk/calc exactly as it looks in the HEAD revision, then make a copy of it:

$ svn copy http://svn.example.com/repos/trunk/calc \
           http://svn.example.com/repos/tags/calc/release-1.0 \
      -m "Tagging the 1.0 release of the 'calc' project."

Committed revision 351.

This example assumes that a /tags/calc directory already exists. After the copy completes, the new release-1.0 directory is forever a snapshot of how the project looked in the HEAD revision at the time you made the copy. Of course you might want to be more precise about exactly which revision you copy, in case somebody else may have committed changes to the project when you weren't looking. So if you know that revision 350 of /trunk/calc is exactly the snapshot you want, you can specify it by passing -r 350 to the svn copy command.

But wait a moment: isn't this tag-creation procedure the same procedure we used to create a branch? Yes, in fact, it is. In Subversion, there's no difference between a tag and a branch. Both are just ordinary directories that are created by copying. Just as with branches, the only reason a copied directory is a "tag" is because humans have decided to treat it that way: as long as nobody ever commits to the directory, it forever remains a snapshot. If people start committing to it, it becomes a branch.

If you are administering a repository, there are two approaches you can take to managing tags. The first approach is "hands off": as a matter of project policy, decide where your tags will live, and make sure all users know how to treat the directories they copy in there. (That is, make sure they know not to commit to them.) The second approach is more paranoid: you can use one of the access-control scripts provided with Subversion to prevent anyone from doing anything but creating new copies in the tags-area (###cross-ref a section that demonstrates how to use these scripts?). The paranoid approach, however, isn't usually necessary. If a user accidentally commits a change to a tag-directory, you can simply undo the change as discussed in the previous section. This is version control, after all.

Sometimes you may want your "snapshot" to be more complicated than a single directory at a single revision.

For example, pretend your project is much larger than our calc example: suppose it contains a number of subdirectories and many more files. In the course of your work, you may decide that you need to create a working copy that is designed to have specific features and bugfixes. You can accomplish this by selectively pinning files or directories at particular revisions (using svn update -r liberally), or by switching files and directories to particular branches (making use of svn switch). When you're done, your working copy is a hodgepodge of repository locations from different revisions. But after testing, you know it's the precise combination of data you need.

Time to make a snapshot. Copying one URL to another won't work here. In this case, you want to make a snapshot of your exact working copy arrangement and store it in the repository. Luckily, the svn copy actually has four different uses (which you can read about in Chapter 8), including the ability to copy a working-copy tree to the repository:

$ ls
./    ../   my-working-copy/

$ svn copy my-working-copy http://svn.example.com/repos/tags/calc/mytag

Committed revision 352.

Now there is a new directory in the repository, /tags/calc/mytag, which is an exact snapshot of your working copy — mixed revisions, urls, and all.

Other users have found interesting uses for this feature. Sometimes there are situations where you have a bunch of local changes made to your working copy, and you'd like a collaborator to see them. Instead of running svn diff and sending a patchfile (which won't capture tree changes), you can instead use svn copy to "upload" your working copy to a private area of the repository. Your collaborator can then either checkout a verbatim copy of your working copy, or use svn merge to receive your exact changes.

There are some standard, recommended ways to organize a repository. Most people create a trunk directory to hold the "main line" of development, a branches directory to contain branch copies, and a tags directory to contain tag copies. If a repository holds only one project, then often people create these top-level directories:

/trunk
/branches
/tags

If a repository contains multiple projects, people often index their layout by branch:

/trunk/paint
/trunk/calc
/branches/paint
/branches/calc
/tags/paint
/tags/calc

…or by project:

/paint/trunk
/paint/branches
/paint/tags
/calc/trunk
/calc/branches
/calc/tags

Of course, you're free to ignore these common layouts. You can create any sort of variation, whatever works best for you or your team. Remember that whatever you choose, it's not a permanent commitment. You can reorganize your repository at any time. Because branches and tags are ordinary directories, the svn move command can move or rename them however you wish. Switching from one layout to another is just a matter of issuing a series of server-side moves; if you don't like the way things are organized in the repository, just juggle the directories around.

Another nice feature of Subversion's model is that branches and tags can have finite lifetimes, just like any other versioned item. For example, suppose you eventually finish all your work on your personal branch of the calc project. After merging all of your changes back into /trunk/calc, there's no need for your private branch directory to stick around anymore:

$ svn delete http://svn.example.com/repos/branches/calc/my-calc-branch \
             -m "Removing obsolete branch of calc project."

Committed revision 375.

And now your branch is gone. Of course it's not really gone: the directory is simply missing from the HEAD revision, no longer distracting anyone. If you look at an earlier revision (by means of svn checkout -r, svn switch -r, or svn list -r), you'll still be able to see your old branch.

If browsing your deleted directory isn't enough, you can always bring it back. Resurrecting data is very easy in Subversion. If there's a deleted directory (or file) that you'd like to bring back into HEAD, simply use svn copy -r to copy it from the old revision:

$ svn copy -r 374 http://svn.example.com/repos/branches/calc/my-calc-branch \
                  http://svn.example.com/repos/branches/calc/my-calc-branch

Committed revision 376.

In our example, your personal branch had a relatively short lifetime: you may have created it to fix a bug or implement a new feature. When your task is done, so is the branch. In software development, though, it's also common to have two "main" branches running side-by-side for very long periods. For example, suppose it's time to release a stable calc project to the public, and you know it's going to take a couple of months to shake bugs out of the software. You don't want people to add new features to the project, but you don't want to tell all developers to stop programming either. So instead, you create a "stable" branch of the software that won't change much:

$ svn copy http://svn.example.com/repos/trunk/calc \
         http://svn.example.com/branches/calc/stable-1.0
         -m "Creating stable branch of calc project."

Committed revision 377.

And now developers are free to continue adding cutting-edge (or experimental) features to /trunk/calc, and you can declare a project policy that only bugfixes are to be committed to /branches/calc/stable-1.0. That is, as people continue to work on the trunk, a human selectively ports bugfixes over to the stable branch. Even after the stable branch has shipped, you'll probably continue to maintain the branch for a long time — that is, as long as you continue to support that release for customers.

Conceptually speaking, a Subversion repository is a sequence of directory trees. Each tree is a snapshot of how the files and directories versioned in your repository looked at some point in time. These snapshots are created as a result of client operations, and are called revisions.

Every revision begins life as a transaction tree. When doing a commit, a client builds a Subversion transaction that mirrors their local changes (plus any additional changes that might have been made to the repository since the beginning of the client's commit process), and then instructs the repository to store that tree as the next snapshot in the sequence. If the commit succeeds, the transaction is effectively promoted into a new revision tree, and is assigned a new revision number. If the commit fails for some reason, the transaction is destroyed and the client is informed of the failure.

Updates work in a similar way. The client builds a temporary transaction tree that mirrors the state of the working copy. The repository then compares that transaction tree with the revision tree at the requested revision (usually the most recent, or “youngest” tree), and sends back information that informs the client about what changes are needed to transform their working copy into a replica of that revision tree. After the update completes, the temporary transaction is deleted.

The use of transaction trees is the only way to make permanent changes to a repository's versioned filesystem. However, it's important to understand that the lifetime of a transaction is completely flexible. In the case of updates, transactions are temporary trees that are immediately destroyed. In the case of commits, transactions are transformed into permanent revisions (or removed if the commit fails). In the case of an error or bug, it's possible that a transaction can be accidentally left lying around in the repository (not really affecting anything, but still taking up space).

In theory, someday whole workflow applications might revolve around more fine-grained control of transaction lifetime. It is feasible to imagine a system whereby each transaction slated to become a revision is left in stasis well after the client finishes describing its changes to repository. This would enable each new commit to be reviewed by someone else, perhaps a manager or engineering QA team, who can choose to promote the transaction into a revision, or abort it.

What does all of this have to do with repository administration? The answer is simple: if you're administering a Subversion repository, you're going to have to examine revisions and transactions as part of monitoring the health of your repository.

Transactions and revisions in the Subversion repository can have properties attached to them. These properties are generic key-to-value mappings, and are used to store information about the tree to which they are attached. The names and values of these properties are stored in the repository's filesystem, along with the rest of your tree data.

Revision and transaction properties are useful for associating information with a tree that is not strictly related to the files and directories in that tree—the kind of information that isn't managed by client working copies. For example, when a new commit transaction is created in the repository, Subversion adds a property to that transaction named svn:date—a datestamp representing the time that the transaction was created. By the time the commit process is finished, and the transaction is promoted to a permanent revision, the tree has also been given a property to store the username of the revision's author (svn:author) and a property to store the log message attached to that revision (svn:log).

Revision and transaction properties are unversioned properties—as they are modified, their previous values are permanently discarded. Also, while revision trees themselves are immutable, the properties attached to those trees are not. You can add, remove, and modify revision properties at any time in the future. If you commit a new revision and later realize that you had some misinformation or spelling error in your log message, you can simply replace the value of the svn:log property with a new, corrected log message.

A hook is a program triggered by some repository event, such as the creation of a new revision or the modification of an unversioned property. Each hook is handed enough information to tell what that event is, what target(s) it's operating on, and the username of the person who triggered the event. Depending on the hook's output or return status, the hook program may continue the action, stop it, or suspend it in some way.

The hooks subdirectory is, by default, filled with templates for various repository hooks.

$ ls repos/hooks/
post-commit.tmpl          pre-revprop-change.tmpl
post-revprop-change.tmpl  start-commit.tmpl
pre-commit.tmpl           

There is one template for each hook that the Subversion repository implements, and by examining the contents of those template scripts, you can see what triggers each such script to run and what data is passed to that script. Also present in many of these templates are examples of how one might use that script, in conjunction with other Subversion-supplied programs, to perform common useful tasks. To actually install a working hook, you need only place some executable program or script into the repos/hooks directory which can be executed as the name (like start-commit or post-commit) of the hook.

On Unix platforms, this means supplying a script or program (which could be a shell script, a Python program, a compiled C binary, or any number of other things) named exactly like the name of the hook. Of course, the template files are present for more than just informational purposes—the easiest way to install a hook on Unix platforms is to simply copy the appropriate template file to a new file that lacks the .tmpl extension, customize the hook's contents, and ensure that the script is executable. Windows, however, uses file extensions to determine whether or not a program is executable, so you would need to supply a program whose basename is the name of the hook, and whose extension is one of the special extensions recognized by Windows for executable programs, such as .exe or .com for programs, and .bat for batch files.

Currently there are five true hooks implemented by the Subversion repository.

Subversion will attempt to execute hooks as the same user who owns the process which is accessing the Subversion repository. In most cases, the repository is being accessed via Apache HTTP server and mod_dav_svn, so this user is the same user that Apache runs as. The hooks themselves will need to be configured with OS-level permissions that allow that user to execute them. Also, this means that any file or programs (including the Subversion repository itself) accessed directly or indirectly by the hook will be accessed as the same user. In other words, be alert to potential permission-related problems that could prevent the hook from performing the tasks you've written it to perform.

The Berkeley DB environment has it own set of default configuration values for things like the number of locks allowed to be taken out at any given time, or the size cutoff for Berkeley's journaling log file, etc. Subversion's filesystem code additionally chooses default values for some of the Berkeley DB configuration options. However, sometimes your particular repository, with its unique collection of data and access patterns, might require a different set of configuration option values.

The folks at Sleepycat (the producers of Berkeley DB) understand that different databases have different requirements, and so they have provided a mechanism for runtime overriding of many of the configuration values for the Berkeley DB environment. Berkeley checks for the presence of a file named DB_CONFIG in each environment directory, and parses the options found in that file for use with that particular Berkeley environment.

The Berkeley configuration file for your repository is located in the db environment directory, at repos/db/DB_CONFIG. Subversion itself creates this file when it creates the rest of the repository. The file initially contains some default options, as well as pointers to the Berkeley DB online documentation so you can read about what those options do. Of course, you are free to add any of the supported Berkeley DB options to your DB_CONFIG file. Just be aware that while Subversion never attempts to read or interpret the contents of the file, and makes no use of the option settings in it, you'll want to avoid any configuration changes that may cause Berkeley DB to behave in a fashion that is unexpected by the rest of the Subversion code. Also, changes made to DB_CONFIG won't take effect until you recover the database environment (using svnadmin recover).

Your Subversion repository will generally require very little attention once it is configured to your liking. However, there are times when some manual assistance from an administrator might be in order. The svnadmin utility provides some helpful functionality to assist you in performing such tasks as

Perhaps the most commonly used of svnadmin's subcommands is setlog. When a transaction is committed to the repository and promoted to a revision, the descriptive log message associated with that new revision (and provided by the user) is stored as an unversioned property attached to the revision itself. In other words, the repository remembers only the latest value of the property, and discards previous ones.

Sometimes a user will have an error in her log message (a misspelling or some misinformation, perhaps). If the repository is configured (using the pre-revprop-change and post-revprop-change hooks; see Section , “Hook Scripts”) to accept changes to this log message after the commit is finished, then the user can “fix” her log message remotely using the svn program's propset command (see Chapter 8, Subversion Complete Reference). However, because of the potential to lose information forever, Subversion repositories are not, by default, configured to allow changes to unversioned properties— except by an administrator.

If a log message needs to be changed by an administrator, this can be done using svnadmin setlog. This command changes the log message (the svn:log property) on a given revision of a repository, reading the new value from a provided file.

$ echo "Here is the new, correct log message" > newlog.txt
$ svnadmin setlog myrepos newlog.txt -r 388

Another common use of svnadmin is to query the repository for outstanding—possibly dead—Subversion transactions. In the event that commit should fail, the transaction is usually cleaned up. That is, the transaction itself is removed from the repository, and any data associated with (and only with) that transaction is removed as well. Occasionally, though, a failure occurs in such a way that the cleanup of the transaction never happens. This could happen for several reasons: perhaps the client operation was inelegantly terminated by the user, or a network failure might have occurred in the middle of an operation, etc. Regardless of the reason, these dead transactions serve only to clutter the repository and consume resources.

You can use svnadmin's lstxns command to list the names of the currently outstanding transactions.

$ svnadmin lstxns myrepos
19
3a1
a45
$

Each item in the resultant output can then be used with svnlook (and its --transaction option) to determine who created the transaction, when it was created, what types of changes were made in the transaction—in other words, whether or not the transaction is a safe candidate for removal! If so, the transaction's name can be passed to svnadmin rmtxns, which will perform the cleanup of the transaction. In fact, the rmtxns subcommand can take its input directly from the output of lstxns!

$ svnadmin rmtxns myrepos `svnadmin lstxns myrepos`
$

If you use these two subcommands like this, you should consider making your repository temporarily inaccessible to clients. That way, no one can begin a legitimate transaction before you start your cleanup. The following is a little bit of shell-scripting that can quickly generate information about each outstanding transaction in your repository:

#!/bin/sh

### Generate informational output for all outstanding transactions in
### a Subversion repository

SVNADMIN=/usr/local/bin/svnadmin
SVNLOOK=/usr/local/bin/svnlook

REPOS=${1}
if [ x$REPOS = x ] ; then
  echo "usage: $0 REPOS_PATH"
  exit
fi

for TXN in `${SVNADMIN} lstxns ${REPOS}`; do 
  echo "---[ Transaction ${TXN} ]-------------------------------------------"
  ${SVNLOOK} info ${REPOS} --transaction ${TXN}
done

You can run the previous script using /path/to/txn-info.sh /path/to/repos. The output is basically a concatenation of several chunks of svnlook info output (see Section , “svnlook”), and will look something like:

$ txn-info.sh myrepos
---[ Transaction 19 ]-------------------------------------------
sally
2001-09-04 11:57:19 -0500 (Tue, 04 Sep 2001)
0
---[ Transaction 3a1 ]-------------------------------------------
harry
2001-09-10 16:50:30 -0500 (Mon, 10 Sep 2001)
39
Trying to commit over a faulty network.
---[ Transaction a45 ]-------------------------------------------
sally
2001-09-12 11:09:28 -0500 (Wed, 12 Sep 2001)
0
$

Usually, if you see a dead transaction that has no log message attached to it, this is the result of a failed update (or update-like) operation. These operations use Subversion transactions under the hood to mimic working copy state. Since they are never intended to be committed, Subversion doesn't require a log message for those transactions. Transactions that do have log messages attached are almost certainly failed commits of some sort. Also, a transaction's datestamp can provide interesting information—for example, how likely is it that an operation begun nine months ago is still active?

In short, transaction cleanup decisions need not be made unwisely. Various sources of information—including Apache's error and access logs, the logs of successful Subversion commits, and so on—can be employed in the decision-making process. Finally, an administrator can often simply communicate with a seemingly dead transaction's owner (via email, for example) to verify that the transaction is, in fact, in a zombie state.

In order to protect the data in your repository, the database back-end uses a locking mechanism. This mechanism ensures that portions of the database are not simultaneously modified by multiple database accessors, and that each process sees the data in the correct state when that data is being read from the database. When a process needs to change something in the database, it first checks for the existence of a lock on the target data. If the data is not locked, the process locks the data, makes the change it wants to make, and then unlocks the data. Other processes are forced to wait until that lock is removed before they are permitted to continue accessing that section of the database.

In the course of using your Subversion repository, fatal errors (such as running out of disk space or available memory) or interruptions can prevent a process from having the chance to remove the locks it has placed in the database. The result is that the back-end database system gets “wedged”. When this happens, any attempts to access the repository hang indefinitely (since each new accessor is waiting for a lock to go away—which isn't going to happen).

First, if this happens to your repository, don't panic. Subversion's filesystem takes advantage of database transactions and checkpoints and pre-write journaling to ensure that only the most catastrophic of events ^[10] can permanently destroy a database environment. A sufficiently paranoid repository administrator will be making off-site backups of the repository data in some fashion, but don't call your system administrator to restore a backup tape just yet.

Secondly, use the following recipe to attempt to “unwedge” your repository:

This procedure fixes almost every case of repository lock-up. Make sure that you run this command as the user that owns and manages the database, not just as root. Part of the recovery process might involve recreating from scratch various database files (shared memory regions, for example). Recovering as root will create those files such that they are owned by root, which means that even after you restore connectivity to your repository, regular users will be unable to access it.

If the previous procedure, for some reason, does not successfully unwedge your repository, you should do two things. First, move your broken repository out of the way and restore your latest backup of it. Then, send an email to the Subversion developer list (at <dev@subversion.tigris.org>) describing your problem in detail. Data integrity is an extremely high priority to the Subversion developers.

A Subversion filesystem has its data spread throughout various database tables in a fashion generally understood by (and of interest to) only the Subversion developers themselves. However, circumstances may arise that call for all, or some subset, of that data to be collected into a single, portable, flat file format. Subversion provides such a mechanism, implemented in a pair of svnadmin subcommands: dump and load.

The most common reason to dump and load a Subversion repository is due to changes in Subversion itself. As Subversion matures, there are times when certain changes made to the back-end database schema cause Subversion to be incompatible with previous versions of the repository. The recommended course of action when you are upgrading across one of those compatibility boundaries is a relatively simple process:

svnadmin dump will output a range of repository revisions that are formatted using Subversion's custom filesystem dump format. The dump format is printed to the standard output stream, while informative messages are printed to the standard error stream. This allows you to redirect the output stream to a file while watching the status output in your terminal window. For example:

$ svnlook youngest myrepos
26
$ svnadmin dump myrepos > dumpfile
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
…
* Dumped revision 25.
* Dumped revision 26.

At the end of the process, you will have a single file (dumpfile in the previous example) that contains all the data stored in your repository in the requested range of revisions.

The other subcommand in the pair, svnadmin load, parses the standard input stream as a Subversion repository dump file, and effectively replays those dumped revisions into the target repository for that operation. It also gives informative feedback, this time using the standard output stream:

$ svnadmin load newrepos < dumpfile
<<< Started new txn, based on original revision 1
     * adding path : A ... done.
     * adding path : A/B ... done.
     …
------- Committed new rev 1 (loaded from original rev 1) >>>

<<< Started new txn, based on original revision 2
     * editing path : A/mu ... done.
     * editing path : A/D/G/rho ... done.

------- Committed new rev 2 (loaded from original rev 2) >>>

…

<<< Started new txn, based on original revision 25
     * editing path : A/D/gamma ... done.

------- Committed new rev 25 (loaded from original rev 25) >>>

<<< Started new txn, based on original revision 26
     * adding path : A/Z/zeta ... done.
     * editing path : A/mu ... done.

------- Committed new rev 26 (loaded from original rev 26) >>>

Note that because svnadmin uses standard input and output streams for the repository dump and load process, people who are feeling especially saucy can try things like this (perhaps even using different versions of svnadmin on each side of the pipe):

$ svnadmin create newrepos
$ svnadmin dump myrepos | svnadmin load newrepos

We mentioned previously that svnadmin dump outputs a range of revisions. Use the --revision option to specify a single revision to dump, or a range of revisions. If you omit this option, all the existing repository revisions will be dumped.

$ svnadmin dump myrepos --revision 23 > rev-23.dumpfile
$ svnadmin dump myrepos --revision 100:200 > revs-100-200.dumpfile

As Subversion dumps each new revision, it outputs only enough information to allow a future loader to re-create that revision based on the previous one. In other words, for any given revision in the dump file, only the items that were changed in that revision will appear in the dump. The only exception to this rule is the first revision that is dumped with the current svnadmin dump command.

By default, Subversion will not express the first dumped revision as merely differences to be applied to the previous revision. For one thing, there is no previous revision in the dump file! And secondly, Subversion cannot know the state of the repository into which the dump data will be loaded (if it ever, in fact, occurs). To ensure that the output of each execution of svnadmin dump is self-sufficient, the first dumped revision is by default a full representation of every directory, file, and property in that revision of the repository.

However, you can change this default behavior. If you add the --incremental option when you dump your repository, svnadmin will compare the first dumped revision against the previous revision in the repository, the same way it treats every other revision that gets dumped. It will then output the first revision exactly as it does the rest of the revisions in the dump range—mentioning only the changes that occurred in that revision. The benefit of this is that you can create several small dump files that can be loaded in succession, instead of one large one, like so:

$ svnadmin dump myrepos 0 1000 > dumpfile1
$ svnadmin dump myrepos 1001 2000 --incremental > dumpfile2
$ svnadmin dump myrepos 2001 3000 --incremental > dumpfile3

These dump files could be loaded into a new repository with the following command sequence:

$ svnadmin load newrepos < dumpfile1
$ svnadmin load newrepos < dumpfile2
$ svnadmin load newrepos < dumpfile3

Another neat trick you can perform with this --incremental option involves appending to an existing dump file a new range of dumped revisions. For example, you might have a post-commit hook that simply appends the repository dump of the single revision that triggered the hook. Or you might have a script like the following that runs nightly to append dump file data for all the revisions that were added to the repository since the last time the script ran.

#!/usr/bin/perl

$repos_path  = '/path/to/repos';
$dumpfile    = '/usr/backup/svn-dumpfile';
$last_dumped = '/var/log/svn-last-dumped';
 
# Figure out the starting revision (0 if we cannot read the last-dumped file,
# else use the revision in that file incremented by 1).
if (open LASTDUMPED, "$last_dumped")
{
    $new_start = <LASTDUMPED>;
    chomp $new_start;
    $new_start++;
    close LASTDUMPED;
}
else
{
    $new_start = 0;
}

# Query the youngest revision in the repos.
$youngest = `svnlook youngest $repos_path`;
chomp $youngest;

# Do the backup.
`svnadmin dump $repos_path $new_start $youngest --incremental >> $dumpfile`;

# Store a new last-dumped revision
open LASTDUMPED, "> $last_dumped" or die;
print LASTDUMPED "$youngest\n";
close LASTDUMPED;

# All done!

Used like this, svnadmin's dump and load commands can be a valuable means by which to backup changes to your repository over time in case of a system crash or some other catastrophic event.

Finally, another possible use of the Subversion repository dump file format is conversion from a different storage mechanism or version control system altogether. Because the dump file format is, for the most part, human-readable, ^[11] it should be relatively easy to describe generic sets of changes—each of which should be treated as a new revision—using this file format.

Despite numerous advances in technology since the birth of the modern computer, one thing unfortunately rings true with crystalline clarity—sometimes, things go very, very awry. Power outages, network connectivity dropouts, corrupt RAM and crashed hard drives are but a taste of the evil that Fate is poised to unleash on even the most conscientious administrator. And so we arrive at a very important topic—how to make backup copies your repository data.

There are generally two types of backup methods available for Subversion repository administrators—incremental and full. We discussed in an earlier section of this chapter how to use svnadmin dump --incremental to perform an incremental backup (see Section , “Migrating a Repository”). Essentially, the idea is to only backup at a given time the changes to the repository since the last time you made a backup.

A full backup of the repository is quite literally a duplication of the entire repository directory (which includes the Berkeley database environment). Now, unless you temporarily disable all other access to your repository, simply doing a recursive directory copy runs the risk of generating a defunct backup, since someone might be currently writing to the database.

Fortunately, Sleepycat's Berkeley DB documents describe a certain order in which database files can be copied which will guarantee a valid backup copy. And better still, you don't have to implement that algorithm yourself, because the Subversion development team has already done so. The hot-backup.py script is found in the tools/backup/ directory of the Subversion source distribution. Given a repository path and a backup location, hot-backup.py will perform the necessary steps for backing up your live repository—without requiring that you bar public repository access at all—and then will clean out the dead Berkeley logfiles from your live repository.

Even if you also have an incremental backup, you might want to run this program on a regular basis. For example, you might consider adding hot-backup.py to a program scheduler (such as crond on Unix systems). Or, if you prefer fine-grained backup solutions, you could have your post-commit hook script call hot-backup.py (see Section , “Hook Scripts”), which will then cause a new backup of your repository to occur with every new revision created. Simply add the following to the hooks/post-commit script in your live repository directory:

(cd /path/to/hook/scripts; ./hot-backup.py ${REPOS} /path/to/backups &)

The resulting backup is a fully functional Subversion repository, able to be dropped in as a replacement for your live repository should something go horribly wrong.

There are benefits to both types of backup methods. The easiest is by far the full backup, which will always result in a perfect working replica of your repository. This again means that should something bad happen to your live repository, you can restore from the backup with a simple recursive directory copy. Unfortunately, if you are maintaining multiple backups of your repository, these full copies will each eat up just as much disk space as your live repository.

Incremental backups using the repository dump format are excellent to have on hand if the database schema changes between successive versions of Subversion itself. Since a full repository dump and load are generally required to upgrade your repository to the new schema, it's very convenient to already have half of that process (the dump part) finished. Unfortunately, the creation of—and restoration from—incremental backups takes longer, as each commit is effectively replayed into either the dumpfile or the repository.

In either backup scenario, repository administrators need to be aware of how modifications to unversioned revision properties affect their backups. Since these changes do not themselves generate new revisions, they will not trigger post-commit hooks, and may not even trigger the pre-revprop-change and post-revprop-change hooks. ^[12] And since you can change revision properties without respect to chronological order—you can change any revision's properties at any time—an incremental backup of the latest few revisions might not catch a property modification to a revision that was included as part of a previous backup.

Often, the best approach to repository backups is a diversified one. You can leverage combinations of full and incremental backups, plus archives of commit emails. The Subversion developers, for example, back up the Subversion source code repository after every new revision is created, and keep an archive of all the commit and property change notification emails. Your solution might be similar, but should be catered to your needs and that delicate balance of convenience with paranoia. And while all of this might not save your hardware from the iron fist of Fate, ^[13] it should certainly help you recover from those trying times.

Subversion's primary network server is the Apache HTTP Server (httpd), speaking the WebDAV/deltaV protocol. This protocol (an extension to HTTP 1.1; see http://www.webdav.org/) takes the ubiquitous HTTP protocol that is core of the World Wide Web, and adds writing—specifically, versioned writing—capabilities. The result is a standardized, robust system that is conveniently packaged as part of the Apache 2.0 software, is supported by numerous pieces of core operating system and third-party products, and which doesn't require network administrators to open up yet another custom port. ^[14]

Much of the following discussion includes references to Apache configuration directives. While some examples are given of the use of these directives, describing them in full is outside the scope of this chapter. The Apache team maintains excellent documentation, publicly available on their website at http://httpd.apache.org. For example, a general reference for the configuration directives is located at http://httpd.apache.org/docs-2.0/mod/directives.html.)

Also, as you make changes to your Apache setup, it is likely that somewhere along the way a mistake will be made. If you are not already familiar with Apache's logging subsystem, you should become aware of it. In your httpd.conf file are directives which specify the on-disk locations of the access and error logs generated by Apache (the CustomLog and ErrorLog directives, respectively). Subversion's mod_dav_svn uses Apache's error logging interface as well. You can always browse the contents of those files for information that might reveal the source of a problem which is not clearly noticeable otherwise.

LoadModule dav_svn_module     modules/mod_dav_svn.so

<Location /repos>
  DAV svn
  SVNPath /absolute/path/to/repository
</Location>

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
</Location>

$ ### First time: use -c to create the file
$ htpasswd -c /etc/svn-auth-file harry
New password: ***** 
Re-type new password: *****
Adding password for user harry
$ htpasswd /etc/svn-auth-file sally
New password: *******
Re-type new password: *******
Adding password for user sally
$

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file
</Location>

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file
  <LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
  </LimitExcept>
</Location>

ServerName svn.red-bean.com

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  SVNIndexXSLT "/svnindex.xsl"
  …
</Location>

As an alternative to Apache, Subversion also provides a stand-alone server program, svnserve. This program is considerably more lightweight than Apache, and much easier to configure. It speaks a custom protocol with Subversion clients over an ordinary TCP/IP connection.

There are two basic ways that svnserve can be used:

Note that these methods of svnserve usage aren't mutually exclusive; you can easily use both techniques on your server at the same time.

$ svnserve
( success ( 1 1 ( ANONYMOUS ) ( ) ) ) 

$ svnserve -d
$ # svnserve is still running, but the user is returned to the prompt 

$ svnserve -d -r /usr/local
…

$ svn checkout svn://example.com/repos/project
…

$ svn checkout svn+ssh://example.com/usr/local/repos/project
Password:
…

$ export SVN_SSH="ssh -l username"

[tunnels]
ssh = ssh -l username

When choosing between Apache HTTP Server and the custom svnserve program, there is no single correct decision. Depending on your particular requirements, one of the available solutions might seem more right for you. And in fact, these servers can run in parallel, each accessing your repositories in its own way, and each without hindering the other. To recap the previous two sections, here's a brief list of the highlights of the two available Subversion servers—choose whatever works best for you and your users.

While Subversion allows you to move around versioned files and directories without any loss of information, doing so can still disrupt the workflow of those who access the repository often and come to expect things to be at certain locations. Try to peer into the future a bit; plan ahead before placing your data under version control. By “laying out” the contents of your repositories in an effective manner the first time, you can prevent a load of future headaches.

There are a few things to consider when setting up Subversion repositories. Let's assume that as repository administrator, you will be responsible for supporting the version control system for several projects. The first decision is whether to use a single repository for multiple projects, or to give each project its own repository, or some compromise of these two.

There are benefits to using a single repository for multiple projects, most obviously the lack of duplicated maintenance. A single repository means that there is one set of hook scripts, one thing to routinely backup, one thing to dump and load if Subversion releases an incompatible new version, and so on. Also, you can easily move data between projects easily, and without losing any historical versioning information.

The downside of using a single repository is that different projects may have different commit mailing lists or different authentication and authorization requirements. Also, remember that Subversion uses repository-global revision numbers. Some folks don't like the fact that even though no changes have been made to their project lately, the youngest revision number for the repository keeps climbing because other projects are actively adding new revisions.

A middle-ground approach can be taken, too. For example, projects can be grouped by how well they relate to each other. You might have a few repositories with a handful of projects in each repository. That way, projects that are likely to want to share data can do so easily, and as new revisions are added to the repository, at least the developers know that those new revisions are at least remotely related to everyone who uses that repository.

After deciding how to organize your projects with respect to repositories, you'll probably want to think about directory hierarchies in the repositories themselves. Because Subversion uses regular directory copies for branching and tagging (see Chapter 4, Branching and Merging), the Subversion community recommends using one of two different approaches here. Both approaches employ the use of directories named trunk, meaning the directory under which the main project development occurs, and branches, which is a directory in which to create various named branches of the main development line.

The first is to place each project in a subdirectory of the root filesystem directory, with trunk and branches directories immediately under each project directory, as demonstrated in Figure 5-1.

Figure 5.1. One Suggested Repository Layout.

The second is do the reverse—to have the trunk and branches directories immediately in the top level of the filesystem, each with subdirectories for all the projects in the repository, as demonstrated in Figure 5-2.

Figure 5.2. Another Suggested Repository Layout.

Lay out your repository in whatever way you see fit. Subversion does not expect or enforce a layout schema—in its eyes, a directory is a directory is a directory. Ultimately, you should choose the repository arrangement that meets your needs and those of the projects that live there.

After deciding how to arrange the projects in your repository, you'll probably want to actually populate the repository with that layout and with initial project data. There are a couple of ways to do this in Subversion. You could use the svn mkdir command (see Chapter 8, Subversion Complete Reference) to create each directory in your skeletal repository layout, one-by-one. A quicker way to accomplish the same task is to use the svn import command (see Section , “svn import”. By first creating the layout in a temporary location on your drive, you can import the whole layout tree into the repository in a single commit:

$ mkdir tmpdir
$ cd tmpdir
$ mkdir projectA
$ mkdir projectA/trunk
$ mkdir projectA/branches
$ mkdir projectB
$ mkdir projectB/trunk
$ mkdir projectB/branches
…
$ svn import file:///path/to/repos --message 'Initial repository layout'
Adding         projectA
Adding         projectA/trunk
Adding         projectA/branches
Adding         projectB
Adding         projectB/trunk
Adding         projectB/branches

Committed revision 1.
$ cd ..
$ rm -rf tmpdir
$

Once you have your skeletal layout in place, you can begin importing actual project data into your repository, if any such data exists yet. Once again, there are several ways to do this. You could use the svn import command. You could checkout a working copy from your new repository, move and arrange project data inside the working copy, and use the svn add and svn commit commands. But once we start talking about such things, we're no longer discussing repository administration. If you aren't already familiar with the svn client program, see Chapter 3, Guided Tour.

The first time that the svn command-line client is executed, it creates a per-user configuration area. On Unix-like systems, this area appears as a directory named .subversion in the user's home directory. On Win32 systems, Subversion creates a folder named Subversion, typically inside the Application Data area of the user's profile directory. However, on this platform the exact location differs from system to system, and is dictated by the Windows registry. We will refer to the per-user configuration area using its Unix name, .subversion.

In addition to the per-user configuration area, Subversion also recognizes the existence of a system-wide configuration area. The gives system administrators the ability to establish defaults for all users on a given machine. Note that the system-wide configuration area does not alone dictate mandatory policy—the settings in the per-user configuration area override those in the system-wide one, and command-line arguments supplied to svn program have the final word on behavior. On Unix-like platforms, the system-wide configuration area is expected to be the /etc/subversion directory; on Windows machines, it again looks for a Subversion directory inside the common Application Data location (again, as specified by the Windows Registry). Unlike the per-user case, the svn program does not attempt to create the system-wide configuration area.

The .subversion directory currently contains three files—two configuration files (config and servers), and a README.txt file which describes the INI format. At the time of their creation, the files contain default values for each of the supported Subversion options, mostly commented out and grouped with textual descriptions about how the values for the key affect Subversion's behavior. To change a certain behavior, you need only to load the appropriate configuration file into a text editor, and modify the desired option value. If at any time you wish to have the default settings restore to one or more of the configuration files, you can simply delete the file, and run some innocuous svn command, such as svn --version, and the missing file will be recreated in its default state.

The per-user configuration area also contains a cache of authentication data. The auth directory holds a set of subdirectories that contain pieces of cached information used by Subversion's various supported authentication methods. This directory is created in such a way that only the user herself has permission to read its contents.

In addition to the usual INI-based configuration area, Subversion clients running on Windows platforms may also use the Windows registry to hold the configuration data. The option names and their values are the same as in the INI files. The “file/section” hierarchy is preserved as well, though addressed in a slightly different fashion—in this schema, files and sections are just levels in the registry key tree.

Subversion looks for system-wide configuration values under the HKEY_LOCAL_MACHINE\Software\Tigris.org\Subversion key. For example, the global-ignores option, which is in the miscellany section of the config file, would be found at HKEY_LOCAL_MACHINE\Software\Tigris.org\Subversion\Config\Miscellany\global-ignores. Per-user configuration values should be stored under HKEY_CURRENT_USER\Software\Tigris.org\Subversion.

Registry-based configuration options are parsed before their file-based counterparts, so are overridden by values found in the configuration files. In other words, configuration priority is granted in the following order on a Windows system:

Also, the Windows Registry doesn't really support the notion of something being “commented out”. However, Subversion will ignore any option key whose name begins with a hash (#) character. This allows you to effectively comment out a Subversion option without deleting the entire key from the Registry, obviously simplifying the process of restoring that option.

The svn command-line client never attempts to write to the Windows Registry, and will not attempt to create a default configuration area there. You can create the keys you need using the REGEDIT program. Alternatively, you can create a .REG file, and then double-click on that file from the Explorer shell, which will cause the data to be merged into your registry.

REGEDIT4

[HKEY_LOCAL_MACHINE\Software\Tigris.org\Subversion\Servers\DEFAULT]
"#http-proxy-host"=""
"#http-proxy-port"=""
"#http-proxy-username"=""
"#http-proxy-password"=""
"#http-proxy-timeout"="0"
"#http-compression"="yes"

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\auth]
"#store-password"="no"

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\helpers]
"#editor-cmd"="notepad"

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\miscellany]
"#global-ignores"="*.o *.lo *.la #*# .*.rej *.rej .*~ *~ .#*"

The previous example shows the contents of a .REG file which contains some of most commonly used configuration options and their default values. Note the presence of both system-wide (for network proxy-related options) and per-user settings (editor programs and password storage, among others). Also note that all the options are effectively commented out. You need only to remove the hash (#) character from the beginning of the option names, and set the values as you desire.

In this section, we will discuss the specific run-time configuration options which are currently supported by Subversion.

[groups]
beanie-babies = *.red-bean.com
collabnet = svn.collab.net

[beanie-babies]
…

[collabnet]
…

Properties can be very useful additions to your working copy. In fact, Subversion itself uses properties to house special information, and as a way to denote that certain special processing might be needed. Likewise, you can use properties for your own purposes. Of course, anything you can do with properties you could also do using regular versioned files, but consider the following example of Subversion property use.

Say you wish to design a website that houses many digital photos, and displays them with captions and a datestamp. Now, your set of photos is constantly changing, so you'd like to have as much of this site automated as possible. These photos can be quite large, so as is common with sites of this nature, you want to provide smaller thumbnail images to your site visitors. You can do this with traditional files. That is, you can have your image123.jpg and an image123-thumbnail.jpg side-by-side in a directory. Or if you want to keep the filenames the same, you might have your thumbnails in a different directory, like thumbnails/image123.jpg. You can also store your captions and datestamps in a similar fashion, again separated from the original image file. Soon, your tree of files is a mess, and grows in multiples with each new photo added to the site.

Now consider the same setup using Subversion's file properties. Imagine having a single image file, image123.jpg, and then properties set on that file named caption, datestamp, and even thumbnail. Now your working copy directory looks much more manageable—in fact, it looks like there are nothing but image files in it. But your automation scripts know better. They know that they can use svn (or better yet, they can use the Subversion language bindings—see Section , “Using Languages Other than C and C++”) to dig out the extra information that your site needs to display without having to read an index file or play path manipulation games.

How (and if) you use Subversion properties is up to you. As we mentioned, Subversion has it own uses for properties, which we'll discuss a little later in this chapter. But first, let's discuss how to manipulate options using the svn program.

The svn command affords a few ways to add or modify file and directory properties. For properties with short, human-readable values, perhaps the simplest way to add a new property is to specify the property name and value on the command-line of propset subcommand.

$ svn propset copyright '(c) 2003 Red-Bean Software' calc/button.c
property `copyright' set on 'calc/button.c'
$

But we've been touting the flexibility that Subversion offers for your property values. And if you are planning to have a multi-line textual, or even binary, property value, you probably do not want to supply that value on the command-line. So the propset subcommand takes a --file (-F) option for specifying the name of a file which contains the new property value.

$ svn propset license -F /path/to/LICENSE calc/button.c
property `license' set on 'calc/button.c'
$

In addition to the propset command, the svn program supplies the propedit command. This command uses the configured editor program (see Section , “Config”) to add or modify properties. When you run the command, svn invokes your editor program on a temporary file that contains the current value of the property (or which is empty, if you are adding a new property). Then, you just modify that value in your editor program until it represents the new value you wish to store for the property, save the temporary file, and then exit the editor program. If Subversion detects that you've actually changed the existing value of the property, it will accept that as the new property value. If you exit your editor without making any changes, no property modification will occur.

$ svn propedit copyright calc/button.c  ### exit the editor without changes
No changes to property `copyright' on `calc/button.c'
$

We should note that, as with other svn subcommands, those related to properties can act on multiple paths at once. This enables you to modify properties on whole sets of files with a single command. For example, we could have done:

$ svn propset copyright '(c) 2002 Red-Bean Software' calc/*
property `copyright' set on 'calc/Makefile'
property `copyright' set on 'calc/button.c'
property `copyright' set on 'calc/integer.c'
…
$

All of this property adding and editing isn't really very useful if you can't easily get the stored property value. So the svn program supplies two subcommands for displaying the names and values of properties stored on files and directories. The svn proplist command will list the names of properties which exist on a path. Once you know the names of the properties on the node, you can request their values individually using svn propget. This command will, given a path (or set of paths) and a property name, print the value of the property to the standard output stream.

$ svn proplist calc/button.c
Properties on 'calc/button.c':
  copyright
  license
$ svn propget copyright calc/button.c
(c) 2003 Red-Bean Software

There's even a variation of the proplist command which will list both the name and value of all of the properties. Simply supply the --verbose (-v) option.

$ svn proplist --verbose calc/button.c
Properties on 'calc/button.c':
  copyright : (c) 2003 Red-Bean Software
  license : ================================================================
Copyright (c) 2003 Red-Bean Software.  All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions 
are met:

1. Redistributions of source code must retain the above copyright
notice, this list of conditions, and the recipe for Fitz's famous
red-beans-and-rice.
…

The last property-related subcommand is propdel. Since Subversion allows you to store properties with empty values, you can't remove a property altogether using propedit or propset. For example, this command will not yield the desired effect:

$ svn propset license '' calc/button.c
property `license' set on 'calc/button.c'
$ svn proplist --verbose calc/button.c
Properties on 'calc/button.c':
  copyright : (c) 2003 Red-Bean Software
  license : 
$

You need to use the propdel command to delete properties altogether. The syntax is similar to the other property commands:

$ svn propdel license calc/button.c
property `license' deleted from ''.
$ svn proplist --verbose calc/button.c
Properties on 'calc/button.c':
  copyright : (c) 2003 Red-Bean Software
$

Now that you are now familiar with all of the property-related svn subcommands, let's see how property modifications affect the usual Subversion workflow. As we mentioned earlier, file and directory properties are versioned, just like your file contents. As a result, Subversion provides the same opportunities for merging—in cleanly or conflicting fashions—someone else's modifications into your own.

$ svn propset svn:log '* button.c: Fix a compiler warning.' -r11 --revprop
property `svn:log' set on repository revision '11'
$

And as with file contents, your property changes are local modifications, only made permanent when you commit them to the repository with svn commit. Your property changes can be easily unmade, too—the svn revert command will restore your files and directories to their un-edited states, contents, properties, and all. Also, you can receive interesting information about the state of your file and directory properties by using the svn status and svn diff commands.

$ svn status calc/button.c
 M     calc/button.c
$ svn diff calc/button.c
Property changes on: calc/button.c
___________________________________________________________________
Name: copyright
   + (c) 2003 Red-Bean Software

$

Notice how the status subcommand displays M in the second column instead of the first. That is because we have modified the properties on calc/button.c, but not modified its textual contents. Had we changed both, we would have seen M in the first column, too (see Section , “svn status”).

% svn update calc
M  calc/Makefile.in
 C calc/button.c
Updated to revision 143.
$ 

$ svn status calc
 C     calc/button.c
?      calc/button.c.prej
$ cat calc/button.c.prej 
prop `linecount': user set to '1256', but update set to '1301'.
$

You might also have noticed the non-standard way that Subversion currently displays property differences. You can still run svn diff and redirect the output to create a usable patch file. The patch program will ignore property patches—as a rule, it ignores any noise it can't understand. This does unfortunately mean that to fully apply a patch generated by svn diff, any property modifications will need to be applied by hand.

As you can see, the presence of property modifications has no outstanding effect on the typical Subversion workflow. Your general patterns of updating your working copy, checking the status of your files and directories, reporting on the modifications you have made, and committing those modifications to the repository are completely immune to the presence or absence of properties. The svn program has some additional subcommands for actually making property changes, but that is the only noticeable asymmetry.

Subversion has no particular policy regarding properties—you can use them for any purpose. Subversion asks only that you not use property names that begin with the prefix svn:. That's the namespace that its sets aside for its own use. In fact, Subversion defines certain properties that have magical effects on the files and directories to which they are attached. In this section, we'll untangle the mystery, and describe how these special properties make your life just a little easier.

$ svn propset svn:ignore -F .cvsignore .
property `svn:ignore' set on '.'
$

$ svn status calc
 M     calc/button.c
?      calc/calculator
?      calc/data.c
?      calc/debug_log
?      calc/debug_log.1
?      calc/debug_log.2.gz
?      calc/debug_log.3.gz

calculator
debug_log*

$ svn status
 M     calc
 M     calc/button.c
?      calc/data.c

$ svn status --no-ignore
 M     calc/button.c
I      calc/calculator
?      calc/data.c
I      calc/debug_log
I      calc/debug_log.1
I      calc/debug_log.2.gz
I      calc/debug_log.3.gz

Here is the latest report from the front lines.
$LastChangedDate$
$Rev$
Cumulus clouds are appearing more frequently as summer approaches.

$ svn propset svn:keywords "LastChangedDate Author" weather.txt
property `svn:keywords' set on 'weather.txt'
$

Here is the latest report from the front lines.
$LastChangedDate: 2002-07-22 21:42:37 -0700 (Mon, 22 Jul 2002) $
$Rev$
Cumulus clouds are appearing more frequently as summer approaches.

Managing vendor branches generally works like this. You create a top-level directory (such as /vendor) to hold the vendor branches. Then you import the third party code into a subdirectory of that top-level directory. You then copy that subdirectory into your main development branch (for example, /trunk) at the appropriate location. You always make your local changes in the main development branch. With each new release of the code you are tracking you bring it into the vendor branch and merge the changes into /trunk, resolving whatever conflicts occur between your local changes and the upstream changes.

Perhaps an example will help to clarify this algorithm. We'll use a scenario where your development team is creating a calculator program that links against a third-party complex number arithmetic library, libcomplex. We'll begin with the initial creation of the vendor branch, and the import of the first vendor drop.

…
$ svn import /path/to/libcomplex-1.0 \
             http://svn.example.com/repos/calc/vendor/libcomplex/current \
             -m 'importing initial 1.0 vendor drop'
…

We now have the current version of the libcomplex source code in /vendor/libcomplex/current. Now, we tag that version (see Section , “Tags”) and then copy it into the main development branch so we can make our customizations to it.

$ svn copy http://svn.example.com/repos/calc/vendor/libcomplex/current  \
           http://svn.example.com/repos/calc/vendor/libcomplex/1.0      \
           -m 'tagging libcomplex-1.0'
…
$ svn copy http://svn.example.com/repos/vendor/libcomplex/1.0  \
           http://svn.example.com/repos/calc/libcomplex        \
           -m 'bringing libcomplex-1.0 into the main branch'
…

We check out our project's main branch—which now includes a copy of the first vendor drop—and we get to work customizing the libcomplex code. Before we know it, our modified version of libcomplex is now completely integrated into our calculator program. ^[24]

A few weeks later, the developers of libcomplex release a new version of their library—version 1.1—which contains some features and functionality that we really want. But we'd like to upgrade to this new version without losing our customizations to the existing version. As you might have guessed, what we essentially would like to do is to replace our current baseline version of libcomplex 1.0 with a copy of libcomplex 1.1, and then re-apply the custom modifications we previously made to that library to the new version.

To perform this upgrade, we checkout a copy of our vendor branch, and replace the current version with the new libcomplex 1.1 source code. After committing this change, our current branch now contains the new vendor drop. We tag the new version, and then merge the differences between the tag of the previous version and the new current version into our main development branch.

$ cd working-copies/calc
$ svn merge http://svn.example.com/repos/vendor/libcomplex/1.0      \
            http://svn.example.com/repos/vendor/libcomplex/current  \
            libcomplex
… # resolve all the conflicts between their changes and our changes
$ svn commit -m 'merging libcomplex-1.1 into the main branch'
…

In the trivial use-case, the new version of our third-party tool would look, from a files-and-directories point of view, just like the previous version. In other words, none of the libcomplex source files would have been deleted, renamed or moved to different locations—in short, in a perfect world, our modifications would apply cleanly to the new version of the library, with absolutely no complications or conflicts.

But things aren't always that simple, and in fact it is quite common for source files to get moved around between releases of software. This complicates the process of ensuring that our modifications are still valid for the new version of code, and can quickly degrade into a situation where we have to manually recreate our customizations in the new version. Once Subversion knows about the history of a given source file—including all its previous locations—the process of merging in the new version of the library is pretty simple. But we are responsible for telling Subversion how the source file layout changed from vendor drop to vendor drop.

Vendor drops that contain more than a few deletes, additions and moves complicate the process of upgrading to each successive version of the third-party data. So Subversion supplies the svn_load_dirs.pl script to assist with this process. This script automates the importing steps we mentioned in the general vendor branch management procedure to make sure that mistakes are minimized. You will still be responsible for using the merge commands to merge the new versions of the third-party data into your main development branch, but svn_load_dirs.pl can help you more quickly and easily arrive at that stage.

In short, svn_load_dirs.pl is an enhancement to svn import that has several important characteristics:

svn_load_dirs.pl takes three mandatory arguments. The first argument is the URL to the base Subversion directory to work in. This argument is followed by the URL—relative the first argument—into which the current vendor drop will be imported. Finally, the third argument is the local directory to import. Using our previous example, a typical run of svn_load_dirs.pl might look like:

$ svn_load_dirs.pl http://svn.example.com/repos/calc/vendor/libcomplex \
                   current                                             \
                   /path/to/libcomplex-1.1
…

You can indicate that you'd like svn_load_dirs.pl to tag the new vendor drop by passing the -t command-line option and specifying a tag name. This tag is another URL relative to the first program argument.

$ svn_load_dirs.pl -t libcomplex-1.1                                   \
                   http://svn.example.com/repos/calc/vendor/libcomplex \
                   current                                             \
                   /path/to/libcomplex-1.1
…

When you run svn_load_dirs.pl, it examines the contents of your existing “current” vendor drop, and compares them with the proposed new vendor drop. In the trivial case, there will be no files that are in one version and not the other, and the script will perform the new import without incident. If, however, there are discrepancies in the file layouts between versions, svn_load_dirs.pl will prompt you for how you would like to resolve those differences. For example, you will have the opportunity to tell the script that you know that the file math.c in version 1.0 of libcomplex was renamed to arithmetic.c in libcomplex 1.1. Any discrepancies not explained by moves are treated as regular additions and deletions.

The script also accepts a separate configuration file for setting properties on files and directories matching a regular expression that are added to the repository. This configuration file is specified to svn_load_dirs.pl using the -p command-line option. Each line of the configuration files is a whitespace-delimited set of two or four values: a Perl-style regular expression to match the added path against, a control keyword (either break or cont), and then optionally a property name and value.

\.png$              break   svn:mime-type   image/png
\.jpe?g$            break   svn:mime-type   image/jpeg
\.m3u$              cont    svn:mime-type   audio/x-mpegurl
\.m3u$              break   svn:eol-style   LF
.*                  break   svn:eol-style   native

For each added path, the configured property changes whose regular expression matches the path are applied in order, unless the control specification is break (which means that no more property changes should be applied to that path). If the control specification is cont—an abbreviation for continue—then matching will continue with the next line of the configuration file.

Any whitespace in the regular expression, property name, or property value must be surrounded by either single or double quote characters. You can escape quote characters that are not used for wrapping whitespace by preceding them with a backslash (\) character. The backslash escapes only quotes when parsing the configuration file, so do not protect any other characters beyond what is necessary for the regular expression.

When referring to Subversion's Repository Layer, we're generally talking about two libraries—the repository library, and the filesystem library. These libraries provide the storage and reporting mechanisms for the various revisions of your version-controlled data. This layer is connected to the Client Layer via the Repository Access Layer, and is, from the perspective of the Subversion user, the stuff at the "other end of the line."

The Subversion Filesystem is accessed via the libsvn_fs API, and is not a kernel-level filesystem that one would install in an operating system (like the Linux ext2 or NTFS), but a virtual filesystem. Rather than storing "files" and "directories" as real files and directories (as in, the kind you can navigate through using your favorite shell program), it uses a database system for its back-end storage mechanism. Currently, the database system in use is Berkeley DB. ^[25] However, there has been considerable interest by the development community in giving future releases of Subversion the ability to use other back-end database systems, perhaps through a mechanism such as Open Database Connectivity (ODBC).

The filesystem API exported by libsvn_fs contains the kinds of functionality you would expect from any other filesystem API: you can create and remove files and directories, copy and move them around, modify file contents, and so on. It also has features that are not quite as common, such as the ability to add, modify, and remove metadata ("properties") on each file or directory. Furthermore, the Subversion Filesystem is a versioning filesystem, which means that as you make changes to your directory tree, Subversion remembers what your tree looked like before those changes. And before the previous changes. And the previous ones. And so on, all the way back through versioning time to (and just beyond) the moment you first started adding things to the filesystem.

All the modifications you make to your tree are done within the context of a Subversion transaction. The following is a simplified general routine for modifying your filesystem:

Once you have committed your transaction, your filesystem modifications are permanently stored as historical artifacts. Each of these cycles generates a single new revision of your tree, and each revision is forever accessible as an immutable snapshot of "the way things were."

Most of the functionality provided by the filesystem interface comes as an action that occurs on a filesystem path. That is, from outside of the filesystem, the primary mechanism for describing and accessing the individual revisions of files and directories comes through the use of path strings like /foo/bar, just as if you were addressing files and directories through your favorite shell program. You add new files and directories by passing their paths-to-be to the right API functions. You query for information about them by the same mechanism.

Unlike most filesystems, though, a path alone is not enough information to identify a file or directory in Subversion. Think of a directory tree as a two-dimensional system, where a node's siblings represent a sort of left-and-right motion, and descending into subdirectories a downward motion. Figure 7-2 shows a typical representation of a tree as exactly that.

Figure 7.2. Files and Directories in Two Dimensions

Of course, the Subversion filesystem has a nifty third dimension that most filesystems do not have—Time! ^[26] In the filesystem interface, nearly every function that has a path argument also expects a root argument. This svn_fs_root_t argument describes either a revision or a Subversion transaction (which is usually just a revision-to-be), and provides that third-dimensional context needed to understand the difference between /foo/bar in revision 32, and the same path as it exists in revision 98. Figure 7-3 shows revision history as an added dimension to the Subversion filesystem universe.

Figure 7.3. Revisioning Time—the Third Dimension!

As we mentioned earlier, the libsvn_fs API looks and feels like any other filesystem, except that it has this wonderful versioning capability. It was designed to be usable by any program interested in a versioning filesystem. Not coincidentally, Subversion itself is interested in that functionality. But while the filesystem API should be sufficient for basic file and directory versioning support, Subversion wants more—and that is where libsvn_repos comes in.

The Subversion repository library (libsvn_repos) is basically a wrapper library around the filesystem functionality. This library is responsible for creating the repository layout, making sure that the underlying filesystem is initialized, and so on. Libsvn_repos also implements a set of hooks—scripts that are executed by the repository code when certain actions take place. These scripts are useful for notification, authorization, or whatever purposes the repository administrator desires. This type of functionality, and other utility provided by the repository library, is not strictly related to implementing a versioning filesystem, which is why it was placed into its own library.

Developers who wish to use the libsvn_repos API will find that it is not a complete wrapper around the filesystem interface. That is, only certain major events in the general cycle of filesystem activity are wrapped by the repository interface. Some of these include the creation and commit of Subversion transactions, and the modification of revision properties. These particular events are wrapped by the repository layer because they have hooks associated with them. In the future, other events may be wrapped by the repository API. All of the remaining filesystem interaction will continue to occur directly with libsvn_fs API, though.

For example, here is a code segment that illustrates the use of both the repository and filesystem interfaces to create a new revision of the filesystem in which a directory is added. Note that in this example (and all others throughout this book), the SVN_ERR macro simply checks for a non-successful error return from the function it wraps, and returns that error if it exists.

/* Create a new directory at the path NEW_DIRECTORY in the Subversion
   repository located at REPOS_PATH.  Perform all memory allocation in
   POOL.  This function will create a new revision for the addition of
   NEW_DIRECTORY.  */
static svn_error_t *
make_new_directory (const char *repos_path,
                    const char *new_directory,
                    apr_pool_t *pool)
{
  svn_error_t *err;
  svn_repos_t *repos;
  svn_fs_t *fs;
  svn_revnum_t youngest_rev;
  svn_fs_txn_t *txn;
  svn_fs_root_t *txn_root;
  const char *conflict_str;

  /* Open the repository located at REPOS_PATH.  */
  SVN_ERR (svn_repos_open (&repos, repos_path, pool));

  /* Get a pointer to the filesystem object that is stored in
     REPOS.  */
  fs = svn_repos_fs (repos);

  /* Ask the filesystem to tell us the youngest revision that
     currently exists.  */
  SVN_ERR (svn_fs_youngest_rev (&youngest_rev, fs, pool));

  /* Begin a new transaction that is based on YOUNGEST_REV.  We are
     less likely to have our later commit rejected as conflicting if we
     always try to make our changes against a copy of the latest snapshot
     of the filesystem tree.  */
  SVN_ERR (svn_fs_begin_txn (&txn, fs, youngest_rev, pool));

  /* Now that we have started a new Subversion transaction, get a root
     object that represents that transaction.  */
  SVN_ERR (svn_fs_txn_root (&txn_root, txn, pool));
  
  /* Create our new directory under the transaction root, at the path
     NEW_DIRECTORY.  */
  SVN_ERR (svn_fs_make_dir (txn_root, new_directory, pool));

  /* Commit the transaction, creating a new revision of the filesystem
     which includes our added directory path.  */
  err = svn_repos_fs_commit_txn (&conflict_str, repos, 
                                 &youngest_rev, txn);
  if (! err)
    {
      /* No error?  Excellent!  Print a brief report of our success.  */
      printf ("Directory '%s' was successfully added as new revision "
              "'%" SVN_REVNUM_T_FMT "'.\n", new_directory, youngest_rev);
    }
  else if (err->apr_err == SVN_ERR_FS_CONFLICT))
    {
      /* Uh-oh.  Our commit failed as the result of a conflict
         (someone else seems to have made changes to the same area 
         of the filesystem that we tried to modify).  Print an error
         message.  */
      printf ("A conflict occurred at path '%s' while attempting "
              "to add directory '%s' to the repository at '%s'.\n", 
              conflict_str, new_directory, repos_path);
    }
  else
    {
      /* Some other error has occurred.  Print an error message.  */
      printf ("An error occurred while attempting to add directory '%s' "
              "to the repository at '%s'.\n", 
              new_directory, repos_path);
    }

  /* Return the result of the attempted commit to our caller.  */
  return err;
} 

In the previous code segment, calls were made to both the repository and filesystem interfaces. We could just as easily have committed the transaction using svn_fs_commit_txn. But the filesystem API knows nothing about the repository library's hook mechanism. If you want your Subversion repository to automatically perform some set of non-Subversion tasks every time you commit a transaction (like, for example, sending an email that describes all the changes made in that transaction to your developer mailing list), you need to use the libsvn_repos-wrapped version of that function—svn_repos_fs_commit_txn. This function will actually first run the "pre-commit" hook script if one exists, then commit the transaction, and finally will run a "post-commit" hook script. The hooks provide a special kind of reporting mechanism that does not really belong in the core filesystem library itself. (For more information regarding Subversion's repository hooks, see Section , “Hook Scripts”.)

The hook mechanism requirement is but one of the reasons for the abstraction of a separate repository library from the rest of the filesystem code. The libsvn_repos API provides several other important utilities to Subversion. These include the abilities to:

As Subversion continues to evolve, the repository library will grow with the filesystem library to offer increased functionality and configurable option support.

If the Subversion Repository Layer is at "the other end of the line", the Repository Access Layer is the line itself. Charged with marshalling data between the client libraries and the repository, this layer includes the libsvn_ra module loader library, the RA modules themselves (which currently includes libsvn_ra_dav, libsvn_ra_local, and libsvn_ra_svn), and any additional libraries needed by one or more of those RA modules, such as the mod_dav_svn Apache module with which libsvn_ra_dav communicates or libsvn_ra_svn's server, svnserve.

Since Subversion uses URLs to identify its repository resources, the protocol portion of the URL schema (usually file:, http:, https:, or svn:) is used to determine which RA module will handle the communications. Each module registers a list of the protocols it knows how to "speak" so that the RA loader can, at runtime, determine which module to use for the task at hand. You can determine which RA modules are available to the Subversion command-line client, and what protocols they claim to support, by running svn --version:

$ svn --version
svn, version 0.25.0 (dev build)
   compiled Jul 18 2003, 16:25:59
 
Copyright (C) 2000-2003 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
 
The following repository access (RA) modules are available:
 
* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
  - handles 'http' schema
* ra_local : Module for accessing a repository on local disk.
  - handles 'file' schema
* ra_svn : Module for accessing a repository using the svn network protocol.
  - handles 'svn' schema

On the client side, the Subversion working copy is where all the action takes place. The bulk of functionality implemented by the client-side libraries exists for the sole purpose of managing working copies—directories full of files and other subdirectories which serve as a sort of local, editable "reflection" of one or more repository locations—and propagating changes to and from the Repository Access layer.

Subversion's working copy library, libsvn_wc, is directly responsible for managing the data in the working copies. To accomplish this, the library stores administrative information about each working copy directory within a special subdirectory. This subdirectory, named .svn is present in each working copy directory and contains various other files and directories which record state and provide a private workspace for administrative action. For those familiar with CVS, this .svn subdirectory is similar in purpose to the CVS administrative directories found in CVS working copies. For more information about the .svn administrative area, see Section , “Inside the Working Copy Administration Area”in this chapter.

The Subversion client library, libsvn_client, has the broadest responsibility; its job is to mingle the functionality of the working copy library with that of the Repository Access Layer, and then to provide the highest-level API to any application that wishes to perform general revision control actions. For example, the function svn_client_checkout takes a URL as an argument. It passes this URL to the RA layer and opens an authenticated session with a particular repository. It then asks the repository for a certain tree, and sends this tree into the working copy library, which then writes a full working copy to disk (.svn directories and all).

The client library is designed to be used by any application. While the Subversion source code includes a standard command-line client, it should be very easy to write any number of GUI clients on top of the client library. New GUIs (or any new client, really) for Subversion need not be clunky wrappers around the included command-line client—they have full access via the libsvn_client API to same functionality, data, and callback mechanisms that the command-line client uses.

Along with Subversion's own datatype, you will see many references to datatypes that begin with "apr_"—symbols from the Apache Portable Runtime (APR) library. APR is Apache's portability library, originally carved out of its server code as an attempt to separate the OS-specific bits from the OS-independent portions of the code. The result was a library that provides a generic API for performing operations that differ mildly—or wildly—from OS to OS. While Apache HTTP Server was obviously the first user of the APR library, the Subversion developers immediately recognized the value of using APR as well. This means that there are practically no OS-specific code portions in Subversion itself. Also, it means that the Subversion client compiles and runs anywhere that the server does. Currently this list includes all flavors of Unix, Win32, BeOS, OS/2, and Mac OS X.

In addition to providing consistent implementations of system calls that differ across operating systems, ^[27] APR gives Subversion immediate access to many custom datatypes, such as dynamic arrays and hash tables. Subversion uses these types extensively throughout the codebase. But perhaps the most pervasive APR datatype, found in nearly every Subversion API prototype, is the apr_pool_t—the APR memory pool. Subversion uses pools internally for all its memory allocation needs (unless an external library requires a different memory management schema for data passed through its API), ^[28] and while a person coding against the Subversion APIs is not required to do the same, they are required to provide pools to the API functions that need them. This means that users of the Subversion API must also link against APR, must call apr_initialize() to initialize the APR subsystem, and then must acquire a pool for use with Subversion API calls. See Section , “Programming with Memory Pools” for more information.

With remote version control operation as the whole point of Subversion's existence, it makes sense that some attention has been paid to internationalization (i18n) support. After all, while "remote" might mean "across the office", it could just as well mean "across the globe." To facilitate this, all of Subversion's public interfaces that accept path arguments expect those paths to be canonicalized, and encoded in UTF-8. This means, for example, that any new client binary that drives the libsvn_client interface needs to first convert paths from the locale-specific encoding to UTF-8 before passing those paths to the Subversion libraries, and then re-convert any resultant output paths from Subversion back into the locale's encoding before using those paths for non-Subversion purposes. Fortunately, Subversion provides a suite of functions (see subversion/include/svn_utf.h) that can be used by any program to do these conversions.

Also, Subversion APIs require all URL parameters to be properly URI-encoded. So, instead of passing file:///home/username/My File.txt as the URL of a file named My File.txt, you need to pass file:///home/username/My%20File.txt. Again, Subversion supplies helper functions that your application can use—svn_path_uri_encode and svn_path_uri_decode, for URI encoding and decoding, respectively.

If you are interested in using the Subversion libraries in conjunction with something other than a C program—say a Python script or Java application—Subversion has some initial support for this via the Simplified Wrapper and Interface Generator (SWIG). The SWIG bindings for Subversion are located in subversion/bindings/swig and are slowly maturing into a usable state. These bindings allow you to call Subversion API functions indirectly, using wrappers that translate the datatypes native to your scripting language into the datatypes needed by Subversion's C libraries.

There is an obvious benefit to accessing the Subversion APIs via a language binding—simplicity. Generally speaking, languages such as Python and Perl are much more flexible and easy to use than C or C++. The sort of high-level datatypes and context-driven type checking provided by these languages are often better at handling information that comes from users. As you know, only a human can botch up the input to a program as well as they do, and the scripting-type language simply handle that misinformation more gracefully. Of course, often that flexibility comes at the cost of performance. That is why using a tightly-optimized, C-based interface and library suite, combined with a powerful, flexible binding language is so appealing.

Let's look at an example that uses Subversion's Python SWIG bindings. Our example will do the same thing as our last example. Note the difference in size and complexity of the function this time!

from svn import fs
import os.path

def crawl_filesystem_dir (root, directory, pool):
  """Recursively crawl DIRECTORY under ROOT in the filesystem, and return
  a list of all the paths at or below DIRECTORY.  Use POOL for all 
  allocations."""

  # Get the directory entries for DIRECTORY.
  entries = fs.dir_entries(root, directory, pool)

  # Initialize our returned list with the directory path itself.
  paths = [directory]

  # Loop over the entries
  names = entries.keys()
  for name in names:
    # Calculate the entry's full path.
    full_path = os.path.join(basepath, name)

    # If the entry is a directory, recurse.  The recursion will return
    # a list with the entry and all its children, which we will add to
    # our running list of paths.
    if fs.is_dir(fsroot, full_path, pool):
      subpaths = crawl_filesystem_dir(root, full_path, pool)
      paths.extend(subpaths)

    # Else, it is a file, so add the entry's full path to the FILES list.
    else:
      paths.append(full_path)

  return paths

An implementation in C of the previous example would stretch on quite a bit longer. The same routine in C would need to pay close attention to memory usage, and need to use custom datatypes for representing the hash of entries and the list of paths. Python has hashes and lists (called "dictionaries" and "sequences", respectively) as built-in datatypes, and provides a wonderful selection of methods for operating on those types. And since Python uses reference counting and garbage collection, users of the language don't have to bother themselves with allocating and freeing memory.

In the previous section of this chapter, we mentioned the libsvn_client interface, and how it exists for the sole purpose of simplifying the process of writing a Subversion client. The following is a brief example of how that library can be accessed via the SWIG bindings. In just a few lines of Python, you can check out a fully functional Subversion working copy!

#!/usr/bin/env python
import sys
from svn import util, _util, _client

def usage():
  print "Usage: " + sys.argv[0] + " URL PATH\n"
  sys.exit(0)

def run(url, path):
  # Initialize APR and get a POOL.
  _util.apr_initialize()
  pool = util.svn_pool_create(None)

  # Checkout the HEAD of URL into PATH (silently)
  _client.svn_client_checkout(None, None, url, path, -1, 1, None, pool)

  # Cleanup our POOL, and shut down APR.
  util.svn_pool_destroy(pool)
  _util.apr_terminate()

if __name__ == '__main__':
  if len(sys.argv) != 3:
    usage()
  run(sys.argv[1], sys.argv[2])

Currently, it is Subversion's Python bindings that are the most complete. Some attention is also being given to the Java bindings. Once you have the SWIG interface files properly configured, generation of the specific wrappers for all the supported SWIG languages (which currently includes versions of Tcl, Python, Perl, Java, Ruby, Mzscheme, Guile, and PHP) should theoretically be trivial. Still, some extra programming is required to compensate for complex APIs that SWIG needs some help generalizing. For more information on SWIG itself, see the project's website at http://www.swig.org.

Perhaps the single most important file in the .svn directory is the entries file. The entries file is an XML document which contains the bulk of the administrative information about a versioned resource in a working copy directory. It is this one file which tracks the repository URLs, pristine revision, file checksums, pristine text and property timestamps, scheduling and conflict state information, last-known commit information (author, revision, timestamp), local copy history—practically everything that a Subversion client is interested in knowing about a versioned (or to-be-versioned) resource!

The following is an example of an actual entries file:

<?xml version="1.0" encoding="utf-8"?>
<wc-entries
   xmlns="svn:">
<entry
   committed-rev="1"
   name="svn:this_dir"
   committed-date="2002-09-24T17:12:44.064475Z"
   url="http://svn.red-bean.com/tests/.greek-repo/A/D"
   kind="dir"
   revision="1"/>
<entry
   committed-rev="1"
   name="gamma"
   text-time="2002-09-26T21:09:02.000000Z"
   committed-date="2002-09-24T17:12:44.064475Z"
   checksum="QSE4vWd9ZM0cMvr7/+YkXQ=="
   kind="file"
   prop-time="2002-09-26T21:09:02.000000Z"/>
<entry
   name="zeta"
   kind="file"
   schedule="add"
   revision="0"/>
<entry
   url="http://svn.red-bean.com/tests/.greek-repo/A/B/delta"
   name="delta"
   kind="file"
   schedule="add"
   revision="0"/>
<entry
   name="G"
   kind="dir"/>
<entry
   name="H"
   kind="dir"
   schedule="delete"/>
</wc-entries>

As you can see, the entries file is essentially a list of entries. Each entry tag represents one of three things: the working copy directory itself (noted by having its name attribute set to "svn:this-dir"), a file in that working copy directory (noted by having its kind attribute set to "file"), or a subdirectory in that working copy (kind here is set to "dir"). The files and subdirectories whose entries are stored in this file are either already under version control, or (as in the case of the file named zeta above) are scheduled to be added to version control when the user next commits this working copy directory's changes. Each entry has a unique name, and each entry has a node kind.

Developers should be aware of some special rules that Subversion uses when reading and writing its entries files. While each entry has a revision and URL associated with it, note that not every entry tag in the sample file has explicit revision or url attributes attached to it. Subversion allows entries to not explicitly store those two attributes when their values are the same as (in the revision case) or trivially calculable from ^[29] (in the url case) the data stored in the "svn:this-dir" entry. Note also that for subdirectory entries, Subversion stores only the crucial attributes—name, kind, url, revision, and schedule. In an effort to reduce duplicated information, Subversion dictates that the method for determining the full set of information about a subdirectory is to traverse down into that subdirectory, and read the "svn:this-dir" entry from its own .svn/entries file. However, a reference to the subdirectory is kept in its parent's entries file, with enough information to permit basic versioning operations in the event that the subdirectory itself is actually missing from disk.

As mentioned before, the .svn directory also holds the pristine "text-base" versions of files. Those can be found in .svn/text-base. The benefits of these pristine copies are multiple—network-free checks for local modifications and "diff" reporting, network-free reversion of modified or missing files, smaller transmission of changes to the server—but comes at the cost of having each versioned file stored at least twice on disk. These days, this seems to be a negligible penalty for most files. However, the situation gets uglier as the size of your versioned files grows. Some attention is being given to making the presence of the "text-base" an option. Ironically though, it is as your versioned files' sizes get larger that the existence of the "text-base" becomes more crucial—who wants to transmit a huge file across a network just because they want to commit a tiny change to it?

Similar in purpose to the "text-base" files are the property files and their pristine "prop-base" copies, located in .svn/props and .svn/prop-base respectively. Since directories can have properties, too, there are also .svn/dir-props and .svn/dir-prop-base files. Each of these property files ("working" and "base" versions) uses a simple "hash-on-disk" file format for storing the property names and values.

The first step in community participation is to find a way to stay on top of the latest happenings. To do this most effectively, you will want to subscribe to the main developer discussion list (<dev@subversion.tigris.org>) and commit mail list (<svn@subversion.tigris.org>). By following these lists even loosely, you will have access to important design discussions, be able to see actual changes to Subversion source code as they occur, and be able to witness peer reviews of those changes and proposed changes. These email based discussion lists are the primary communication media for Subversion development. See the Mailing Lists section of the website for other Subversion-related lists you might be interested in.

But how do you know what needs to be done? It is quite common for a programmer to have the greatest intentions of helping out with the development, yet be unable to find a good starting point. After all, not many folks come to the community having already decided on a particular itch they would like to scratch. But by watching the developer discussion lists, you might see mentions of existing bugs or feature requests fly by that particularly interest you. Also, a great place to look for outstanding, unclaimed tasks is the Issue Tracking database on the Subversion website. There you will find the current list of known bugs and feature requests. If you want to start with something small, look for issues marked as "bite-sized".

To edit the code, you need to have the code. This means you need to check out a working copy from the public Subversion source repository. As straightforward as that might sound, the task can be slightly tricky. Because Subversion's source code is versioned using Subversion itself, you actually need to "bootstrap" by getting a working Subversion client via some other method. The most common methods include downloading the latest binary distribution (if such is available for your platform), or downloading the latest source tarball and building your own Subversion client. If you build from source, make sure read the INSTALL file in the top level of the source tree for instructions.

After you have a working Subversion client, you are now poised to checkout a working copy of the Subversion source repository from http://svn.collab.net/repos/svn/trunk: ^[31]

$ svn checkout http://svn.collab.net/repos/svn/trunk subversion
A  HACKING
A  INSTALL
A  README
A  autogen.sh
A  build.conf
...

The above command will checkout the bleeding-edge, latest version of the Subversion source code into a subdirectory named subversion in your current working directory. Obviously, you can adjust that last argument as you see fit. Regardless of what you call the new working copy directory, though, after this operation completes, you will now have the Subversion source code. Of course, you will still need to fetch a few helper libraries (apr, apr-util, etc.)—see the INSTALL file in the top level of the working copy for details.

Now that you have a working copy containing the latest Subversion source code, you will most certainly want to take a cruise through the HACKING file in that working copy's top-level directory. The HACKING file contains general instructions for contributing to Subversion, including how to properly format your source code for consistency with the rest of the codebase, how to describe your proposed changes with an effective change log message, how to test your changes, and so on. Commit privileges on the Subversion source repository are earned—a government by meritocracy. ^[32] The HACKING file is an invaluable resource when it comes to making sure that your proposed changes earn the praises they deserve without being rejected on technicalities.

With the code and community policy understanding in hand, you are ready to make your changes. It is best to try to make smaller but related sets of changes, even tackling larger tasks in stages, instead of making huge, sweeping modifications. Your proposed changes will be easier to understand (and therefore easier to review) if you disturb the fewest lines of code possible to accomplish your task properly. After making each set of proposed changes, your Subversion tree should be in a state in which the software compiles with no warnings.

Subversion has a fairly thorough ^[33] regression test suite, and your proposed changes are expected to not cause any of those tests to fail. By running make check (in Unix) from the top of the source tree, you can sanity-check your changes. The fastest way to get your code contributions rejected (other than failing to supply a good log message) is to submit changes that cause failure in the test suite.

In the best-case scenario, you will have actually added appropriate tests to that test suite which verify that your proposed changes actually work as expected. In fact, sometimes the best contribution a person can make is solely the addition of new tests. You can write regression tests for functionality that currently works in Subversion as a way to protect against future changes that might trigger failure in those areas. Also, you can write new tests that demonstrate known failures. For this purpose, the Subversion test suite allows you to specify that a given test is expected to fail (called an XFAIL), and so long as Subversion fails in the way that was expected, a test result of XFAIL itself is considered a success. Ultimately, the better the test suite, the less time wasted on diagnosing potentially obscure regression bugs.

After making your modifications to the source code, compose a clear and concise log message to describe those changes and the reasons for them. Then, send an email to the developers list containing your log message and the output of svn diff (from the top of your Subversion working copy). If the community members consider your changes acceptable, someone who has commit privileges (permission to make new revisions in the Subversion source repository) will add your changes to the public source code tree. Recall that permission to directly commit changes to the repository is granted on merit—if you demonstrate comprehension of Subversion, programming competency, and a "team spirit", you will likely be awarded that permission.

While Subversion has different switches for its subcommands, all switches are global—that is, each switch is guaranteed to mean the same thing regardless of the subcommand you use it with. For example, --verbose (-v) always means “verbose output”, regardless of the subcommand you use it with.

Switches in svnlook are global, just like in svn and svnadmin, however, most switches only apply to one subcommand since the functionality of svnlook is (intentionally) limited in scope.