Comprehensive source backup

Sun, 14 December, 2008

I've wanted a way to back up all my projects to offsite storage in the laziest way possible. Here's the function in, you guessed it, my .bashrc:

 1 # Back up all repos off site
 2 function bak {
 3         # mount my project partition
 4         m1
 5
 6         # Find and iterate all git repos
 7         sudo find /* -type d -name ".git" |
 8         while read repo; do
 9
10                 # Ignore these repos
11                 [ $repo == "/home/erik/tests/ffExtOne/.git" ] && continue
12                 [ $repo == "/home/erik/tests/ffExtTwo/ext/.git" ] && continue
13                 [ $repo == "/home/erik/tests/glibOne/.git" ] && continue
14                 [ $repo == "/home/erik/djangosites/yystclair/.git" ] && continue
15                 [ $repo == "/mnt/hda1/devel/projects/gnc/repo/.git" ] && continue
16                 [ $repo == "/mnt/hda1/devel/gitSandbox/.git" ] && continue
17                 [ $repo == "/mnt/hda1/devel/gitSandbox/ignoreTest/.git" ] && continue
18
19                 # Send repo to remote server
20                 sudo rsync --verbose --stats --compress --relative --progress --recursive --times --delete $repo rsync://xxx.net/MyRsyncHomeForSource
21         done
22 }

It works by scanning the entire filesystem for .git directories, and rsync'ing them to my offsite location. By using the --relative flag, the full paths are copied under a single remote rsync location, instead of just .git (which would collide with prior .git's). The full path is enough to remind me what's in each repository.

When I create a new repo for a project, I don't have to take a single extra step to make sure it's part of the source backup. I can optionally specify repos to ignore, but the function errs on the side of inclusion.

It took about 20 minutes to run the first time. Since subsequent backups will be diffs, it will be faster in the future. Using locate instead of find could speed up the process, as long as I was sure to updatedb prior to pushing to make sure that any brand-new repos were included.

About Me

Erik Mackdanz is a software developer in Austin, Texas, along with everybody else.

Links