Auto-builds and tests all the branches of your git projects, showing pass/fail results on a web page/RSS feed. Isolates failures to the first commit that caused the problem.
Welcome to gitbuilder, an autobuilder tool for your favourite git-managed development project.
gitbuilder retrieves the latest version of your project from its git repository, then builds the most recent versions of all the tags and all the branches in the entire repository. If any of them have build failures, it automatically does a "git bisect" style operation to try to track down the most recent version that compiled successfully. With that information, you should be able to identify exactly which commit caused the problem, as well as exactly who was responsible.
gitbuilder's results are available via the web and via an RSS feed. To see what the results look like, visit the gitbuilder site for the Versaplex project:
or its RSS feed:
Installing gitbuilder is supposed to be easy. It should run on any system with perl, bash, and git installed. Follow these steps to get started:
Get a copy of the latest gitbuilder repository. (If you're reading this, maybe you already did this step!):
git clone git://github.com/apenwarr/gitbuilder.git
Go into the gitbuilder directory and clone a copy of the application to test. For your convenience, we made a handy test project that you can use to try out gitbuilder's features, which you can clone with a command like the following:
cd gitbuilder git clone git://github.com/apenwarr/builder-test.git build
!!NOTE!! See the third word on the second line? You have to clone your project into a directory called 'build', not the normal directory git would assign it by default. Otherwise gitbuilder wouldn't know how to build it!
Make a build.sh script with the instructions required to build your project. It would sure be nice if everyone's project could just be built by checking it out and typing "make", right? But we know that's not the case, because there are often things like autoconf involved. Luckily, we provided a handy sample build.sh script to go with the handy sample project:
cp build.sh.example build.sh
To see gitbuilder's output, you'll need a web server configured to run .cgi scripts properly. Configuring your web server is beyond the scope of this document, but if you already have such a server, maybe something like this will work for you:
ln -s $PWD/out ~/public_html/gitbuilder
If it worked, you should now be able to go to this page in your web browser:
(where YOURUSERNAMEGOES_HERE is replaced with your actual username, of course.)
Okay, we're ready to go! Start the autobuilder process!
Now you can go back to your web browser and reload the page repeatedly. If you're using the sample builder-test project, you should see gitbuilder do some simple bisection across a few branches as it tries to track down the broken and non-broken versions automatically.
Now, maybe the computer you use for autobuilding is wide open on the Internet, so your internal web server can be shared with everyone. In that case, congratulations! You're done. Just send the URL to all your friends.
But if you like to have a little bit of security between your development network and the Internet, you'll need to do one more step. You can rsync the gitbuilder output files to your favourite public Internet server. (Bonus: the public server doesn't need to have git or a compiler installed!) We provided a handy script to do this for you automatically:
Now that your gitbuilder is working, you probably want to have it continue to building new versions automatically. gitbuilder is designed to make it safe to do that. Try adding a line like this to your crontab (using crontab -e):
* * * * * nice ~/src/gitbuilder/start >/dev/null 2>&1
This restarts the autobuilder every minute, in case it had stopped in the meantime. If it was already running, the "start" script is smart enough not to start it again. Of course, you can change the cron settings to run less often if that's what you want.
If you're using recent Linux 2.6.x kernels and you want to eliminate the effect on your CPU and disk of running gitbuilder in the background, you should try out ionice. You can change your cron command to something like this:
* * * * * ionice -c3 nice -20 ~/src/gitbuilder/start >/dev/null 2>&1
ionice is extremely powerful; if your system supports it the above command will make it so that your gitbuilder will only use the disk if nobody else is using it right now. On my system, this makes gitbuilder completely unnoticeable, even though it spends most of its time building in the background. Cool!
Note: in the above, we redirected the output to /dev/null because otherwise you'd get an email every minute. But redirecting your error output isn't a very safe thing to do; how will you know if something actually goes wrong?
If you want a better way to monitor your cron jobs, try out my cron2rss project:
gitbuilder is a collection of several tiny scripts that do simple things. You might want to know about these scripts in case you want to customize how gitbuilder works. Here we go:
start: just basically runs this command: ./runlock lock ./autobuilder.sh
stop: stop the autobuilder, basically by running: kill $(cat lock.lock)
runlock: runs a program only if the given lockfile isn't locked. This is how we ensure that the autobuilder script doesn't run more than once simultaneously.
lock.lock: the file created by runlock if the first parameter is 'lock'.
autobuilder.sh: fetches your build/ directory, chooses branches with branches.sh, chooses revisions to build on each branch with next-rev.sh, and then runs run-build.sh on each selected revision.
run-build.sh: checks out a given revision from git in the build/ directory, then runs build.sh.
build.sh: a script provided by you that builds your application. See build.sh.example.
branches.sh: get a list of interesting branches in the build/ directory.
revlist.sh: given a branch name, gets a list of all the revisions starting from that branch's HEAD and stopping at the first revision that has been built successfully in the past. Basically, this lists all the recent revisions on a branch that haven't been built successfully yet.
next-rev.sh: given a branch name, picks out exactly one revision from revlist.sh's output that will most help narrow down where any problems might have come from.
rsync-to: a simple script to rsync the out/ directory to a web server somewhere so you can show it to the public.
out/index.cgi: the main script for generating an HTML view of your autobuilder.
out/rss.cgi: the main script for generating an RSS view of your autobuilder.
out/log.cgi: a simple script for highlighting and viewing build log output.
gitbuilder's output data format is very simple. It consists of files in the out/pass/, out/fail/, and out/ignore/ directories.
out/pass/: files in here are named after the SHA-1 of the git commit that we tried to build. If a particular commit shows up here, it means that we've successfully build that version in the past, so we don't need to try building it anymore. The content of each file is the stdout/stderr of build.sh on the successful run.
out/fail/: just like out/pass/, only for failed builds.
out/ignore/: this directory is never written to by gitbuilder, but you can create files here yourself (using the 'touch' command), named after the SHA-1 of commits you want to ignore. For example, if you have a tag that you're tired of seeing in the autobuilder output, you can get its SHA-1 (available by running ./branches.sh | grep BRANCHNAME) and create a file here.
Or maybe you know that your project simply never builds successfully before a particular revision. For example, maybe build.sh tries to do "make test", but your project didn't have "make test" before last week, so all the builds before that will surely fail. In that case, you can pick the version before you added "make test" and reference it in the out/ignore/ directory. gitbuilder will then happily ignore all the versions before that one when it tries to track down problems.
HINT: Did you screw up your build.sh and end up with a lot of incorrect pass/fail indicators? It's safe to just "rm out/pass/* out/fail/*" and let gitbuilder start over again.
Good luck, and let me know how it works for you!
-- Avery [email protected]