Why I avoid using Subversion externals
I’ve been using Subversion for a few years now and one of the tricks I’ve picked up on is avoiding the use of externals like the plague. I first stumbled upon this idea when including a large code base as an external in one of my own projects.
Subversion operations on the project in question were lightning fast. I could make a code change, and the commit operation would take only the blink of an eye. Likewise, updating other working copies with my changes was similarly instantaneous.
Then I included a large dependency as a Subversion external. From that point on, Subversion operations were very sluggish and it soon became a chore to update my working copies.
It quickly occurred to me that it was the externals definition that was the cause of the slowdown. Essentially, including another working copy as an external causes a huge amount of excess work for Subversion each time you perform an operation, particularly updates.
The problem
When you run an update, Subversion will contact the server hosting the external repository, asking for any updates, downloading them, and applying them as necessary. If you watch your Subversion client when you do this, it takes a good few seconds for Subversion to make the initial connection even when there are no changes. In addition to this, if you have a large number of directories and files in your external, the time it takes to traverse these can also have a sizeable impact on Subversion’s responsiveness.
So what are the benefits to externals?
Well, for one thing, if a third party wants to include your code in their own project, they ensure that all the dependencies are automatically satisfied upon checkout. But if you’re not writing a framework or some other kind of software library intended to be included within another system, then this is not such an important point. In fact if you’re like me and usually make one-time projects, the only time you would even use externals is during development, since it’s generally considered best practice not to use Subversion in a production environment at all.
The solution
Here is what I do in my projects to keep Subversion responsive and fast. I would typically have a directory for my dependencies; let’s use the symfony convention and call it lib/vendor:
lib/
vendor/
software_library_1/
software_library_2/
In a typical Subversion externals situation you might set the following for the svn:externals property of the vendor directory:
software_library_2 http://svn.software.example.org/library_2/trunk
Instead of doing the above, I would simply set the svn:ignore property of the vendor directory by running the following command (from the root of the project):
…and then run the following commands from the root of the project to create the individual working copies for each dependency:
svn co http://svn.software.example.org/library_2/trunk lib/vendor/software_library_2
The end result is largely the same, except that Subversion will completely ignore your dependencies when performing an update, resulting in a drastically reduced amount of time needed to perform the operation.
Of course there is a slight disadvantage here, in that you must manually update all your dependencies when required. Realistically, this is not a problem most of the time unless you are including a software library that undergoes rapid changes. With that said, I usually write a small script to handle the work for me, like so (example in PHP):
update_repos.php:
set_time_limit(0);
define('SVN_CMD', 'svn');
require_once(dirname(__FILE__).'/repos.php');
$return_var = null;
foreach ($repositories as $repository)
{
list($path, $url) = $repository;
$path = dirname(__FILE__).'/'.$path;
if (!file_exists($path)) mkdir($path);
if (!is_dir($path)) throw new Exception('creation of WC directory failed');
if (!dir_is_empty($path))
{
$cmds = array(
SVN_CMD.' revert -R '.escapeshellarg($path),
SVN_CMD.' up '.escapeshellarg($path),
);
}
else
{
$cmds = array(
SVN_CMD.' co '.escapeshellarg($url).' '.escapeshellarg($path),
);
}
foreach ($cmds as $cmd)
{
passthru($cmd, $return_var);
if ($return_var) throw new Exception('execution of SVN command failed');
}
}
/**
* @return boolean
*/
function dir_is_empty($path)
{
foreach (scandir($path) as $item)
{
if ($item != '.' && $item != '..') return false;
}
return true;
}
repos.php:
Place these two files in the root of your project, configure repos.php and you’re all set. You’ll never need to use an externals definition again! Well, maybe… 
5 Comments »
RSS feed for comments on this post. TrackBack URL
[...] Czytaj więcej: Ezzatron » Why I avoid using Subversion externals [...]
Interesting points – another weakness of externals comes up when you are integrating a third party’s development branch (as opposed to a tagged release) as you could inadvertently run an update and get an incompatible development release.
Externals is one of those useful tools that works great within an organisation (say you have one team working on a base/common framework, and other teams working on projects dependant on that) – since the round trip of checking for updates on the external(s) is likely not to be noticed. But it then gets overused.
I wouldn’t mind seeing something in-between. Like an option on update to exclude externals (maybe even set true as default).
Actually, there is an option: –ignore-externals, but I agree that it should probably be on by default.
Also, you can specify a particular revision in your external definition, something I’ve not tried before, and this may help in situations where there are no useful branches or tags available.
Ah, so there is – I must have missed that when I looked at the manual yesterday
I use version in tags in external definition, and make tags directory readonly to prevent accidently changing the external project.