But LiquidWeb doesn’t offer Subversion. And I will no longer do web work without it.

For some time I’d been considering leaving LiquidWeb because the lack of svn was now hindering work on my own sites. For the same reason, I’ve had to pass them over several times when clients asked for hosting recommendations. Then the other night, I stumbled across a discussion about installing Subversion on a shared host. Why didn’t I try that years ago?

Installation Instructions

These instructions assume basic proficiency with the Unix command line. Note that the goal is to install the SVN client, plan on hosting your repositories somewhere else.

Connect to your account with ssh and create a working directory, mine’s called _src:

cd
mkdir _src
cd _src

Next, use wget to pull down the subversion sources. You could also use curl, but wget does the same with less typing. Choose a version from this list of Subversion sources. The parallel “subversion-deps” download includes all dependent sources required to build Subversion.

wget http://subversion.tigris.org/downloads/subversion-1.4.6.tar.gz
wget http://subversion.tigris.org/downloads/subversion-deps-1.4.6.tar.gz

Next, untar the sources and dive into the directory:

tar -xzvf subversion-1.4.6.tar.gz
tar -xzvf subversion-deps-1.4.6.tar.gz
cd subversion-1.4.6

One step, maybe (32-bit?)

At this point, depending on your server configuration, you might be able to install with the following two commands:

./configure --prefix=$HOME --without-berkeley-db \
     --with-ssl --with-editor=/usr/bin/vim \
     --without-apxs --without-apache
make && make install

That worked on some servers but not others, I spent many hours clumsily trying to figure out why. I found that the build process was linking to external libraries instead of the ones in subversion-deps, despite the configure directives.

On the 64-bit server (x86_64) this was a big problem, even though everything worked normally on two 32-bit (i686/i386) servers. Check server architecture with either arch, uname -a or cat /proc/cpuinfo. The solution for 64-bit machines is to build the three main components beforehand and then tell Subversion where to find them.

Build them, use them (64-bit?)

The following commands systematically build each of the three shared libraries from the Subversion-deps sources. This should guarantee that the files Subversion links to will be compatible. In each step, we explicitly enable compilation of shared libraries and prefix the files into our home directory. This looks worse than it is.

Build apr, apr-util and neon, in that order:

cd apr
./configure --enable-shared --prefix=$HOME
make && make install

cd ../apr-util
./configure --enable-shared --prefix=$HOME \
     --with-expat=builtin --with-apr=$HOME \
     --without-berkeley-db
make && make install

cd ../neon
./configure --enable-shared --prefix=$HOME \
     --with-libs=$HOME --with-ssl
make && make install

With those out of the way, we can now configure and build Subversion. According to the notes in Subversion’s configure file, the --with-ssl flag is not necessary since it’s just passed to neon and we already built neon. --without-apxs and --without-apache prevent Subversion from trying to install its Apache modules. Remember to point to the libraries we just built (in $HOME): (note: you may need to add the --without-serf flag as well, see this comment)

cd ../
./configure --prefix=$HOME --without-berkeley-db \
     --with-editor=/usr/bin/vim --with-apr=$HOME \
     --with-apr-util=$HOME --with-neon=$HOME \
     --without-apxs --without-apache
make && make install

Subversion should now be installed! It’s likely you already have :~/bin in your user $PATH, if so, you can try it out right away:

svn --version

   svn, version 1.4.6 (r28521)
      compiled Jan 29 2008, 11:05:47

So far, I’ve run this against two LiquidWeb shared accounts, one LiquidWeb CPanel container on a VPS and a HostingMatters shared account running JailShell. After some limited testing accessing different repositories with https and svn+ssh, everything seems to be working.

Now I just need to figure out where to put my repositories, here are some free Subversion hosts I’ve used or are considering:

• Beanstalk 20 MB free
• Assembla 500 MB free?! really?
• Unfuddle 15 MB free
• Google Code 100 MB free, must be open source

Two other sites which aren’t free, but I’m keeping in mind for the future: Wush and Project Locker.

The getting there was the hardest part

While I was finally able to streamline this down to a fairly simple process, it was not easy to get there. I don’t do much compiling from source, so I’m very clumsy about troubleshooting. If any of these steps can be simplified, please leave a note.

Here are some of the errors I saw along the way:
/usr/lib/libexpat.so: could not read symbols: File in wrong format
/usr/lib/libexpat.so: could not read symbols: Invalid operation
/home/joe/_src/subversion-1.4.6/neon/src/.libs/libneon.a: could not read symbols: Bad value 

The first two errors indicate that the build had grabbed the 32-bit libexpat from /usr/lib instead of the 64-bit version from /usr/lib64. However, redirecting to the 64-bit libraries introduced other problems such as the libneon.a bad value error. As mentioned above, what I needed to do was pull libraries from the subverson-deps code. To use those, I needed to compile each one first.

Resources:

• Subversion’s INSTALL file
• ./configure -help
• These commands, from this original post almost worked.
• Installing Mephisto and Subversion on a Shared Host, first inkling this was possible
• similar problem with with /usr/lib instead of /usr/lib64.
• GATTAGA’s post was the first clue about working around the libexpat problem.
• SVNforum: libneon.a: could not read symbols: Bad value This was the post that gave me the final bit of information I needed

Related: How to install Git on a shared host

Update, November 2010: Sometime this year Liquid Web disallowed compiler access. I now host my sites at WebFaction and A2 Hosting.

wget
The server I was pulling from did not have rsync installed and would not let me connect via SSH (secure shell). I could telnet, but that didn’t help me. Since rsync was out of the question, I found wget, which is often used to mirror sites via ftp. One thing I couldn’t get to work was to copy directly from the old server to the new one, so I decided to download the whole site to my hard drive and then sync it up to the new server.

I don’t have the developer tools installed yet, but thankfully Apple has a pre-compiled package available from the OS X web site: Wget 1.8.1

Wget is very easy to use. The only stumbling block I had was the need to point to my www directory explicitly, wget wouldn’t follow the symlink (before last night, I didn’t know what a symlink was. They’re basically aliases). I found the explicit path by getting info on a file from the server using Fetch. Once I had the path correct, wget worked perfectly with the following command:

wget -m --passive-ftp ftp://[user]:[password]@[host][explicit path to root directory]/

The commands at the beginning tell wget to mirror (m) and to use passive FTP (–passive-ftp).

rsync

I first learned about rsync while looking for an open source (free) disk mirroring solution for a file server. At the time, rsync didn’t support OS X’s HFS+ filesystem so icons and creator codes weren’t duplicated. Since then macosxlabs.org has developed RsyncX which I hope to try out soon.

I used rsync for two different things. First, I wanted to back up the site I just downloaded by burning it to a CD. OS X doesn’t seem to be able to create a disk image from a folder lke OS 9 could do, and the one shareware application which claimed that ability kept returning errors. I ended up creating a blank CD-master disk image with Disk Copy, then using rsync to duplicate the downloaded folder to the disk image. This is the command I used:

rsync -vr [source path, w/o trailing slash] [disk image path]

The -vr command tells rsync to be “verbose” (v) while duplicating and to recursively copy all directories (r).

The other use for rsync was to mirror my local site onto the new remote server.

rsync -vtrp -e ssh [local path]/ [user]@host:[remote directory]/

The additional commands tell rsync to preserve the original file times (t) and permissions (p).

A trailing slash on the source path tells rsync copy the files from that directory into the target directory. If there is no trailing slash, the directory itself will be copied and created if necessary.

The following examples use a fictional file system which contains:

sourceDir/

      a.file
      b.file

destinationDir/

      1.file

      2.file

These two simplified examples demostrate the effect of the trailing slash:

rsync /sourceDir /destinationDir/

would copy the directory sourceDir into destinationDir resulting in:



destinationDir/

      1.file

      2.file

      sourceDir/

            a.file

            b.file

rsync /sourceDir/ /destinationDir/

would sync the contents of sourceDir into destinationDir resulting in:



destinationDir/

      1.file
      2.file
      a.file
      b.file

My last stumbling block was specifying the target directory correctly. All the examples I could find had the target starting with a slash, but when I tried that, it bounced up to the root of the server (not my local root folder) because I’m uploading to a directory before switching my domain over. Once I realized that the “mkdir…permission denied (1)” error was a result of trying to create a directory outside of my personal space rsync worked perfectly. After several hours of searching for answers of course.

I didn’t find any one resource which answered all my questions, but the following sites are good places to start. Otherwise, Google is your friend.

• The man pages at rsync. samba.org are a lot easier to read than the ones in the terminal.
• Rsync tutorial