These are the steps:

$ mkdir -p ~/opt/python-2.7
$ mkdir ~/src
$ cd ~/src
$ curl -LO http://www.python.org/ftp/python/2.7/Python-2.7.tgz
$ tar -xzvf Python-2.7.tgz
$ cd Python-2.7
$ ./configure --prefix=$HOME/opt/python2.7
$ make
$ make install

Next, create ~/opt/bin if it doesn’t exist already and symlink the new Python binary:

$ mkdir -p ~/opt/bin
$ ln -s ~/opt/python-2.7/bin/python2.7 ~/opt/bin/python

Finally, be sure your $SHELL’s $PATH is configured to look for binaries in ~/opt/bin. I use Bash, so I added the following line to ~/.bashrc:

export PATH=$HOME/opt/bin:$PATH

Ok, so why bother?

I’m lazy. I wanted to save myself a bunch of calls to file.close() by using the more modern with open syntax. Most shared hosts seem to consider Python unimportant, the two I use have Python 2.4.3 installed which is six years old. I couldn’t even import __future__ because __future__ hadn’t been written yet.

Waylan Limberg’s post about installing multiple versions of Python was helpful.

Django just isn’t designed to run under CGI.
It won’t run under OS/2, either.*

Well ok, but running Django under CGI is not impossible. It just kind of really sucks. But anyway, to prove it’s possible if not workable, here’s how I got it running on two standard cPanel shared hosts using plain old slow and clunky CGI.

virtualenv

First, install virtualenv. This makes locally managing modules fantastically easy by creating self-contained Python virtual environments. Installing couldn’t be simpler: Get the script, run the script, source your environment.

$ mkdir ~/src && cd ~/src
$ curl -LO http://bitbucket.org/ianb/virtualenv/get/tip.gz
$ tar -xvzf tip.gz
$ python virtualenv/virtualenv.py --distribute ~/python_virtualenv
New python executable in /home/joe/python_virtualenv/bin/python
Installing distribute.............................................
..................................................done.

$ source ~/python_virtualenv/bin/activate 

Now, install Django using pip, which was automatically installed by virtualenv. After sourcing the virtual environment, this works from anywhere.

$ pip install Django
Downloading/unpacking Django
  Downloading Django-1.1.1.tar.gz (5.6Mb): 5.6Mb downloaded
  Running setup.py egg_info for package Django
Installing collected packages: Django
  Running setup.py install for Django
    changing mode of build/scripts-2.4/django-admin.py from 664 to 775
    changing mode of /home/joe/python_virtualenv/bin/django-admin.py to 775
Successfully installed Django

If your host doesn’t block GCC, use pip to be sure your MySQL interface (MySQLdb) is up to date:

$ pip install -U MySQL-python
...
Successfully installed MySQL-python

Django requires MySQLdb version 1.2.1p2 or higher.

Yolk prints a nice, clean list of everything installed in your Python environment, install and run:

$ pip install yolk
$ yolk -l

Django          - 1.1.1        - active
MySQL-python    - 1.2.3c1      - active
pip             - 0.6.1        - active
setuptools      - 0.6c11       - active
yolk            - 0.4.1        - active

At this point, I started a new Django project, assigned a database and filled in the necessary values in settings.py. I put the Django project files into the virtual environment to keep everything in the same place. This might not be the best practice, but it makes sense to me.

$ cd ~/python_virtualenv/
$ django-admin.py startproject testproject

The sane part is finished, now onto the kludgery.

Django.cgi

All the CGI shim solutions I found pointed back to a script Paul Sargent uploaded to ticket 2407 back in summer of 2006. It still works: django.cgi

Three lines need editing:

Line 1: Point the CGI’s shebang to the virtualenv Python binary.

#!/home/joe/python_virtualenv/bin/python

Line 95: Add the directory above the Django project directory to Python’s sys.path.

sys.path.append("/home/joe/python_virtualenv")

Line 97: Add the project’s settings to os.environ.

os.environ['DJANGO_SETTINGS_MODULE'] = 'testproject.settings'

htaccess

For Django to respond to URL requests, those urls need to be fed into the django.cgi script. For testing I routed everything from /django to the cgi script by adding the following lines to my top-level htaccess file:

RewriteEngine on
RewriteRule ^cgi-bin/ - [L]
RewriteRule ^django/(.*)$ /cgi-bin/django.cgi/$1 [QSA,L]

The second line isn’t necessary unless pulling Django urls from the webroot, without it, the redirects would loop.

At this point, the Django site should load from /django/… urls.

Finally, as a quick fix for admin media files, I symlinked Django’s admin media directory from my web root:

ln -s ~/python_virtualenv/lib/python2.4/site-packages/django/contrib/admin/media ~/www/media

Conclusion

I spent quite a few hours spread across a couple days researching and figuring out how to get the first install working. The second installation only took about 5 minutes from start until editing Django’s admin pages.

Running Django through CGI is possible, but it is dog slow. There appears to be some caching after the first request, but that first page load often takes an excruciatingly long time.

Further reading, possible improvements

• 21.4. wsgiref — WSGI Utilities and Reference Implementation — Python v2.6.4 documentation
• Django snippets: django under apache / mod_fcgid

The servers I was working with are both running the almost six year old Python 2.4.3. The wsigref module was introduced with Python 2.5. My goal was to get Django running without compiling anything since some hosts deny access to GCC.

References

These sites were helpful in figuring this out.

• Notes on using pip and virtualenv with Django « SaltyCrane Blog
• JeffCroft.com: Setting up Django on Dreamhost
• seamusc.com – How to get Django working on digiweb.ie using django.cgi
• Lithium Hosting – Knowledgebase – How to install Django on a basic shared hosting plan with plain CGI
• Django revisited – PeterSmith: Blog

The two hosts I tested on were LiquidWeb and A2Hosting. Both have been excellent, dependable hosts. Neither has any Python support to speak of on their shared plans. A2 blocks access to GCC.

But LiquidWeb doesn’t offer Subversion. And I will no longer do web work without it.

For some time I’d been considering leaving LiquidWeb because the lack of svn was now hindering work on my own sites. For the same reason, I’ve had to pass them over several times when clients asked for hosting recommendations. Then the other night, I stumbled across a discussion about installing Subversion on a shared host. Why didn’t I try that years ago?

Installation Instructions

These instructions assume basic proficiency with the Unix command line. Note that the goal is to install the SVN client, plan on hosting your repositories somewhere else.

Connect to your account with ssh and create a working directory, mine’s called _src:

cd
mkdir _src
cd _src

Next, use wget to pull down the subversion sources. You could also use curl, but wget does the same with less typing. Choose a version from this list of Subversion sources. The parallel “subversion-deps” download includes all dependent sources required to build Subversion.

wget http://subversion.tigris.org/downloads/subversion-1.4.6.tar.gz
wget http://subversion.tigris.org/downloads/subversion-deps-1.4.6.tar.gz

Next, untar the sources and dive into the directory:

tar -xzvf subversion-1.4.6.tar.gz
tar -xzvf subversion-deps-1.4.6.tar.gz
cd subversion-1.4.6

One step, maybe (32-bit?)

At this point, depending on your server configuration, you might be able to install with the following two commands:

./configure --prefix=$HOME --without-berkeley-db \
     --with-ssl --with-editor=/usr/bin/vim \
     --without-apxs --without-apache
make && make install

That worked on some servers but not others, I spent many hours clumsily trying to figure out why. I found that the build process was linking to external libraries instead of the ones in subversion-deps, despite the configure directives.

On the 64-bit server (x86_64) this was a big problem, even though everything worked normally on two 32-bit (i686/i386) servers. Check server architecture with either arch, uname -a or cat /proc/cpuinfo. The solution for 64-bit machines is to build the three main components beforehand and then tell Subversion where to find them.

Build them, use them (64-bit?)

The following commands systematically build each of the three shared libraries from the Subversion-deps sources. This should guarantee that the files Subversion links to will be compatible. In each step, we explicitly enable compilation of shared libraries and prefix the files into our home directory. This looks worse than it is.

Build apr, apr-util and neon, in that order:

cd apr
./configure --enable-shared --prefix=$HOME
make && make install

cd ../apr-util
./configure --enable-shared --prefix=$HOME \
     --with-expat=builtin --with-apr=$HOME \
     --without-berkeley-db
make && make install

cd ../neon
./configure --enable-shared --prefix=$HOME \
     --with-libs=$HOME --with-ssl
make && make install

With those out of the way, we can now configure and build Subversion. According to the notes in Subversion’s configure file, the --with-ssl flag is not necessary since it’s just passed to neon and we already built neon. --without-apxs and --without-apache prevent Subversion from trying to install its Apache modules. Remember to point to the libraries we just built (in $HOME): (note: you may need to add the --without-serf flag as well, see this comment)

cd ../
./configure --prefix=$HOME --without-berkeley-db \
     --with-editor=/usr/bin/vim --with-apr=$HOME \
     --with-apr-util=$HOME --with-neon=$HOME \
     --without-apxs --without-apache
make && make install

Subversion should now be installed! It’s likely you already have :~/bin in your user $PATH, if so, you can try it out right away:

svn --version

   svn, version 1.4.6 (r28521)
      compiled Jan 29 2008, 11:05:47

So far, I’ve run this against two LiquidWeb shared accounts, one LiquidWeb CPanel container on a VPS and a HostingMatters shared account running JailShell. After some limited testing accessing different repositories with https and svn+ssh, everything seems to be working.

Now I just need to figure out where to put my repositories, here are some free Subversion hosts I’ve used or are considering:

• Beanstalk 20 MB free
• Assembla 500 MB free?! really?
• Unfuddle 15 MB free
• Google Code 100 MB free, must be open source

Two other sites which aren’t free, but I’m keeping in mind for the future: Wush and Project Locker.

The getting there was the hardest part

While I was finally able to streamline this down to a fairly simple process, it was not easy to get there. I don’t do much compiling from source, so I’m very clumsy about troubleshooting. If any of these steps can be simplified, please leave a note.

Here are some of the errors I saw along the way:
/usr/lib/libexpat.so: could not read symbols: File in wrong format
/usr/lib/libexpat.so: could not read symbols: Invalid operation
/home/joe/_src/subversion-1.4.6/neon/src/.libs/libneon.a: could not read symbols: Bad value 

The first two errors indicate that the build had grabbed the 32-bit libexpat from /usr/lib instead of the 64-bit version from /usr/lib64. However, redirecting to the 64-bit libraries introduced other problems such as the libneon.a bad value error. As mentioned above, what I needed to do was pull libraries from the subverson-deps code. To use those, I needed to compile each one first.

Resources:

• Subversion’s INSTALL file
• ./configure -help
• These commands, from this original post almost worked.
• Installing Mephisto and Subversion on a Shared Host, first inkling this was possible
• similar problem with with /usr/lib instead of /usr/lib64.
• GATTAGA’s post was the first clue about working around the libexpat problem.
• SVNforum: libneon.a: could not read symbols: Bad value This was the post that gave me the final bit of information I needed

Related: How to install Git on a shared host

Update, November 2010: Sometime this year Liquid Web disallowed compiler access. I now host my sites at WebFaction and A2 Hosting.