Joe Maller.com

Django via CGI on shared hosting

Django just isn’t designed to run under CGI.
It won’t run under OS/2, either.*

Well ok, but running Django under CGI is not impossible. It just kind of really sucks. But anyway, to prove it’s possible if not workable, here’s how I got it running on two standard cPanel shared hosts using plain old slow and clunky CGI.

virtualenv

First, install virtualenv. This makes locally managing modules fantastically easy by creating self-contained Python virtual environments. Installing couldn’t be simpler: Get the script, run the script, source your environment.

$ mkdir ~/src && cd ~/src
$ curl -LO http://bitbucket.org/ianb/virtualenv/get/tip.gz
$ tar -xvzf tip.gz
$ python virtualenv/virtualenv.py --distribute ~/python_virtualenv
New python executable in /home/joe/python_virtualenv/bin/python
Installing distribute.............................................
..................................................done.

$ source ~/python_virtualenv/bin/activate 

Now, install Django using pip, which was automatically installed by virtualenv. After sourcing the virtual environment, this works from anywhere.

$ pip install Django
Downloading/unpacking Django
  Downloading Django-1.1.1.tar.gz (5.6Mb): 5.6Mb downloaded
  Running setup.py egg_info for package Django
Installing collected packages: Django
  Running setup.py install for Django
    changing mode of build/scripts-2.4/django-admin.py from 664 to 775
    changing mode of /home/joe/python_virtualenv/bin/django-admin.py to 775
Successfully installed Django

If your host doesn’t block GCC, use pip to be sure your MySQL interface (MySQLdb) is up to date:

$ pip install -U MySQL-python
...
Successfully installed MySQL-python

Django requires MySQLdb version 1.2.1p2 or higher.

Yolk prints a nice, clean list of everything installed in your Python environment, install and run:

$ pip install yolk
$ yolk -l

Django          - 1.1.1        - active 
MySQL-python    - 1.2.3c1      - active 
pip             - 0.6.1        - active 
setuptools      - 0.6c11       - active 
yolk            - 0.4.1        - active 

At this point, I started a new Django project, assigned a database and filled in the necessary values in settings.py. I put the Django project files into the virtual environment to keep everything in the same place. This might not be the best practice, but it makes sense to me.

$ cd ~/python_virtualenv/
$ django-admin.py startproject testproject

The sane part is finished, now onto the kludgery.

Django.cgi

All the CGI shim solutions I found pointed back to a script Paul Sargent uploaded to ticket 2407 back in summer of 2006. It still works: django.cgi

Three lines need editing:

Line 1: Point the CGI’s shebang to the virtualenv Python binary.

#!/home/joe/python_virtualenv/bin/python

Line 95: Add the directory above the Django project directory to Python’s sys.path.

sys.path.append("/home/joe/python_virtualenv")

Line 97: Add the project’s settings to os.environ.

os.environ['DJANGO_SETTINGS_MODULE'] = 'testproject.settings'

htaccess

For Django to respond to URL requests, those urls need to be fed into the django.cgi script. For testing I routed everything from /django to the cgi script by adding the following lines to my top-level htaccess file:

RewriteEngine on
RewriteRule ^cgi-bin/ - [L]
RewriteRule ^django/(.*)$ /cgi-bin/django.cgi/$1 [QSA,L]

The second line isn’t necessary unless pulling Django urls from the webroot, without it, the redirects would loop.

At this point, the Django site should load from /django/… urls.

Finally, as a quick fix for admin media files, I symlinked Django’s admin media directory from my web root:

ln -s ~/python_virtualenv/lib/python2.4/site-packages/django/contrib/admin/media ~/www/media

Conclusion

I spent quite a few hours spread across a couple days researching and figuring out how to get the first install working. The second installation only took about 5 minutes from start until editing Django’s admin pages.

Running Django through CGI is possible, but it is dog slow. There appears to be some caching after the first request, but that first page load often takes an excruciatingly long time.

Further reading, possible improvements

The servers I was working with are both running the almost six year old Python 2.4.3. The wsigref module was introduced with Python 2.5. My goal was to get Django running without compiling anything since some hosts deny access to GCC.

References

These sites were helpful in figuring this out.

The two hosts I tested on were LiquidWeb and A2Hosting. Both have been excellent, dependable hosts. Neither has any Python support to speak of on their shared plans. A2 blocks access to GCC.


Microsoft and Yahoo: LAMP, meet WAMP

After the obvious desire to takeover Yahoo’s unmatched traffic, the thing that most struck me about Microsoft’s proposed Yahoo! acquisition was what they’d do with Yahoo’s extensive foundation of Open Source Software.

Historically, Microsoft has had a deep institutional phobia about OSS. But Yahoo! uses PHP extensively, and Rasmus Lerdorf, the creator of PHP, has worked at Yahoo! since 2002.

This seems to make no sense. Unless the OSS and PHP backend is something Microsoft wants.

On January 31st, Mary Jo Foley published notes from an interview with Sam Ramji, Microsoft’s Director of Platform Technology Strategy. Foley rightly highlighted this quote from Ramji:

“Our focus is getting OSS on top of Windows, and I’m focused on (providing) interoperability between the LAMP (Linux, Apache, MySQL, PHP) and Windows stacks.”

She also posted this PowerPoint slide:
LAMP, meet WAMP

Boom, as they say. Microsoft wants to legitimize Windows as the foundation for a parallel WAMP stack. What better way to prove the viability of WAMP than running the biggest PHP web site in the world?

Microsoft may have finally realized that Open Source can be seen as a competitor, but also as free labor. Google and Apple, along with Yahoo! realized that a long time ago. Why try and compete directly when you can subvert it by becoming the dominant platform that software runs on? Instant credibility, and instant influence.

So far we’ve only seen the first chapter of this story, or perhaps the first act of this tragedy. The next phase looks like it may turn out to be Microsoft, Google and others fighting over Yahoo’s unfortunate carcass and tearing it to shreds.

Steve Ballmer has giant brass balls and Microsoft most likely anticipated that Google would do something to interfere with the acquisition. Microsoft is on the move. This should be an interesting week.

Disclosure: I’m currently holding Yahoo! stock and have previously owned stock in Microsoft.


Multiple RegisterResource directives broken in Leopard

This is sort of a follow up on the old mod_rendezvous article I wrote for O’Reilly. While cleaning up my virtual hosts I discovered a bug in 10.5’s handling of multiple RegisterResource directives in mod_bonjour. This is expanded from a bug report I submitted to Apple (rdar://problem/5628484).

I keep functional mirrors all my development sites in separate Apache Virtual Hosts. Each one then gets it’s own port, which allows me to check them on local networks, in Parallels and, if I want, remotely via IP address.

To advertise two local vhosts over bonjour, something like this in httpd.conf should work:

<IfModule bonjour_module>
RegisterResource "Site 1" / 9001
RegisterResource "Site 2" / 9002
</IfModule>

After restarting Apache (sudo apachectl graceful), local copies of Safari should see the two sites, “Site 1” and “Site 2” in Bonjour bookmark listings. In 10.4, they show up. In 10.5, only “Site 2” shows up. No matter how many directives are included, only the last one will be visible.

I’d love to be wrong about this, but it seems that something broke this function in Leopard.

A faster way of checking Bonjour entries is to open a terminal window and run the following command:

mdns -B _http._tcp

That will show a live updating list of current multicast (Bonjour) entries. Under 10.4, I get the following after adding the above directives to httpd.conf:

16:10:14.517  Add     0 local.     _http._tcp.     Site 1
16:10:14.667  Add     0 local.     _http._tcp.     Site 2

However with 10.5, I get this:

16:12:52.597  Add     1 local.     _http._tcp.     Site 2
16:12:52.598  Add     1 local.     _http._tcp.     Site 2
16:12:52.598  Add     0 local.     _http._tcp.     Site 2

What I’d really love to do is figure out how to register and respond to multiple Bonjour names. That way I could have each vhost be a named host and each staged site accessible at a url like site1.local and site2.local. So far I haven’t had any luck getting that working.