A commenter on my last post was curious how we use virtualenv to deploy sites and how it is helpful. It goes a bit deeper than that. There are a few parts that make everything work.

Pieces of the puzzle

1. Philosophy of modular, pluggable applications with well-defined dependencies. Ideally I would love for someone to be able to check off the applications that they need in a project and everything is ready. We’re not there yet, but it is a great goal.

2. Easy source repository creation and access management. We started on Subversion, and creating a new repository for a new modular app was a bit of a pain. If there is pain, it is avoided, so the repository wouldn’t get created. ProjectMgr (and then django-repositories) evolved out of that problem. Easy creation, sharing and access management from the Django admin.

3. Automated project and application setup. This piece fit into place with something called Django Project Skeleton I saw from Eric Florenzano and for the life of me, I can’t remember where and can’t locate it now. Somewhere he published some code for creating a project with virtualenv with a simple command. So major props to Eric for the very cool idea and code.

   We modified it a bit, and continue to, to fit our needs, but have published the code at our site. Updated: We’ve modified it and released it in two parts. The script is now a gist. Check it out with git:

   git clone git://gist.github.com/444408.git
   

   The project template example can be checked out at:

   https://github.com/callowayproject/project_tmpl

   We also have a version that generates a skeleton for a modular app under the name app-skeleton. Check it out with git:

   git clone git://github.com/washingtontimes/django-app-skeleton.git

4. Easy deployment of projects. Making a project easy to deploy requires:

   • self-contained environments (which we get through virtualenv in number 3)
   • easy installation of all dependencies (which we get through virtualenv and pip in number 3)
   • inclusion of all configurations, including web server, project and local settings (which are included in the template)

Creating a blank project

The process is fairly simple (for certain definitions of simple):

1. Open up a command line prompt and change to the project-skeleton directory

2. Execute python create_project.py

3. Answer the questions:

   • Project name
   • Destination directory (Full path to where the project will be saved)
   • Virtual environment name (we name it the same as the project name)

That is all there is to create a project. It is easy enough that our designer, who knows no Python, can easily create one and start flushing out a new site without a developer’s help. The only other things he needs to know:

• How to install additional external apps by modifying setup/requirements.txt
• Adding the external apps to settings.py and possibly urls.py
• The commands: ./manage.py syncdb and ./manage.py runserver
• Git commands like git init , git add , and git commit

The project structure

Let me explain a little about how the project template is structured.

apps/ Project-specific applications go here. There is one app called django_ext where Eric has put some common things used in projects.

bin/ Shell scripts for updating, installing and pushing project code

conf/ Configuration files including wsgi configuration, web servers and proxy configurations. Currently we only use the apache configuration and it is symlinked to the appropriate directory on the server.

lib/ Where project-specific python libraries go. The difference between an app and a library? If it needs to be included in Django’s settings.py, then it is an app; if it just needs to be imported, then it is a library.

media/ Cascading Style Scripts, Javascripts and images go here. The apache configuration is set to serve the media files from this directory.

setup/ Files required to set up the project. The _bootstrap.sh is here as well as the requirements.txt file for pip indicating dependencies. Actual gzipped packages could be included so that reliance on PyPI is not necessary.

templates/ Where you put your templates, with some basic starting ones.

Sharing the new project

Since the workflow typically starts with the designer, the entire project will be on his computer (in a local git repository). After the designer has worked out a few things (or everything) and wants to deploy it or have a developer work on it with him, he uses our ProjectMgr (now refactored to django-repositories) to create a git repository.

When he creates the repository, he also adds users and groups for commit access and sets the public viewing flag on or off.

Back in his command window, he adds the new remote repository with a command like:

git remote add origin http://opensource.washingtontimes.com/git/public/coolsite.git/

And then updates the remote repository:

git push origin master

Using a shared project

With a shared project using it, or deploying it is pretty straightforward::

git clone http://opensource.washingtontimes.com/git/public/coolsite.git/
cd coolsite
./setup/_bootstrap.sh
workon coolsite
./bin/install.sh

The ./setup/_bootstrap.sh sets up the virtualenv for you and downloads code, if necessary. Now you can workon coolsite and get going. Updates to code are pushed (git push origin master) and pulled (git pull).

We have several commands that make managing updates easier in the bin directory:

ext-status.sh Show the modifications (status) of all editable packages (works with Bazaar, Subversion, Git and Mercurial)

install.sh Create a symbolic link to the editable packages in the virtualenv, add a postactivate hook for virtualenv that automatically changes the directory to the project and run pip install -U -r setup/requirements.txt

pull-ext.sh Update all editable packages (works with Bazaar, Subversion, Git and Mercurial)

push-ext.sh Push out changes to the remote repository. (Only works with git)

upgrade.sh Update all project code and external editable packages. (Assumes the current project is stored in a git repository)

Conclusion

Although our designer, Jonathan Hensley, is not a Python programmer, he can take a request for a demo site and spec it out very quickly, without ever engaging a developer. This has made things easier on everyone. As a result, we gain some easier deployment and updating on sites because of the independent structure of their development and design.