We have been experimenting with A/B testing using Google’s Website Optimizer. The tool is very nice, but we ran into issues with A/B testing in pages with dynamic content.

Website Optimizer is really geared for optimizing static pages and elements (like sign-up pages). You have to put the HTML for A and B versions in Website Optimizer up front. That is fine for some CSS changes or button element changes, but not very useful in changing an item whose content is variable.

Another method that Website Optimizer uses is different URLs for the same page (but not using query strings) and Website Optimizer handles the redirect. That could work, we thought, if only we could deliver different over-ridden templates based on the sub-domain. So version A could be www.washingtontimes.com/news/2010/Feb/16/some-story-name/ and version B could be www2.washingtontimes.com/news/2010/Feb/16/some-story-name/.

But Django doesn’t let you change the TEMPLATE_DIRS setting on the fly.

Looking at Loaders

I started investigating how template loaders work in Django, and found a big problem: the loaders don’t have access to the request. Without the request information, you obviously can substitute template directories based on request headers.

Then my co-worker, Jonathan Hensley, mentioned django-ab. Django-ab looks like a great project, but we couldn’t use it since the majority of our traffic is served from our Varnish caching servers. However, in his code, John Boxall has a template loader that alters the template delivered, based on request information.

Minding the Middleware

What John did was use a middleware to store the request in the local thread.

I’ve simplified it to:

try:
    from threading import local
except ImportError:
    from django.utils._threading_local import local

_thread_locals = local()
def get_current_request():
    return getattr(_thread_locals, 'request', None)

class RequestMiddleware(object):
    def process_request(self, request):
        print request.META['HTTP_USER_AGENT']
        _thread_locals.request = request

With this middleware added into your MIDDLEWARE_CLASSES setting, you can get access to the request.

Relooking at Loaders

One nice thing about Django’s templating is you have a search path for templates; if you don’t find the template in one directory, look in the next directory. This allows you to define over-ride templates. The order of the items in the TEMPLATE_LOADERS setting is very important, as it looks through the list and takes the first template it finds. So our template loader is going to define exception templates, or over-ridden template directories, and should be first in the list:

TEMPLATE_LOADERS = (
    'dynamicloader.loader.load_template_source',
    'django.template.loaders.filesystem.load_template_source',
    'django.template.loaders.app_directories.load_template_source',
}

With the middleware in place, a template loader can now access the request. (Note: Django 1.2 changes the implementation of template loaders. My examples will show the 1.0-1.1 implementation.)

The primary template load function is pretty straight forward:

def load_template_source(template_name, template_dirs=None):
    for filepath in get_template_sources(template_name, template_dirs):
        try:
            file = open(filepath)
            try:
                return (file.read().decode(settings.FILE_CHARSET), filepath)
            finally:
                file.close()
        except IOError:
            pass
    raise TemplateDoesNotExist(template_name)
load_template_source.is_usable = True

load_template_source calls get_template_sources which returns a list of paths to that template. load_template_source either returns the contents of the file or raises a TemplateDoesNotExist exception.

The code for finding the template doesn’t make much sense without seeing how you define where to look. To make it flexible, we went with a set of nested dictionaries, in the format:

{
    'HEADER': {
        re.compile('header_val'): (directory, directory,),
    }
}

This nested dictionary format allows for one or more headers to map to one or more potential values. (Note: If there are any other ideas out there for doing this better, please let us know). Also note that the values for each header are compiled regular expressions, which is very helpful for HTTP_USER_AGENT values.

def get_template_sources(template_name, template_dirs=None):
    request = get_current_request()
    if request:
        # Loop through the request.META mapping attributes
        for key, val in TEMPLATE_MAP.items():
            # Get the value from the request for that key
            req_val = request.META.get(key, None)
            if req_val is not None:
                # The request value exists, 
                for key, val in val.items():
                    if key.search(req_val):
                        for filepath in val:
                            yield safe_join(filepath, template_name)

Summary and Conclusion

The initial code is up on our site and on github. The code has passed our initial tests and will be going into some high-level testing very soon, and then into production.

To get it working:

• Add 'dynamicloader.middleware.RequestMiddleware' to your MIDDLEWARE_CLASSES setting.

• Add 'dynamicloader.loader.load_template_source' to the top of your TEMPLATE_LOADERS setting.

• Define a setting named DYN_TEMPLATE_MAP in the format shown above.