Request for Comments: Auto-Installation of Apps in Django

The problem of pluggable Django apps

Lately we've come across a bit of an issue with dependencies. We use pip and virtualenv for our projects and it works great for installing Python dependencies.

However it is not so great if the dependency is a Django pluggable application. There is more you have to do: add all the dependent apps in INSTALLED_APPS and add the appropriate entry in your base urls.py. This can be a problem if you aren't familiar with the app, let alone its dependent apps.

Recently I used pip to test out what I thought was a stand-alone Django pluggable app, only to discover dependency after dependency. This isn't very pluggable in my book.

Proposal: A method of auto-installing apps

The Django community has developed several patterns that make development easier. Many of them are related to how you configure the settings.py file for your project. See examples from Usware from EuroDjangoCon. I'm proposing a new one, and would like assistance in refining it.

  • A pluggable application would contain its own settings.py file containing one or more standard settings: APP_NAME, VERSION, and BASE_URL_CONF.
  • In the project's settings.py file, you would loop through one or more paths containing Django apps/Python packages.
  • For each Python package, if it is not already in INSTALLED_APPS, attempt to import the settings module and see if it has an APP_NAME attribute.
  • If it does, add it to INSTALLED_APPS and then check for a BASE_URL_CONF attribute.
  • Add that to a list so you can append/prepend it to your urls.py.

This system IMHO is:

  • Simple to use
  • Easily turned off by adding the app to INSTALLED_APPS
  • Doesn't interfere with any other packages or functionality
  • Easily extensible or customizable

Example code is in a gist at git, but I'll show it here:

import importlib, os
APP_DIRS = (os.path.abspath(os.path.join(PROJECT_ROOT, 'test')),)
sys.path.extend(APP_DIRS)
BASE_URL_CONFS = []
for app_dir in APP_DIRS:
    for app in os.listdir(app_dir):
        if not app.startswith('.') and app not in INSTALLED_APPS:
            try:
                app_settings = importlib.import_module('%s.settings' % app)
                if getattr(app_settings, 'APP_NAME', '') != '':
                    print "Auto Installed %s" % app
                    INSTALLED_APPS = INSTALLED_APPS + (app,)
                base_url_conf = getattr(app_settings, 'BASE_URL_CONF', '')
                if base_url_conf != '':
                    BASE_URL_CONFS.append(base_url_conf)
            except ImportError:
                pass

I'm sure there can be some tweaking to the code, but it gets the idea across. Please feel free to send comments to me at coordt _at_ washingtontimes.com, or leave a comment down below.

16 Comments
For many non-trivial apps, a syncdb will also be required before the app could function. Do you also plan to automate that? Not sure that's a good idea.
No, I wouldn't recommend automating a syncdb. But if all the dependent apps were installed at the same time as the primary app, the syncdb that you would probably do for the primary, would also cover the dependents.
Order is important for urlpatterns, so automatically appending new patterns for each app has potential for great surprise. More importantly, a module should not be imported just because it happens to be on the path one keeps Django-pluggable apps.
You're right that order is important with urls, but if you append them, previous items would over ride the auto-installed. Plus, you can still add the app to INSTALLED_APPS if there is a conflict that you want to specifically handle. While you wouldn't want to point this to a general folder with all sorts of pluggable apps, in a virtualenv, you would only have the apps/packages you needed. Typically you would point it to the virtualenv's site-packages directory.
Please post entire articles in rss feeds. Personally I avoid reading feeds which only put a snippet in the feed.
Good point. Done.
Hi Corey. I've been working on something similar. You might be interested in: <a href="http://irukado.org/2009/09/15/blitzen-django-app-installation-only-faster/">Blitzen: Django App Installation, only faster</a>
Hi Corey. I've been working on something similar. You might be interested in: <a href="http://irukado.org/2009/09/15/blitzen-django-app-installation-only-faster/">Blitzen: Django App Installation, only faster</a>
Sorry about the double-post. You might want to check this form, as it's giving page not found errors on submit, then posting anyway.
This completely fails in one very common case of a bunch of apps in a directory that are used in multiple projects. Not every app in the directory is used in any particular project, but you install them all with your scheme, with all the unintended side-effects that brings to the table. Auto-detection is very fragile for precisely these sorts of reasons. So whilst this might work for your particular situations, it's more dangerous than useful, in general. Since it's only about three lines of code to populate INSTALLED_APPS from a directory, it's one of those things that people can easily enough do ad hoc, so I doubt a huge spec or anything is needed. I've seen a lot of projects doing things like this and a lot where it wouldn't be appropriate. Keep it simple (just pull in everything from a particular module and don't mix apps and non-apps in that module, for example) and don't view it as something needed great detail. The three-line solution is the right way here.
Attempts of auto-installing apps will inevitably fail, not to say that the idea looks plain stupid (sorry). This is the kind of thing you cannot automate, because the number of unknown variables, and the impact of those variables interacting, will ultimately introduce hard to find bugs and mess everything. E.g.: URLs, data models, order of signals being executed, conflicting settings, and so on... When its said that Django applications are "pluggable", is meaning "easy to extend". Not "install and go". Django is not a CMS.
Populating INSTALLED_APPS from a directory isn't even 3 lines of code. Django's installed apps can take a "wildcard", so "myproject.externalapps.*" for example would expand to all apps in that directory.
i'm not an expert, but i think this is not only good, is necesary, i think when i run django for first time i could run commands like startproject or startapp in the webbrowser aomething like webpy, this is a interesting idea for me, i will read more about this
Hello Corey, first of all, I'm a Brazilian developer, and my English is a little sucks. Well, I was working in a project to develop a e-learning system that needs to be able to bring a good install experience of new apps in it. I had the same thought that you, and this solution, actually, can be possible to implement, if your context is always the same. That is, as develop a plugin system like Drupal or Wordpress does, where you can define the rules to make a pluggable app that fits with your specifications. Otherwise, a bit of manual labor will always be necessary.
Hmm. No offense to anyone, but in the interests of exploring the issue honestly and adequately... I think it's much less sensible to install additional, potentially insecure, but entirely unused code in a web project, when version control branches should manage that sort of stuff for you. The whole principle of django is Don't Repeat Yourself, and yet Django makes me do that a lot every time I want a new featureset on a site. Many of the features in Django itself seem to be aimed at this functionality --- signal dispatch, get_absolute_url(), etc., and yet the main part to make it actually work is missing. Frankly, I think this should have been a design goal from day 1, and I think the arguments against automating this are nothing but the usual (and understandable) denial of the changes necessary to make it happen. Of course, if anyone can show a REAL situation where something is necessary, but conflicts with automation, I'd be more likely to accept the argument. But having a webcode directory full of apps you don't intend to use as apps isn't one of them, as far as I can see. If django was developed to allow for that, it would be developed based on worst practice. I think it's time to admit that django has been aiming for something, but not quite achieved it yet, and that just a little work remains.
Excellent approach! Despite things might be a bit more complicated to handle if you want to be part of the standard django distribution with it. I think the idea behind your proposal is excellent (a big part of zopes success story was the easy installation of ZObject based applications right out of the box).Django is missing that functionality! Still your proposal is (currently) to simple to work in all cases, so i really hope you refine it and pick up the suggestions that where posted here. I also could imagine an interactive assistant that guides through the decisions that have to made to install an application. So, all in all +1 for the idea
Commenting is disabled for this entry.
If you feel there is still something worth mentioning about this entry please contact the author or the site admin.