The problem

Django comes with a fancy tool to manage data needed by the project, it’s called fixtures. However the problem with fixtures is that they are:

• hard to create (¿dumpdata really?)
• hard to evolve (¿what if the model gets changed?)
• hard to modify (beyond trivial modifications)
• they overwritte whatever in db (this can be really harmful)

Data Driven apps throught south data migrations

For all these reasons they don’t seem to be handy to manage data in data driven apps. So, what can we do then?. Actually the answer is a third party app you may know called south. As you may know south cames up with a solution to do schema evolution (aka schema migrations), but it also carries data migrations. Using this tool in your data driven apps has a number of advantages:

• Your schema can evolve (this is the key feature of south)
• When you apply a datamigration, only required data is loaded
• The way you write data is much more similar to regular ORM queries

Let’s see how to create a data migration (schema migrations are out of the scope of this post).

The first thing you need to do is create the skeleton for the data migration.

$ ./manage.py datamigration <your_app> <migration_name>

This will create a new data migration. You may note the class is inherited from DataMigration instead of SchemaMigration. This is important if you may write a migration by hand as SchemaMigrations don’t let you add, modify or delete data. It has a forward and a backward method as usual, and a dict representing the models.

Your migration belongs to an app. If you want to handle data from that app you can do simply prepending “orm.” to the usual Django Orm statement. If you want to handle data from a different app you should prepend “orm[‘other_app’]“. Let’s see it in an example

class Migration(DataMigration):

    def forwards(self, orm):
        data1 = orm.<Model1>.objects.get_or_create(
            arg1="arg1",
            arg2=5)
        data2 = orm['other_app].<Model2>.objects.get_or_create(
            foo="bar",
            arg2=data1)           

If your project use multiples databases, then you may experience some problems running your tests. You have two approaches the first one would set ‘SOUTH_TESTS_MIGRATE’ to False in settings, but then your data migrations won’t be applied, therefore you’ll need to create that data in DB by yourself. The second one (and my favorite) it’s to exclude multiple database from testing environment; to achieve this you just need to set ‘DATABASE_ROUTERS’ to the empty list.

With the new 1.7 release (but the time of this writing a release candidate yet) the migration syntax it’s different, but the idea underneath it’s the same.