How Does It Work?

Django Evolution tracks all the apps in your Django project, recording information on the structure of models, their fields, indexes, and so on.

When you make any change to a model that would need to be reflected in the database, Django Evolution will tell you that you’ll need an evolution file to apply those changes, and will suggest one for you.

Evolution files describe one or more changes made to models in an app. They can:

Add fields
Change the attributes on fields
Rename fields
Delete fields
Change the indexes or constraints on a model
Rename models
Delete models
Rename apps
Delete apps
Transition an app to Django’s migrations
Run arbitrary SQL

Django Evolution looks at the last-recorded state of your apps, the current state, and the evolution files. If those evolution files are enough to update the database to the current state, then Django Evolution will process them, turning them into an optimized list of SQL statements, and apply them to the database.

This can be done for the entire database as a whole, or for specific apps in the database.

Since some apps (particularly Django’s own apps) make use of migrations (on Django 1.7 and higher), Django Evolution will also handle applying those migrations. It will do this in cooperation with the evolution files that it also needs to apply. However, it’s worth pointing out that migrations are never optimized the way evolutions are (this is currently a limitation in Django).

Frequently Asked Questions

Who maintains Django Evolution?

Originally, Django Evolution was built by two guys in Perth, Australia: Ben Khoo and Russell Keith-Magee (a core developer on Django).

Since then, Django Evolution has been taken over by Beanbag, Inc.. We have a vested interest in keeping this alive, well-maintained, and open source for Review Board and other products.

Where do I go for support?

We have a really old mailing list over at Google Groups, where you can ask questions. Truthfully, this group is basically empty these days, but you can still ask there and we’ll answer!

We also provide commercial support. You can reach out to us if you’re using Django Evolution in production and want the assurance of someone you can reach 24/7 if something goes wrong.

What about bug reports?

You can report bugs on our bug tracker, hosted on Splat.

When you file a bug, please be as thorough as possible. Ideally, we’d like to see the contents of your django_project_version and django_evolution tables before and after the upgrade, along with any evolution files, models, and error logs.

How do I contribute patches/pull requests?

We’d love to work with you on your contributions to Django Evolution! It’ll make our lives easier, for sure :)

While we don’t work with pull requests, we do accept patches on reviews.reviewboard.org, our Review Board server. You can get started by cloning our GitHub repository, and install RBTools (the Review Board command line tools).

To post new changes from your feature branch for review, run:

$ rbt post

To update an existing review request:

$ rbt post -u

See the RBTools documentation for more usage info.

Why evolutions and not migrations?

While most new projects would opt for Django’s own migrations, there are a few advantages to using evolutions:

Evolutions are faster to apply than migrations when upgrading between arbitrary versions of the schema.

Migrations are applied one at a time. If you have 10 migrations modifying one table, then you’ll trigger a table rebuild 10 times, which is slow – particularly if there’s a lot of data in that table.

Evolutions going through an optimization process before they’re applied, determining the smallest amount of changes needed. 10 evolutions for a table will generally only trigger a single table rebuild.

When you fully own the databases you’re upgrading, this may not matter, as you’re probably applying new migrations as you write them. However, if you are distributing self-installed web services (such as Review Board), administrators may not upgrade often. Evolutions help keep these large upgrades from taking forever.
There’s a wide range of Django support.

If you are still maintaining legacy applications on Django 1.6, it may be hard to transition to newer versions. By switching to Django Evolution, there’s a transition path. You can use evolutions for the apps you control without conflicting with migrations, and begin the upgrade path to modern versions of Django.

At any time, you can easily switch some or all of your apps from evolutions to migrations, and Django Evolution will take care of it automatically.
Django Evolution is easier for some development processes.

During development, you may make numerous changes to your database, necessitating schema changes that you wouldn’t want to apply in production. With migrations, you’d need to squash those development-only migration files, which doesn’t play as well if some beta users have only a subset of those migrations applied.

Can I switch apps from evolutions to migrations?

Yes, you can! The MoveToDjangoMigrations mutation will instruct Django Evolution to use migrations instead of evolutions for any future changes. Before it hands your app off entirely, it will apply any unapplied evolutions, ensuring a sane starting point for your new migrations.

Can I switch apps from migrations to evolutions?

No, it’s one way for now. We might add this if anyone wants it in the future. For now, we assume that people using migrations are satisfied with that, and aren’t looking to move to evolutions.

Why do my syncdb/migrate commands act differently?

Starting in Django Evolution 2.0, the evolve command has taken over all responsibilities for creating and updating the database, replacing syncdb and migrate.

For compatibility, those two commands have been replaced, wrapping evolve instead. Some functionality had to be stripped away from the original commands, though.

Our syncdb and migrate commands don’t support loading initial_data fixtures. This feature was deprecated in Django 1.7 and removed in 1.9, and keeping support between Django versions is tricky. We’ve opted not to include it (at least for now).

Our migrate command doesn’t support specifying explicit migration names to apply, or using --fake to pretend migrations were applied.

It’s possible we’ll add compatibility in the future, but only if demand is strong.

Installing Django Evolution

To install Django Evolution, simply run:

$ pip install django_evolution

You’ll probably want to add that as a package dependency to your project.

Then add django_evolution to your project’s INSTALLED_APPS.

Writing Evolutions

Evolution files describe a set of changes made to an app or its models. These are Python files that live in the appdir/evolutions/ directory. The name of the file (minus the .py extension) is called an evolution label, and can be whatever you want, so long as it’s unique for the app. These files look something like:

myapp/evolutions/my_evolution.py

from __future__ import unicode_literals

from django_evolution.mutations import AddField


MUTATIONS = [
    AddField('MyModel', 'my_field', models.CharField, max_length=100,
             null=True),
]

Evolution files can make use of any supported App and Model Mutations (classes like AddField above) to describe the changes made to your app or models.

Once you’ve written an evolution file, you’ll need to place its label in the app’s appdir/evolutions/__init__.py in a list called SEQUENCE. This specifies the order in which evolutions should be processed. These look something like:

myapp/evolutions/__init__.py

from __future__ import unicode_literals

SEQUENCE = [
    'my_evolution',
]

Example

Let’s go through an example, starting with a model.

blogs/models.py

class Author(models.Model):
    name = models.CharField(max_length=50)
    email = models.EmailField()
    date_of_birth = models.DateField()


class Entry(models.Model):
    headline = models.CharField(max_length=255)
    body_text = models.TextField()
    pub_date = models.DateTimeField()
    author = models.ForeignKey(Author)

At this point, we’ll assume that the project has been previously synced to the database using something like ./manage.py syncdb or ./manage.py migrate --run-syncdb. We will also assume that it does not make use of migrations.

Modifying Our Model

Perhaps we decide we don’t actually need the birthdate of the author. It’s just extra data we’re doing nothing with, and increases the maintenance burden. Let’s get rid of it.

 class Author(models.Model):
     name = models.CharField(max_length=50)
     email = models.EmailField()
-    date_of_birth = models.DateField()

The field is gone, but it’s still in the database. We need to generate an evolution to get rid of it.

We can get a good idea of what this should look like by running:

$ ./manage.py evolve --hint

Which gives us:

#----- Evolution for blogs
from __future__ import unicode_literals

from django_evolution.mutations import DeleteField


MUTATIONS = [
    DeleteField('Author', 'date_of_birth'),
]
#----------------------

Trial upgrade successful!

As you can see, we got some output showing us what the evolution file might look like to delete this field. We’re also told that this worked – this evolution was enough to update the database based on our changes. If we had something more complex (like adding a non-null field, requiring some sort of initial value), then we’d be told we still have changes to make.

Let’s dump this sample file in blogs/evolutions/remove_date_of_birth.py:

blogs/evolutions/remove_date_of_birth.py

from __future__ import unicode_literals

from django_evolution.mutations import DeleteField


MUTATIONS = [
    DeleteField('Author', 'date_of_birth'),
]

(Alternatively, we could have run ./manage.py evolve -w remove_date_of_birth, which would create this file for us, but let’s start off this way.)

Now we need to tell Django Evolution we want this in our evolution sequence:

blogs/evolutions/remove_date_of_birth.py

from __future__ import unicode_literals

SEQUENCE = [
    'remove_date_of_birth',
]

We’re done with the hard work! Time to apply the evolution:

$ ./manage.py evolve --execute

You have requested a database upgrade. This will alter tables and data
currently in the "default" database, and may result in IRREVERSABLE
DATA LOSS. Upgrades should be *thoroughly* reviewed and tested prior
to execution.

MAKE A BACKUP OF YOUR DATABASE BEFORE YOU CONTINUE!

Are you sure you want to execute the database upgrade?

Type "yes" to continue, or "no" to cancel: yes

This may take a while. Please be patient, and DO NOT cancel the
upgrade!

Applying database evolution for blogs...
The database upgrade was successful!

Tada! Now if you look at the columns for your blogs_author table, you’ll find that date_of_birth is gone.

You can make changes to your models as often as you need to. Add and delete the same field a dozen times across dozens of evolutions, if you like. Evolutions are automatically optimized before applied, resulting in the smallest set of changes needed to get your database updated.

Adding Dependencies

New in version 2.1.

Both individual evolution modules and the main myapp/evolutions/__init__.py module can define other evolutions or migrations that must be applied before or after the individual evolution or app as a whole.

This is done by adding any of the following to the appropriate module:

AFTER_EVOLUTIONS:: A list of specific evolutions (tuples in the form of (app_label, evolution_label)) or app labels (a single string) that must be applied before this evolution can be applied.
BEFORE_EVOLUTIONS:: A list of specific evolutions (tuples in the form of (app_label, evolution_label)) or app labels (a single string) that must be applied sometime after this evolution is applied.
AFTER_MIGRATIONS:: A list of migration targets (tuples in the form of (app_label, migration_name) that must be applied before this evolution can be applied.
BEFORE_MIGRATIONS:: A list of migration targets (tuples in the form of (app_label, migration_name) that must be applied sometime after this evolution is applied.

Django Evolution will apply the evolutions and migrations in the right order based on any dependencies.

This is important to set if you have evolutions that a migration may depend on (e.g., a swappable model that the migration requires), or if your evolutions are being applied in the wrong order (often only a problem if there are evolutions depending on migrations).

Note

It’s up to you to decide where to put these.

You may want to define this as its own empty initial.py evolution at the beginning of the SEQUENCE list, or to a more specific evolution within.

So, let’s look at an example:

blogs/evolutions/add_my_field.py

from __future__ import unicode_literals

from django_evolution.mutations import ...


BEFORE_EVOLUTIONS = [
    'blog_exporter',
    ('myapi', 'add_blog_fields'),
]

AFTER_MIGRATIONS = [
    ('fancy_text', '0001_initial'),
]

MUTATIONS = [
    ...
]

This will ensure this evolution is applied before both the blog_exporter app’s evolutions/models and the myapi app’s add_blog_fields evolution. At the same time, it’ll also ensure that it will be applied only after the fancy_text app’s 0001_initial migration has been applied.

Similarly, these can be added to the top-level evolutions/__init__.py file for an app:

blogs/evolutions/__init__.py

from __future__ import unicode_literals


BEFORE_EVOLUTIONS = [
    'blog_exporter',
    ('myapi', 'add_blog_fields'),
]

AFTER_MIGRATIONS = [
    ('fancy_text', '0001_initial'),
]

SEQUENCE = [
    'add_my_field',
]

This is handy if you need to be sure that this module’s evolutions or model creations always happen before or after that of another module, no matter which models may exist or which evolutions may have already been applied.

Hint

Don’t add dependencies if you don’t need to. Django Evolution will try to apply the ordering in the correct way. Use dependencies when it gets it wrong.

Make sure you test not only upgrades but the creation of brand-new databases, to make sure your dependencies are correct in both cases.

MoveToDjangoMigrations

If an evolution uses the MoveToDjangoMigrations mutation, dependencies will automatically be created to ensure that your evolution is applied in the correct order relative to any new migrations in that app.

That means that this:

MUTATIONS = [
    MoveToDjangoMigrations(mark_applied=['0001_initial'])
]

implies:

AFTER_MIGRATIONS = [
    ('myapp', '0001_initial'),
]

App and Model Mutations

Evolutions are composed of one or more mutations, which mutate the state of the app or models. There are several mutations included with Django Evolution, which we’ll take a look at here.

Field Mutations

AddField

AddField is used to add new fields to a table. It takes the following parameters:

class AddField(model_name, field_name, field_type, initial=None, **field_attrs)

Parameters:

model_name (str) – The name of the model the field was added to.
field_name (str) – The name of the new field.
field_type (type) – The field class.
initial – The initial value to set for the field. Each row in the table will have this value set once the field is added. It’s required if the field is non-null.
field_attrs (dict) – Attributes to pass to the field constructor. Only those that impact the schema of the table are considered (for instance, null=... or max_length=..., but not help_text=....

For example:

from django.db import models
from django_evolution.mutations import AddField


MUTATIONS = [
    AddField('Book', 'publish_date', models.DateTimeField, null=True),
]

ChangeField

ChangeField can make changes to existing fields, altering the attributes (for instance, increasing the maximum length of a CharField).

Note

This cannot be used to change the field type.

It takes the following parameters:

class ChangeField(model_name, field_name, initial=None, **field_attrs)

Parameters:

model_name (str) – The name of the model containing the field.
field_name (str) – The name of the field to change.
field_type –
The new type of field. This must be a subclass of Field.

This will do its best to change one field type to another, but not all field types can be changed to another type. Some types may be database-specific.

New in version 2.2.
initial – The new initial value to set for the field. If the field previously allowed null values, but null=False is being passed, then this will update all existing rows in the table to have this initial value.
field_attrs (dict) – The field attributes to change. Only those that impact the schema of the table are considered (for instance, null=... or max_length=..., but not help_text=....

For example:

from django.db import models
from django_evolution.mutations import ChangeField


MUTATIONS = [
    ChangeField('Book', 'name', max_length=100, null=False),
]

DeleteField

DeleteField will delete a field from the table, erasing its data from all rows. It takes the following parameters:

class DeleteField(model_name, field_name)

Parameters:

model_name (str) – The name of the model containing the field to delete.
field_name (str) – The name of the field to delete.

For example:

from django.db import models
from django_evolution.mutations import ChangeField


MUTATIONS = [
    ChangeField('Book', 'name', max_length=100, null=False),
]

RenameField

RenameField will rename a field in the table, preserving all stored data. It can also set an explicit column name (in case the name is only changing in the model) or a ManyToManyField table name.

If working with a ManyToManyField, then the parent table won’t actually have a real column backing it. Instead, the relationships are all maintained using the “through” table created by the field. In this case, the db_column value will be ignored, but db_table can be set.

It takes the following parameters:

class RenameField(model_name, old_field_name, new_field_name, db_column=None, db_table=None)

Parameters:

model_name (str) – The name of the model containing the field to delete.
old_field_name (str) – The old name of the field on the model.
new_field_name (str) – The new name of the field on the model.
db_column (str) – The explicit name of the column on the table to use. This may be the original column name, if the name is only being changed on the model (which means no database changes may be made).
db_table (str) –
The explicit name of the “through” table to use for a ManyToManyField. If changed, then that table will be renamed. This is ignored for any other types of fields.

If the table name hasn’t actually changed, then this may not make any changes to the database.

For example:

from django_evolution.mutations import RenameField


MUTATIONS = [
    RenameField('Book', 'isbn_number', 'isbn', column_name='isbn_number'),
    RenameField('Book', 'critics', 'reviewers',
                db_table='book_critics')
]

Model Mutators

ChangeMeta

ChangeMeta can change certain bits of metadata about a model. For example, the indexes or unique-together constraints. It takes the following parameters:

class ChangeMeta(model_name, prop_name, new_value)

Parameters:

model_name (str) – The name of the model containing the field to delete.
prop_name (str) – The name of the property to change, as documented below.
new_value – The new value for the property.

The properties that can be changed depend on the version of Django. They include:

db_table_comment:

A comment to apply to the table’s schema.

This requires Django 4.2 or higher.

Version Added:: 2.3

index_together:

Groups of fields that should be indexed together in the database.

This is represented by a list of tuples, each of which groups together multiple field names that should be indexed together in the database.

index_together support requires Django 1.5 or higher. The last versions of Django Evolution to support Django 1.5 was the 0.7.x series.

indexes:

Explicit indexes to create for the model, optionally grouping multiple fields together and optionally naming the index.

This is represented by a list of dictionaries, each of which contain a fields key and an optional name key. Both of these correspond to the matching fields in Django’s Index class.

indexes support requires Django 1.11 or higher.

unique_together:

Groups of fields that together form a unique constraint. Rows in the database cannot repeat the same values for those groups of fields.

This is represented by a list of tuples, each of which groups together multiple field names that should be unique together in the database.

unique_together support is available in all supported versions of Django.

For example:

from django_evolution.mutations import ChangeMeta


MUTATIONS = [
    ChangeMeta('Book', 'index_together', [('name', 'author')]),
]

Changed in version 2.0: Added support for indexes.

DeleteModel

DeleteModel removes a model from the database. It will also remove any “through” models for any of its ManyToManyFields. It takes the following parameters:

class DeleteModel(model_name)

Parameters:: model_name (str) – The name of the model to delete.

For example:

from django_evolution.mutations import DeleteModel


MUTATIONS = [
    DeleteModel('Book'),
]

RenameModel

RenameModel will rename a model and update all relations pointing to that model. It requires an explicit underlying table name, which can be set to the original table name if only the Python-side model name is changing. It takes the following parameters:

class RenameModel(old_model_name, new_model_name, db_table)

Parameters:

old_model_name (str) – The old name of the model.
new_model_name (str) – The new name of the model.
db_table (str) – The explicit name of the underlying table.

For example:

from django_evolution.mutations import RenameModel


MUTATIONS = [
    RenameModel('Critic', 'Reviewer', db_table='books_reviewer'),
]

App Mutators

DeleteApplication

DeleteApplication will remove all the models for an app from the database, erasing all associated data. This mutation takes no parameters.

Note

Make sure that any relation fields from other models to this app’s models have been removed before deleting an app.

In many cases, you may just want to remove the app from your project’s INSTALLED_APPS, and leave the data alone.

For example:

from django_evolution.mutations import DeleteApplication


MUTATIONS = [
    DeleteApplication(),
]

MoveToDjangoMigrations

MoveToDjangoMigrations will tell Django Evolution that any future changes to the app or its models should be handled by Django’s migrations instead evolutions. Any unapplied evolutions will be applied before applying any migrations.

This is a one-way operation. Once an app moves from evolutions to migrations, it cannot move back.

Since an app may have had both evolutions and migrations defined in the tree (in order to work with both systems), this takes a mark_applied= parameter that lists the migrations that should be considered applied by the time this mutation is run. Those migrations will be recorded as applied and skipped.

class MoveToDjangoMigrations(mark_applied=['0001_initial'])

Parameters:: mark_applied (list) – The list of migrations that should be considered applied when running this mutation. This defaults to the 0001_initial migration.

For example:

from django_evolution.mutations import MoveToDjangoMigrations


MUTATIONS = [
    MoveToDjangoMigrations(mark_applied=['0001_initial',
                                         '0002_book_add_isbn']),
]

New in version 2.0.

RenameAppLabel

RenameAppLabel will rename the stored app label for the app, updating all references made in other models. It won’t change indexes or any database state, however.

Django 1.7 moved to an improved concept of app labels that could be customized and were guaranteed to be unique within a project (we’ll call these modern app labels). Django 1.6 and earlier generated app labels based on the app’s module name (legacy app labels).

Because of this, older stored project signatures may have grouped together models from two different apps (both with the same app labels) together. Django Evolution will try to untangle this, but in complicated cases, you may need to supply a list of model names for the app (current and possibly older ones that have been removed). Whether you need to do this is entirely dependent on the structure of your project. Test it in your upgrades.

This takes the following parameters:

class RenameAppLabel(old_app_label, new_app_label, legacy_app_label=None, model_names=None)

Parameters:

old_app_label (str) – The old app label that’s being renamed.
new_app_label (str) – The new modern app label to rename to.
legacy_app_label (str) – The legacy app label for the new app name. This provides compatibility with older versions of Django and helps with transition apps and models.
model_names (list) – The list of model names to move out of the old signature and into the new one.

For example:

from django_evolution.mutations import RenameAppLabel


MUTATIONS = [
    RenameAppLabel('admin', 'my_admin', legacy_app_label='admin',
                   model_names=['Report', 'Config']),
]

New in version 2.0.

Other Mutators

SQLMutation

SQLMutation is an advanced mutation used to make arbitrary changes to a database and to the stored project signature. It may be used to make changes that cannot be made by other mutators, such as altering tables not managed by Django, changing a table engine, making metadata changes to the table or database, or modifying the content of rows.

SQL from this mutation cannot be optimized alongside other mutations.

This takes the following parameters:

class SQLMutation(tag, sql, update_func=None)

Parameters:

tag (str) – A unique identifier for this SQL mutation within the app.
sql (list/str) – A list of SQL statements, or a single SQL statement as a string, to execute. Note that this will be database-dependent.
update_func (callable) – A function to call to perform additional operations or update the project signature.

Note

There’s some caveats with providing an update_func.

Django Evolution 2.0 introduced a new form for this function that takes in a django_evolution.mutations.Simulation object, which can be used to access and modify the stored project signature. This is safe to use (well, relatively – try not to blow anything up).

Prior versions supported a function that took two arguments: The app label of the app that’s being evolved, and a serialized dictionary representing the project signature.

If using the legacy style, it’s possible that you can mess up the signature data, since we have to serialize to an older version of the signature and then load from that. Older versions of the signature don’t support all the data that newer versions do, so how well this works is really determined by the types of evolutions that are going to be run.

We strongly recommend updating any SQLMutation calls to use the new-style function format, for safety and future compatibility.

For example:

from django_evolution.mutations import SQLMutation


def _update_signature(simulation):
    pass


MUTATIONS = [
    SQLMutation('set_innodb_engine',
                'ALTER TABLE my_table ENGINE = MYISAM;',
                update_func=_update_signature),
]

Changed in version 2.0: Added the new-style update_func.

Management Commands

evolution-project-sig

The evolution-project-sig command is used to list, show, and delete stored project signatures.

This is really only useful if you’re working to recover from a bad state where you’ve undone the changes made by an evolution and need to re-apply it. It should never be used under normal use, especially on a production database.

By default, this command will confirm before making any changes to the database. You can use --noinput to avoid the confirmation step.

Example

To list project signatures:

$ ./manage.py evolution-project-sig --list

To show the latest project signature:

$ ./manage.py evolution-project-sig --show

To show a specific project signature:

$ ./manage.py evolution-project-sig --show --id <ID>

To delete a project signature:

$ ./manage.py evolution-project-sig --delete --id <ID>

Arguments

---delete: Delete a project signature.

--list: List project signatures and their associated evolutions.

--show: Show the current project signature, or an older one if using --id.

--id: Specify the ID of a project signature.

--noinput: Delete without prompting for confirmation.

evolve

The evolve command is responsible for setting up databases and applying any evolutions or migrations.

This is a replacement for both the syncdb and migrate commands in Django. Running either of this will wrap evolve (though not all of the command’s arguments will be supported when Django Evolution is enabled).

Creating/Updating Databases

To construct a new database or apply updates, you will generally just run:

$ ./manage.py evolve --execute

This is the most common usage for evolve. It will create any missing models and apply any unapplied evolutions or migrations.

Changed in version 2.0: evolve now replaces both syncdb and migrate. In previous versions, it had to be run after syncdb.

Generating Hinted Evolutions

When making changes to a model, it helps to see how the evolution should look before writing it. Sometimes the evolution will be usable as-is, but sometimes you’ll need to tweak it first.

To generate a hinted evolution, run:

$ ./manage.py evolve --hint

Hinted evolutions can be automatically written by using --write, saving you a little bit of work:

$ ./manage.py evolve --hint --write my_new_evolution

This will take any app with a hinted evolution and write a appdir/evolutions/my_new_evolution.py file. You will still need to add your new evolution to the SEQUENCE list in appdir/evolutions/__init__.py.

If you only want to write hints for a specific app, pass the app labels on the command line, like so:

$ ./manage.py evolve --hint --write my_new_evolution my_app

Arguments

<APP_LABEL...>: Zero or more specific app labels to evolve. If provided, only these apps will have evolutions or migrations applied. If not provided, all apps will be considered for evolution.

--database <DATABASE>: The name of the configured database to perform the evolution against.

--hint: Display sample evolutions that fulfill any database changes for apps and models managed by evolutions. This won’t include any apps or models managed by migrations.

--noinput: Perform evolutions automatically without any input.

--purge: Remove information on any non-existent applications from the stored project signature. This won’t remove the models themselves. For that, see DeleteModel or DeleteApplication.

--sql: Display the generated SQL that would be run if applying evolutions. This won’t include any apps or models managed by migrations.

-w <EVOLUTION_NAME>, --write <EVOLUTION_NAME>: Write any hinted evolutions to a file named appdir/evolutions/EVOLUTION_NAME. This will not include the evolution in appdir/evolutions/__init__.py.

-x, --execute: Execute the evolution process, applying any evolutions and migrations to the database.

Warning

This can be used in combination with --hint to apply hinted evolutions, but this is generally a bad idea, as the execution is not properly repeatable or trackable.

list-evolutions

The list-evolutions command lists all the evolutions that have so far been applied to the database. It can be useful for debugging, or determining if a specific evolution has yet been applied.

Example

$ ./manage.py list-evolutions my_app
Applied evolutions for 'my_app':
    add_special_fields
    update_app_label
    change_name_max_length

Arguments

<APP_LABEL...>: Zero or more specific app labels to list. If provided, only evolutions on these apps will be shown.

mark-evolution-applied

The mark-evolution-applied command is used to mark evolutions as already applied in the database.

This is really only useful if you’re working to recover from a bad state where you’ve undone the changes made by an evolution and need to re-apply it. It should never be used under normal use, especially on a production database.

By default, this command will confirm before marking the evolution as applied. You can use --noinput to avoid the confirmation step.

Example

$ ./manage.py mark-evolution-applied --app-label my_app \
      change_name_max_length

Arguments

EVOLUTION_LABEL ...: One or more specific evolution labels to mark as applied. This is required if --all isn’t specified.

--all: Mark all unapplied evolutions as applied.

--app-label <APP_LABEL>: An app label the evolutions apply to.

--noinput: Mark as applied without prompting for confirmation.

wipe-evolution

The wipe-evolution command is used to remove evolutions from the list of applied evolutions.

This is really only useful if you’re working to recover from a bad state where you’ve undone the changes made by an evolution and need to re-apply it. It should never be used under normal use, especially on a production database.

By default, this command will confirm before wiping the evolution from the history. You can use --noinput to avoid the confirmation step.

To see the list of evolutions that can be wiped, run list-evolutions.

Example

$ ./manage.py wipe-evolution --app-label my_app change_name_max_length

Arguments

EVOLUTION_LABEL ...: One or more specific evolution labels to remove from the database. If the same evolution names exist for multiple apps, they’ll all be removed. To isolate them to a specific app, use --app-label.

--app-label <APP_LABEL>: An app label to limit evolution labels to. Only evolutions on this app will be wiped.

--noinput: Perform the wiping procedure automatically without any input.

Glossary

evolution label

The name of a particular evolution for an app. These must be unique within an app, but do not have to be unique within a project.

legacy app label

legacy app labels

The form of app label used in Django 1.6 and earlier. Legacy app labels are generated solely from the app’s module name.

migrations

Django 1.7+’s built-in method of managing changes to the database schema. See the migrations documentation.

modern app label

modern app labels

The form of app label used in Django 1.7 and later. Modern app labels default to being generated from the app’s module name, but can be customized.

project signature

project signatures

A stored representation of all the apps and models in your project. This is stored in the django_project_version table, and is a critical part in determining how the database has evolved and what changes need to be made.

In Django Evolution 2.0 and higher, this is stored as JSON data. In prior versions, this was stored as Pickle protocol 0 data.

Release Notes

2.x Releases

Django Evolution 2.3

Release date: October 15, 2023

Installation

Django Evolution 2.3 is compatible with Django 1.6-4.2, and Python 2.7 and 3.6-3.12.

To install Django Evolution 2.3, run:

$ pip3 install django_evolution==2.3

To learn more, see:

New Features

Added support for Python 3.12 and Django 4.2.
Added support for evolving table comments on Django 4.2.

This is done through ChangeMeta.
Added advanced management commands for working with project signatures and marking evolutions as applied.

mark-evolution-applied will mark one or more evolutions as applied to your database, without modifying any schema.

evolution-project-sig will let you list project signatures, show a stored project signature, or delete project signatures.

These are advanced and dangerous commands. They should only be run if you know what you’re doing, as part of diagnosing and fixing a failed database upgrade.
Added debug logging for the evolution process.

If Python’s logging is set up to enable debug output, then the evolution process will provide information on the new models generation, mutations, and evolutions begin run. This can aid in debugging efforts.

Contributors

Christian Hammond
David Trowbridge

Django Evolution 2.2

Release date: October 3, 2022

New Features

Added support for Django 3.2 through 4.1.

This includes full support for django.db.models.Index, and compatibility with database backend changes made in these versions.
Added support for changing a field’s type in ChangeField.

This can be done by passing in the new field class to field_type=....
Added a new settings.DJANGO_EVOLUTION setting.

This is in the form of:
```
DJANGO_EVOLUTION = {
    'CUSTOM_EVOLUTIONS': {
        '<app_label>': ['<evolution_module>', ...],
    },
    'ENABLED': <bool>,
}
```
This replaces settings.CUSTOM_EVOLUTIONS and settings.DJANGO_EVOLUTION_ENABLED, both of which are now deprecated and will emit deprecation warnings.

Bug Fixes

General

Fixed generating SQL to execute while in a transaction on Django 2.0+.

Indexes/Constraints

Fixed ordering issues when dropping and re-creating indexes when changing db_index and unique states.
Fixed deferring constraints and indexes when injecting new models into the database.

The constraints and indexes were being added too soon, which could cause problems when applying more complicated batches of evolution.
Fixed issues with setting non-string initial data from a callable.
Fixed attempting to temporarily remove indexes and constraints that reference models not yet injected into the database.
Fixed edge cases with the tracking of standard vs. unique indexes in database state on Django 1.6.

MySQL

Fixed bad attempts at applying defaults to certain field types.

Django Evolution will no longer apply a default on text, blob, json, and all short/medium/long variations of those.

Python Compatibiltiy

Fixed an unintended deprecation warning with the collections module when running on Python 3.10.

Contributors

Christian Hammond
David Trowbridge

Django Evolution 2.1.4

Release date: February 28, 2022

Bug Fixes

Fixed a crash when applying Django compatibility patches on Django < 2.0 when mysqlclient isn’t installed.

Contributors

Christian Hammond
David Trowbridge

Django Evolution 2.1.3

Release date: January 25, 2022

Compatibility Changes

Patched compatibility between modern versions of mysqlclient and Django <= 1.11.

Django, up through 1.11, attempted to access a bytes key in an internal mapping on the database connection handle supplied by mysqlclient. This wasn’t intended to be present, and was due to a Python 2/3 compatibility issue.

They worked around this for a while, but dropped that support in the recent 2.1 release. To maintain compatibility, Django Evolution now patches Django’s own copy of the mapping table to restore the right behavior.
Patched Python 3.10+’s collections module to include legacy imports when using Django 2.0 or older.

Django 2.0 and older made use of some imports that no longer exist on Python 3.10. Django Evolution will now bring back this support when running this combination of versions of Django.

Bug Fixes

During upgrade, evolutions are no longer applied to newly-added models.
Fixed comparison issues between unique_together state from very old databases and newer evolutions.

This could lead to issues applying evolutions that only supply a unique_together baseline, or that differ in terms of using tuples or lists.
Fixed an edge case where the django_evolution app could be loaded too early when setting up a new database, causing crashes.
Updated to avoid using some deprecated Python and Django functionality.

We had some imports and function calls that were emitting deprecation warnings, depending on the versions of Python and Django. Code has been update to use modern imports and calls where possible,

Contributors

Christian Hammond
David Trowbridge

Django Evolution 2.1.2

Release date: January 19, 2021

Bug Fixes

Fixed a regression with adding new non-NULL columns on SQLite databases.
Fixed a possible data loss bug when changing NULL columns to non-NULL on SQLite databases.

Contributors

Christian Hammond

Django Evolution 2.1.1

Release date: January 17, 2021

Bug Fixes

Fixed changing a DecimalField’s decimal_places and max_digits attributes.
Changed the “No upgrade required” text to “No database upgrade required.”

While not a bug, this does help avoid confusion when running as part of a project’s upgrade process, when database changes aren’t the only changes being made.

Contributors

Christian Hammond

Django Evolution 2.1

Release date: November 16, 2020

New Features

Dependency management for evolutions and migrations.

Evolutions can now specify other evolutions and migrations that must be applied either before or after. This allows evolutions to, for instance, introduce a model that would be required by another migration (useful for Django apps that have migrations that depend on a swappable model specified in settings).

Django Evolution will determine the correct order in which to apply migrations and evolutions, so as to correctly create or update the database.

Dependencies can be defined per-evolution or per-app. They can depend on specific evolutions or on app evolutions for an app, or on specific migrations.

See Adding Dependencies for more information.
Improved transaction management.

Transactions are managed a bit more closely now, allowing more operations to be performed in a transaction at a time and for those operations to be rolled back if anything goes wrong. This should improve reliability of an upgrade.

Bug Fixes

General

Fixed the order in which models are created.

There was a regression in 2.0 where models could be created in the wrong order, causing issues with applying constraints between those models.
Fixed error messages in places if stored schema signatures were missing.

Previously, some missing schema signatures could lead to outright crashes, if things went wrong. There’s now checks in more places to ensure there’s at least a reasonable error message.

MySQL/MariaDB

Fixed preserving the db_index= values for fields on Django 1.8 through 1.10.

These versions of Django “temporarily” unset the db_index attribute on fields when generating SQL for creating indexes, and then never restore it. We now monkey-patch these versions of Django to restore these values.

Contributors

Christian Hammond

Django Evolution 2.0

Release date: August 13, 2020

New Features

All-New Documentation

We have new documentation for Django Evolution, covering installation, usage, a FAQ, and all release notes.

Support for Python 3

Django Evolution is now fully compatible with Python 2.7 and 3.5 through 3.8, allowing it to work across all supported versions of Django.

Speaking of that…

Support for Django 1.6 through 3.1

Django Evolution 2.0 supports Django 1.6 through 3.1. Going forward, it will continue to support newer versions of Django as they come out.

This includes modern features, like Meta.indexes and Meta.conditions.

We can offer this due to the new cooperative support for Django’s schema migrations.

Compatibility with Django Migrations

Historically, Django Evolution has been a standalone schema migration framework, and was stuck with supporting versions of Django prior to 1.7, since evolutions and migrations could not co-exist.

That’s been resolved. Django Evolution now controls the entire process, applying both migrations and evolutions together, ensuring a smooth upgrade. Projects get the best of both worlds:

The ability to use apps that use migrations (most everything, including Django itself)
Optimized upgrades for the project’s own evolution-based models (especially when applying large numbers of evolutions to the same table)

New Evolve Command

In Django Evolution 2.0, the evolve command becomes the sole way of applying both evolutions and migrations, replacing the migrate/syncdb commands.

To set up or upgrade a database (using both evolutions and migrations), you’ll simply run evolve --execute. This will work across all versions of Django.

The old migrate and syncdb commands will still technically work, but they’ll wrap evolve --execute.

This can all be disabled by setting DJANGO_EVOLUTION_ENABLED = False in settings.py.

Note

initial_data fixtures will no longer be loaded. These have already been deprecated in Django, but it’s worth mentioning for users of older versions of Django.

Also, the migrate command will no longer allow individual migrations to be applied.

Moving Apps to Migrations

Projects can transition some or all of their apps to migrations once the last of the evolutions are applied, allowing them to move entirely onto migrations if needed. This is done with the new MoveToMigrations mutation.

Simply add one last evolution for an app:

from django_evolution.mutations import MoveToDjangoMigrations


 MUTATIONS = [
     MoveToDjangoMigrations(),
 ]

This will apply after the last evolution is applied, and from then on all changes to the models will be controlled via migrations.

Note

Once an app has been moved to migrations, it cannot be moved back to evolutions.

Improved Database Compatibility

Support for constraints on modern versions of MySQL/MariaDB.

Modern versions of MySQL and MariaDB are now explicitly supported, allowing projects using Django 2.2+ to take advantage of CHECK constraints. This requires MySQL 8.0.16+ or MariaDB 10.2.1+ on Django 3.0+.
Faster and safer SQLite table rebuilds.

Changes to SQLite databases are now optimized, resulting in far fewer table rebuilds when changes are made to a model.
Support for SQLite 3.25+ column renaming.

SQLite 3.25 introduced ALTER TABLE ... RENAME COLUMN syntax, which is faster than a table rebuild and avoids a lot of issues with preserving column references.
We use Django 1.7’s schema rewriting for more of the SQL generation.

This helps ensure future compatibility with new releases of Django, and allows for leveraging more of Django’s work toward database compatibility.

Project-Defined Custom Evolutions

Projects can provide a new settings.CUSTOM_EVOLUTIONS setting to define custom evolution modules for apps that don’t otherwise make use of evolutions or migrations. The value is a mapping of app module names (same ones you’d see in settings.INSTALLED_APPS to an evolutions module path.

This looks like:

CUSTOM_EVOLUTIONS = {
    'other_project.contrib.foo': 'my_project.compat.foo.evolutions',
}

Evolver API

The entire evolution/migration process can now be controlled programmatically through the Evolver class. This allows an entire database, or just select apps, to be evolved without calling out to a management command.

While most projects will not have a need for this, it’s available to those that might want some form of specialized control over the evolution process (for automation, selectively evolving models from an extension/plug-in, or providing an alternative management/upgrade experience).

During an evolution, new signals are emitted, allowing apps to hook into the process and perform any updates they might need:

New Database Signature Format

Django Evolution stores a representation of the database in the Version table, in order to track what’s been applied and what changes have been made since.

Historically, this has used some older structured data schema serialized in Pickle Protocol 0 format. As of Django Evolution 2.0, it’s now using a new schema stored in JSON format, which is designed for future extensibility.

Internally, this is represented by a set of classes with a solid API that’s independent of the storage format. This eases the addition of new features, and makes it easier to diagnose problems or write custom tools.

Warning

This will impact any SQLMutations that modify a signature. These will need to be updated to use the new classes, instead of modifying the older schema dictionaries.

Bug Fixes

SQLite

Fixed constraint references from other tables when renaming primary key columns.
Fixed restoring all table indexes after rebuilding a table.

Contributors

Christian Hammond

0.7 Releases

Django Evolution 0.7.8

Release date: June 14, 2018

Packaging

Eggs and wheels are now built only for Python 2.7.

Older versions of Python are no longer packaged. Source tarballs may work, but we recommend that anyone still on older versions of Python upgrade at their earliest convenience.

Bug Fixes

Fixed an issue generating unique_together constraints on Postgres in some configurations.

Depending on the table/index names, unique_together constraints could fail to generate on Posrgres, since the names weren’t being escaped.

Contributors

Christian Hammond

Django Evolution 0.7.7

Release date: May 25, 2017

New Features

Added a note about backing up the database and not cancelling before executing an evolution.

The confirmation prompt for executing an evolution now suggests backing up the database first. This is only shown in interactive mode.

After the user has confirmed, they’re told it may take time and to not cancel the upgrade.
Added more output when performing evolutions for apps.

When evolving the database, a message is now outputted to the console for each app being evolved. This gives a sense of progress for larger evolutions.

If the evolution fails, an error message will be shown listing the app that failed evolution, the specific SQL statement that failed, and the database error. This can help when diagnosing and recovering from the problem.
Added an option for writing hinted evolution files.

There’s now an evolve -w/--write option that can be used with evolve --hint that writes the hinted evolution to the appropriate directories in the tree. This takes the name that should be used for the evolution file.

This will not update the evolutions/__init__.py file.

Bug Fixes

Fixed issues with evolution optimizations when renaming models.

Django Evolution’s evolution optimization code had issues when applying a series of evolutions that add a ForeignKey field to a newly-introduced model that is then renamed in the same batch. The resulting field would still point to the original model, resulting in a KeyError.

Contributors

Christian Hammond

Django Evolution 0.7.6

Release date: December 1, 2015

Bug Fixes

Fixed a false positive with schema errors when applying evolutions on MySQL.

When applying new evolutions along with baseline schemas for new models, two version history entries are created, one for the new baselines, and one for the new, final schema. On MySQL, this can happen so quickly that they’ll end up with the same timestamp (as there isn’t a lot of precision in these fields).

Due to internal sort orders, the next evolution then finds the version entry for the baseline schema, and not the final evolved schema, causing it to fail saying that there are changes that couldn’t be applied.

This fixes this problem by improving the sorting order.
Fixed issues evolving certain changes from old database schemas.

Old database schemas didn’t track certain information, like the index_together information. The code was previously assuming the existence of this information and failing if it wasn’t there. Evolving from these older schemas now works.

Contributors

Barret Rennie
Christian Hammond

Django Evolution 0.7.5

Release date: April 13, 2015

Bug Fixes

Mutations on fields with the same name across different models no longer results in conflicts.

With the new optimizer in Django Evolution 0.7, it was possible for mutations to be incorrectly optimized out if, for example, a field was added in one model and then later changed in another model, if both fields had the same name. This was due to the way in which we mapped mutations, and would result in an error in the validation stage before attempting any database modifications. There are no longer any conflicts between same-named field.
Indexes are no longer created/deleted unnecessarily.

If setting an index for a field, and it already exists in the database, there’s no longer an attempt at creating it. Likewise, there’s no longer an attempt at deleting an index that does not exist.

Contributors

Christian Hammond

Django Evolution 0.7.4

Release date: September 15, 2014

New Features

Add a RenameModel mutation for handling model renames.

The new RenameModel mutation allows an evolution to indicate that a model has been renamed. This handles updating the signature for any related ForeignKey or ManyToManyField fields and generating any SQL to perform the table rename (if needed).

Contributors

Christian Hammond

Django Evolution 0.7.3

Release date: July 24, 2014

Bug Fixes

Fixed issues evolving unique_together attributes on models.

When adding unique_together constraints and then changing them within a single evolve operation, any constraints listed more than once would result in unnecessary duplicate SQL statements. These would cause errors that would prevent the transaction from completing.
Adding and removing a unique_together constraint within an evolve operation no longer breaks on PostgreSQL.
Errors importing a database backend on a modern Django no longer results in unrelated errors about settings.DATABASE_ENGINE.

Contributors

Christian Hammond

Django Evolution 0.7.2

Release date: June 2, 2014

Bug Fixes

Fixed a crash from no-op column renames on PostgreSQL.

When attempting to rename a column on PostgreSQL and specifying a “new” name that was the same as the old name, the result would be a crash. This is similar to the bug fixed in Django Evolution 0.7.1.

Contributors

Christian Hammond

Django Evolution 0.7.1

Release date: May 21, 2014

New Features

Fixed a crash from no-op column renames on MySQL.

When attempting to rename a column on MySQL and specifying a “new” name that was the same as the old name, the result would be a crash. Likewise, there were crashes when renaming a ManyToManyField.

Contributors

Christian Hammond

Django Evolution 0.7

Release date: February 3, 2014

Packaging

Fixed the unit tests module being accidentally bundled with the package. (Bug #134)
Fixed the missing NEWS file in the releases. (Bug #130)

Compatibility Changes

Added compatibility with Django 1.5 and 1.6 (Bug #136).
Dropped compatibility for versions of Django prior to 1.4.10.

New Features

Added better support for dealing with indexes in the database.

Django changed how index names were generated over time, leading to issues when evolving old databases. We now scan the database prior to evolution, gather the indexes, and look them up based on field data dynamically, guaranteeing we find the correct index.

It’s also more resilient now when using custom indexes placed by an administrator.
Added support for evolving unique_together and index_together fields.

unique_together was previously stored, but ignored, meaning that changes to a unique_together would not ever apply to an existing database.

index_together, on the other hand, is new in Django 1.5, and was never even stored.

There’s now a ChangeMeta mutation that allows for changing unique_together and index_together.

Models making use of unique_together or index_together will have to supply evolutions defining the current, correct values. These will appear when running evolve --hint.
Optimized the SQL before altering the database.

Mutations are now pre-processed and their output post-processed in order to reduce the number of table-altering mutations. This should massively reduce the amount of time it takes to update a database, particularly when there are multiple AddField, ChangeField, or DeleteField mutations on a single table.

This is the biggest change in this release, and while it’s been tested on some large sets of mutations, there may be regressions. Please report any issues you find.

Custom field mutation classes will need to be updated to work with these changes.

Bug Fixes

Fixed a number of issues with constraints on different databases. (Bug #127)
Fixed an invalid variable reference when loading SQL evolution files. (Bug #121)
SQL evolution files no longer break if there are blank lines. (Bug #111)
Booleans are now normalized correctly when saving in the database. (Bug #125)

Previously, invalid boolean values would be used, causing what should have been a “false” value to be “true”.

Usage

The evolve command no longer recommends running evolve --hint --execute, which can easily cause unwanted problems.

Testing

Added easier unit testing for multiple database types.

The ./tests/runtests.py script now takes a database type as an argument. The tests will be run against that type of database.

To make use of this, copy test_db_settings.py.tmpl to test_db_settings.py and fill in the necessary data.
Fixed all the known unit test failures.
Rewrote the test suite for better reporting and maintainability.

Contributors

Christian Hammond

Django Evolution 0.7 Beta 1

Release date: January 14, 2014

Packaging

Fixed the unit tests module being accidentally bundled with the package. (Bug #134)
Fixed the missing NEWS file in the releases. (Bug #130)

Compatibility Changes

Added compatibility with Django 1.5 (Bug #136).
Dropped compatibility for versions of Django prior to 1.4.10.

New Features

Added better support for dealing with indexes in the database.

Django changed how index names were generated over time, leading to issues when evolving old databases. We now scan the database prior to evolution, gather the indexes, and look them up based on field data dynamically, guaranteeing we find the correct index.

It’s also more resilient now when using custom indexes placed by an administrator.
Added support for evolving unique_together and index_together fields.

unique_together was previously stored, but ignored, meaning that changes to a unique_together would not ever apply to an existing database.

index_together, on the other hand, is new in Django 1.5, and was never even stored.

There’s now a ChangeMeta mutation that allows for changing unique_together and index_together.

Models making use of unique_together or index_together will have to supply evolutions defining the current, correct values. These will appear when running evolve --hint.
Optimized the SQL before altering the database.

Mutations are now pre-processed and their output post-processed in order to reduce the number of table-altering mutations. This should massively reduce the amount of time it takes to update a database, particularly when there are multiple AddField, ChangeField, or DeleteField mutations on a single table.

This is the biggest change in this release, and while it’s been tested on some large sets of mutations, there may be regressions. Please report any issues you find.

Custom field mutation classes will need to be updated to work with these changes.

Bug Fixes

Fixed a number of issues with constraints on different databases. (Bug #127)
Fixed an invalid variable reference when loading SQL evolution files. (Bug #121)
SQL evolution files no longer break if there are blank lines. (Bug #111)
Booleans are now normalized correctly when saving in the database. (Bug #125)

Previously, invalid boolean values would be used, causing what should have been a “false” value to be “true”.

Usage

The evolve command no longer recommends running evolve --hint --execute, which can easily cause unwanted problems.

Testing

Added easier unit testing for multiple database types.

The ./tests/runtests.py script now takes a database type as an argument. The tests will be run against that type of database.

To make use of this, copy test_db_settings.py.tmpl to test_db_settings.py and fill in the necessary data.
Fixed all the known unit test failures.
Rewrote the test suite for better reporting and maintainability.

Contributors

Christian Hammond

0.6 Releases

Django Evolution 0.6.9

Release date: March 13, 2013

Bug Fixes

Django Evolution no longer applies upgrades that match the current state.

When upgrading an old database, where a new model has been introduced and evolutions were added on that model, Django Evolution would try to apply the mutations after creating that baseline, resulting in confusing errors.

Now we only apply mutations for parts of the database that differ between the last stored signature and the new signature. It should fix a number of problems people have hit when upgrading extremely old databases.

Contributors

Christian Hammond

Django Evolution 0.6.8

Release date: February 8, 2013

New Features

Added two new management commands: list-evolutions and wipe-evolution.

list-evolutions lists all applied evolutions. It can take one or more app labels, and will restrict the output to those apps.

wipe-evolution will wipe one or more evolutions from the database. This should only be used if absolutely necessary, and can cause problems. It is useful if there’s some previously applied evolutions getting in the way, which can happen if a person is uncareful with downgrading and upgrading again.

Contributors

Christian Hammond

Django Evolution 0.6.7

Release date: April 12, 2012

Bug Fixes

Don’t fail when an app doesn’t contain any models.

Installing a baseline for apps without models was failing. The code to install a baseline evolution assumed that all installed apps would have models defined, but this wasn’t always true. We now handle this case and just skip over such apps.

Contributors

Christian Hammond

Django Evolution 0.6.6

Release date: April 1, 2012

New Features

Generate more accurate sample evolutions.

The sample evolutions generated with evolve --hint should now properly take into account import paths for third-party database modules. Prior to this, such an evolution had to be modified by hand to work.
Generate PEP-8-compliant sample evolutions.

The evolutions are now generated according to the standards of PEP-8. This mainly influences blank lines around imports and the grouping of imports.
Support Django 1.4’s timezone awareness in the Version model.

The Version model was generating runtime warnings when creating an instance of the model under Django 1.4, due to using a naive (non-timezone-aware) datetime. We now try to use Django’s functionality for this, and fall back on the older methods for older versions of Django.

Contributors

Christian Hammond

Django Evolution 0.6.5

Release date: August 15, 2011

New Features

Added a built-in evolution to remove the Message model in Django 1.4 SVN.

Django 1.4 SVN removes the Message model from django.contrib.auth. This would break evolutions, since there wasn’t an evolution for this. We now install one if we detect that the Message model is gone.

Bug Fixes

Fixed the version association for baseline evolutions for apps.

The new code for installing a baseline evolution for new apps in Django Evolution 0.6.4 was associating the wrong Version model with the Evolution. This doesn’t appear to cause any real-world problems, but it does make it harder to see the proper evolution history in the database.

Contributors

Christian Hammond

Django Evolution 0.6.4

Release date: June 22, 2011

New Features

Install a baseline evolution history for any new apps.

When upgrading an older database using Django Evolution when a new model has been added and subsequent evolutions were made on that model, the upgrade would fail. It would attempt to apply those evolutions on that model, which, being newly created, would already have those new field changes.

Now, like with an initial database, we install a baseline evolution history for any new apps. This will ensure that those evolutions aren’t applied to the models in that app.

Bug Fixes

Fixed compatibility with Django SVN in the unit tests.

In Django SVN r16053, get_model() and get_models() only return installed modules by default. This is calculated in part by a new AppCache.app_labels dictionary, along with an existing AppCache.app_store, neither of which we properly populated.

We now set both of these (though, app_labels only on versions of Django that have it). This allows the unit tests to pass, both with older versions of Django and Django SVN.

Contributors

Christian Hammond

Django Evolution 0.6.3

Release date: May 9, 2011

Bug Fixes

Fixed multi-database support with different database backends.

The multi-database support only worked when the database backends matched. Now it should work with different types. The unit tests have been verified to work now with different types of databases.
Fixed a breaking with PostgreSQL when adding non-null columns with default values. (Bugs #58 and #74)

Adding new columns that are non-null and have a default value would break with PostgreSQL when the table otherwise had data in it. The SQL for adding a column is an ALTER TABLE followed by an UPDATE to set all existing records to have the new default value. PostgreSQL, however, doesn’t allow this within the same transaction.

Now we use two ALTER TABLEs. The first adds the column with a default value, which should affect existing records. The second drops the default. This should ensure that the tables have the data we expect while at the same time keeping the field attributes the same as what Django would generate.

Contributors

Christian Hammond

Django Evolution 0.6.2

Release date: November 19, 2010

New Features

Add compatibility with Django 1.3.

Django 1.3 introduced a change to the Session.expire_date field’s schema, setting db_index to True. This caused Django Evolution to fail during evolution, with no way to provide an evolution file to work around the problem. Django Evolution now handles this by providing the evolution when running with Django 1.3 or higher.

Contributors

Christian Hammond

Django Evolution 0.6.1

Release date: October 25, 2010

Bug Fixes

Fixed compatibility problems with both Django 1.1 and Python 2.4.

Contributors

Christian Hammond

Django Evolution 0.6

Release date: October 24, 2010

New Features

Added support for Django 1.2’s ability to use multiple databases.

This should use the existing routers used in your project. By default, operations will happen on the ‘default’ database. This can be overridden during evolution by passing --database=<dbname> to the evolve command.

Patch by Marc Bee and myself.

Contributors

Christian Hammond
Marc Bee

0.5 Releases

Django Evolution 0.5.1

Release date: October 13, 2010

New Features

Made the evolve management command raise CommandError instead of sys.exit() on failure. This makes it callable from third party software.

Patch by Mike Conley.
Made the evolve functionality available through an evolve() function in the management command, allowing the rest of the command-specific logic to be skipped (such as console output and prompting).

Patch by Mike Conley.

Bug Fixes

Fixed incorrect defaults on SQLite when adding null fields. (Bug #49)

On SQLite, adding a null field without a default value would cause the field name to be the default. This was due to attempting to select the field name from the temporary table, but since the table didn’t exist, the field name itself was being used as the value.

We are now more explicit about the fields being selected and populated. We have two lists, and no longer assume both are identical. We also use NULL columns for temporary table fields unconditionally.

Patch by myself and Chris Beaven.

Contributors

Chris Beaven
Christian Hammond
Mike Conley

Django Evolution 0.5

Release date: May 18, 2010

Initial public release.

Django Evolution Documentation

Django Evolution is a database schema migration tool for projects using the Django web framework. Its job is to help projects make changes to a database’s schema – the structure of the tables and columns and indexes – in the fastest way possible (incurring minimum downtime) and in a way that works across all Django-supported databases.

This is very similar in concept to the built-in migrations support in Django 1.7 and higher. Django Evolution predates both Django’s own migrations, and works alongside it to transition databases taking advantage of the strengths of both migrations and evolutions.

While most will be fine with migrations, there’s a couple reasons why you might find Django Evolution a worthwhile addition to your project:

You’re still stuck on Django 1.6 or earlier and need to make changes to your database.

Django 1.6 is the last version without built-in support for migrations, and there are still codebases out there using it. Django Evolution can help keep upgrades manageable, and make it easier to transition all or part of your codebase to migrations when you finally upgrade.
You’re distributing a self-installable web application, possibly used in large enterprises, where you have no control over when people are going to upgrade.

Django’s migrations assume some level of planning around when changes are made to the schema and when they’re applied to a database. The more changes you make, and the more versions in-between what the user is running and what they upgrade to, the longer the upgrade time.

If a customer is in control of when they upgrade, they might end up with years of migrations that need to be applied.

Migrations apply one-by-one, possibly triggering the rebuild of a table many times during an upgrade. Django Evolution, on the other hand, can apply years worth of evolutions at once, optimized to perform as few table changes as possible. This can take days, hours or even seconds off the upgrade time.

Django Evolution officially supports Django 1.6 through 4.2.

Questions So Far?

How Does It Work?

Let’s Get Started

Reference

Project Versioning Policy

Beginning with 2.0, Django Evolution uses semantic versioning, in major.minor.micro form.

We will bump major any time there is a backwards-incompatible change to:

Evolution definition format
Compatibility with older versions of Django, Python, or databases
The evolve management command’s arguments or behavior
Public Python API

We will bump minor any time there’s a new feature.

We will bump micro any time there’s just bug or packaging fixes.

Module and Class References

Note

Most of the codebase should not be considered stable API, as many parts will change.

The code documented here is a subset of the codebase. Backend database implementations and some internal modules are not included.

Public API

`django_evolution`	Django Evolution version and package information.
`django_evolution.conf`	Configuration for Django Evolution.
`django_evolution.consts`	Constants used throughout Django Evolution.
`django_evolution.deprecation`	Internal support for handling deprecations in Django Evolution.
`django_evolution.errors`	Standard exceptions for Django Evolution.
`django_evolution.evolve`	Main interface for evolving applications.
`django_evolution.evolve.base`	Base classes for evolver-related objects.
`django_evolution.evolve.evolver`	Main Evolver interface for performing evolutions and migrations.
`django_evolution.evolve.evolve_app_task`	Task for evolving an application.
`django_evolution.evolve.purge_app_task`	Task for purging an application.
`django_evolution.models`	Database models for tracking project schema history.
`django_evolution.mutations`	Mutations for models, fields, and applications.
`django_evolution.mutations.add_field`	Mutation that adds a field to a model.
`django_evolution.mutations.base`	Base support for mutations.
`django_evolution.mutations.change_field`	Mutation that changes attributes on a field.
`django_evolution.mutations.change_meta`	Mutation that changes meta properties on a model.
`django_evolution.mutations.delete_application`	Mutation that deletes an application.
`django_evolution.mutations.delete_field`	Mutation for deleting fields from a model.
`django_evolution.mutations.delete_model`	Mutation that deletes a model.
`django_evolution.mutations.move_to_django_migrations`	Mutation that moves an app to Django migrations.
`django_evolution.mutations.rename_app_label`	Mutation that renames the app label for an application.
`django_evolution.mutations.rename_field`	Mutation that renames a field on a model.
`django_evolution.mutations.rename_model`	Mutation that renames a model.
`django_evolution.mutations.sql_mutation`	Mutation for executing SQL statements.
`django_evolution.serialization`	Serialization and deserialization.
`django_evolution.signals`	Signals for monitoring the evolution process.
`django_evolution.signature`	Classes for working with stored evolution state signatures.

Private API

`django_evolution.diff`	Support for diffing project signatures.
`django_evolution.mock_models`	Utilities for building mock database models and fields.
`django_evolution.mutators`	Mutators responsible for applying mutations.
`django_evolution.mutators.app_mutator`	Mutator that applies changes to an app.
`django_evolution.mutators.model_mutator`	Mutator that applies changes to a model.
`django_evolution.mutators.sql_mutator`	Mutator that applies arbitrary SQL to the database.
`django_evolution.placeholders`	Placeholder objects for hinted evolutions.
`django_evolution.support`	Constants indicating available Django features.
`django_evolution.compat.apps`	Compatibility functions for the application registration.
`django_evolution.compat.commands`	Compatibility module for management commands.
`django_evolution.compat.datastructures`	Compatibility imports for data structures.
`django_evolution.compat.db`	Compatibility functions for database-related operations.
`django_evolution.compat.models`	Compatibility functions for model-related operations.
`django_evolution.compat.picklers`	Picklers for working with serialized data.
`django_evolution.compat.py23`	Compatibility functions for Python 2 and 3.
`django_evolution.db.common`	Common evolution operations backend for databases.
`django_evolution.db.mysql`	Evolution operations backend for MySQL/MariaDB.
`django_evolution.db.postgresql`	Evolution operations backend for Postgres.
`django_evolution.db.sql_result`	Classes for storing SQL statements and Alter Table operations.
`django_evolution.db.sqlite3`	Evolution operations backend for SQLite.
`django_evolution.db.state`	Database state tracking for in-progress evolutions.
`django_evolution.utils.apps`	Utilities for working with apps.
`django_evolution.utils.datastructures`	Utilities for working with data structures.
`django_evolution.utils.evolutions`	Utilities for working with evolutions and mutations.
`django_evolution.utils.graph`	Dependency graphs for tracking and ordering evolutions and migrations.
`django_evolution.utils.migrations`	Utility functions for working with Django Migrations.
`django_evolution.utils.models`	Utilities for working with models.
`django_evolution.utils.sql`	Utilities for working with SQL statements.