Need help with django-test-migrations?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

wemake-services
253 Stars 23 Forks MIT License 254 Commits 4 Opened issues

Description

Test django schema and data migrations, including migrations' order and best practices.

Services available

!
?

Need anything else?

Contributors list

django-test-migrations

wemake.services Build status codecov Python Version PyPI - Django Version wemake-python-styleguide

Features

  • Allows to test
    django
    schema and data migrations
  • Allows to test both forward and rollback migrations
  • Allows to test the migrations order
  • Allows to test migration names
  • Allows to test database configuration
  • Fully typed with annotations and checked with
    mypy
    , PEP561 compatible
  • Easy to start: has lots of docs, tests, and tutorials

Read the announcing post. See real-world usage example.

Installation

pip install django-test-migrations

We support several

django
versions:
  • 1.11
  • 2.2
  • 3.1
  • 3.2

Other versions might work too, but they are not officially supported.

Testing django migrations

Testing migrations is not a frequent thing in

django
land. But, sometimes it is totally required. When?

When we do complex schema or data changes and what to be sure that existing data won't be corrupted. We might also want to be sure that all migrations can be safely rolled back. And as a final touch we want to be sure that migrations are in the correct order and have correct dependencies.

Testing forward migrations

To test all migrations we have a

Migrator
class.

It has three methods to work with:

  • .apply_initial_migration()
    which takes app and migration names to generate a state before the actual migration happens. It creates the
    before state
    by applying all migrations up to and including the ones passed as an argument.
  • .apply_tested_migration()
    which takes app and migration names to perform the actual migration
  • .reset()
    to clean everything up after we are done with testing

So, here's an example:

from django_test_migrations.migrator import Migrator

migrator = Migrator(database='default')

Initial migration, currently our model has only a single string field:

Note:

We are testing migration 0002_someitem_is_clean, so we are specifying

the name of the previous migration (0001_initial) in the

.apply_initial_migration() method in order to prepare a state of the database

before applying the migration we are going to test.

old_state = migrator.apply_initial_migration(('main_app', '0001_initial')) SomeItem = old_state.apps.get_model('main_app', 'SomeItem')

Let's create a model with just a single field specified:

SomeItem.objects.create(string_field='a') assert len(SomeItem._meta.get_fields()) == 2 # id + string_field

Now this migration will add is_clean field to the model:

new_state = migrator.apply_tested_migration( ('main_app', '0002_someitem_is_clean'), ) SomeItem = new_state.apps.get_model('main_app', 'SomeItem')

We can now test how our migration worked, new field is there:

assert SomeItem.objects.filter(is_clean=True).count() == 0 assert len(SomeItem._meta.get_fields()) == 3 # id + string_field + is_clean

Cleanup:

migrator.reset()

That was an example of a forward migration.

Backward migration

The thing is that you can also test backward migrations. Nothing really changes except migration names that you pass and your logic:

migrator = Migrator()

Currently our model has two field, but we need a rollback:

old_state = migrator.apply_initial_migration( ('main_app', '0002_someitem_is_clean'), ) SomeItem = old_state.apps.get_model('main_app', 'SomeItem')

Create some data to illustrate your cases:

...

Now this migration will drop is_clean field:

new_state = migrator.apply_tested_migration(('main_app', '0001_initial'))

Assert the results:

...

Cleanup:

migrator.reset()

Testing migrations ordering

Sometimes we also want to be sure that our migrations are in the correct order. And all our

dependecies = [...]
are correct.

To achieve that we have

plan.py
module.

That's how it can be used:

from django_test_migrations.plan import all_migrations, nodes_to_tuples

main_migrations = all_migrations('default', ['main_app', 'other_app']) assert nodes_to_tuples(main_migrations) == [ ('main_app', '0001_initial'), ('main_app', '0002_someitem_is_clean'), ('other_app', '0001_initial'), ('main_app', '0003_update_is_clean'), ('main_app', '0004_auto_20191119_2125'), ('other_app', '0002_auto_20191120_2230'), ]

This way you can be sure that migrations and apps that depend on each other will be executed in the correct order.

Test framework integrations 🐍

We support several test frameworks as first-class citizens. That's a testing tool after all!

Note that the Django

post_migrate
signal's receiver list is cleared at the start of tests and restored afterwards. If you need to test your own
post_migrate
signals then attach/remove them during a test.

pytest

We ship

django-test-migrations
with a
pytest
plugin that provides two convinient fixtures:
  • migrator_factory
    that gives you an opportunity to create
    Migrator
    classes for any database
  • migrator
    instance for the
    'default'
    database

That's how it can be used:

import pytest

@pytest.mark.django_db() def test_pytest_plugin_initial(migrator): """Ensures that the initial migration works.""" old_state = migrator.apply_initial_migration(('main_app', None))

with pytest.raises(LookupError):
    # Models does not yet exist:
    old_state.apps.get_model('main_app', 'SomeItem')

new_state = migrator.apply_tested_migration(('main_app', '0001_initial'))
# After the initial migration is done, we can use the model state:
SomeItem = new_state.apps.get_model('main_app', 'SomeItem')
assert SomeItem.objects.filter(string_field='').count() == 0

unittest

We also ship an integration with the built-in

unittest
framework.

Here's how it can be used:

from django_test_migrations.contrib.unittest_case import MigratorTestCase

class TestDirectMigration(MigratorTestCase): """This class is used to test direct migrations."""

migrate_from = ('main_app', '0002_someitem_is_clean')
migrate_to = ('main_app', '0003_update_is_clean')

def prepare(self):
    """Prepare some data before the migration."""
    SomeItem = self.old_state.apps.get_model('main_app', 'SomeItem')
    SomeItem.objects.create(string_field='a')
    SomeItem.objects.create(string_field='a b')

def test_migration_main0003(self):
    """Run the test itself."""
    SomeItem = self.new_state.apps.get_model('main_app', 'SomeItem')

    assert SomeItem.objects.count() == 2
    assert SomeItem.objects.filter(is_clean=True).count() == 1

Choosing only migrations tests

In CI systems it is important to get instant feedback. Running tests that apply database migration can slow down tests execution, so it is often a good idea to run standard, fast, regular unit tests without migrations in parallel with slower migrations tests.

pytest

django_test_migrations
adds
migration_test
marker to each test using
migrator_factory
or
migrator
fixture. To run only migrations test, use
-m
option:
pytest -m migration_test  # runs only migraion tests
pytest -m "not migration_test"  # runs all except migraion tests

unittest

django_test_migrations
adds
migration_test
tag to every
MigratorTestCase
subclass. To run only migrations tests, use
--tag
option:
python mange.py test --tag=migration_test  # runs only migraion tests
python mange.py test --exclude-tag=migration_test  # runs all except migraion tests

Django Checks

django_test_migrations
comes with 2 groups of Django's checks for:
  • detecting migrations scripts automatically generated names
  • validating some subset of database settings

Testing migration names

django
generates migration names for you when you run
makemigrations
. And these names are bad (read more about why it is bad)! Just look at this:
0004_auto_20191119_2125.py

What does this migration do? What changes does it have?

One can also pass

--name
attribute when creating migrations, but it is easy to forget.

We offer an automated solution:

django
check that produces an error for each badly named migration.

Add our check into your

INSTALLED_APPS
:
INSTALLED_APPS = [
    # ...

# Our custom check:
'django_test_migrations.contrib.django_checks.AutoNames',

]

And then in your CI run:

python manage.py check --deploy

This way you will be safe from wrong names in your migrations.

Do you have a migrations that cannot be renamed? Add them to the ignore list:

# settings.py

DTM_IGNORED_MIGRATIONS = { ('main_app', '0004_auto_20191119_2125'), ('dependency_app', '0001_auto_20201110_2100'), }

And we won't complain about them.

Or you can completely ignore entire app:

# settings.py

DTM_IGNORED_MIGRATIONS = { ('dependency_app', ''), ('another_dependency_app', ''), }

Database configuration

Add our check to

INSTALLED_APPS
:
INSTALLED_APPS = [
    # ...

# Our custom check:
'django_test_migrations.contrib.django_checks.DatabaseConfiguration',

]

Then just run

check
management command in your CI like listed in section above.

Related projects

You might also like:

  • django-migration-linter - Detect backward incompatible migrations for your django project.
  • wemake-django-template - Bleeding edge django template focused on code quality and security with both
    django-test-migrations
    and
    django-migration-linter
    on board.

Credits

This project is based on work of other awesome people:

License

MIT.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.