run migrations on test database if --reuse-db is set

[trbs]

Run migrations on the test database(s) when --reuse-db options is enabled and the test database already exists.

This helps people with large (and slow) migration sets which otherwise would have to drop the test database or manually run the migrations on it.

[alvinchow86]

FYI there's a --nomigrations option now which will skip migrations and build a test database from scratch fairly quickly if you are doing --create-db

[trbs]

@alvinchow86 Yeah I use that (in other projects) as well.

In this project however I also got data migrations which are needed for the tests to succeed.
Also with a large number of migrations Django 1.7 becomes very slow. Django 1.8 improves on this a bit but is still not fast enough (last time I tried) to run through all the migrations on each test run.

[blueyed]

Thanks for addressing this.

My pet peeve is data migrations.

Skipping migrations is problematic with data migrations, when this data is being used in tests.
For example: https://github.com/cordery/django-countries-plus/blob/master/countries_plus/migrations/0002_auto_20150120_2225.py

I've thought that this could be handled better by leaning more towards Django's internal methods, which I've started in #203. What do you think about that?

As far as I can see, your method won't help with data migrations (when transactional_db is involved, and the db is flushed): the apps are being considered to be fully migrated already.

A workaround is to use the post_migrate signal for your data migrations:

# app/management/__init__.py
# Handle initial data in Django 1.7+.

from django.db.models import signals
from django.core.management import call_command

def load_initial_data(app_config, sender, **kwargs):
    if sender.label == "movies":
        call_command('loaddata', 'initial_data.yaml', app_label='movies')

signals.post_migrate.connect(load_initial_data, weak=False)

In this context it's also noteworthy that Django runs its tests in a particular order. I've started working to handle this in the same way for pytest-django, but it won't solve the data migration issue probably either (alone).

[blueyed]

Some more notes: using serialized_rollback might help, but is very slow (with lots of data).
The relevant Django ticket: https://code.djangoproject.com/ticket/22487

[trbs]

@blueyed your probably right.

Does Django automatically rollup the migrations in your PR ? (without rerunning them ?)

[blueyed]

@trbs
No, #203 (now merged) does not help with (data) migrations nor will it run migrate automatically.

It is being discussed to add a warning / and/or abort in this case: https://code.djangoproject.com/ticket/24484

I think this PR is good as-is, but I'd like to have some tests for this, especially also in regard to the output that is being generated for different verbosity levels.

[landscape-bot]

Code quality remained the same when pulling 4b76b24 on trbs:master into 1e72766 on pytest-dev:master.

[blueyed]

This PR itself (running migrate) would be covered by #261 also.

The issue about keeping your data from the migrations seems still not be solvable in a straight-forward way.. :/

[blueyed]

The command in Django 1.8 looks a bit different: https://github.com/django/django/blob/stable/1.8.x/django/db/backends/base/creation.py#L360-L369:

    call_command(
        'migrate',
        verbosity=max(verbosity - 1, 0),
        interactive=False,
        database=self.connection.alias,
        test_flush=not keepdb,
    )

What do you think about using #261 (Django's --keepdb), which would cover Django 1.8+, and then this for Django 1.7.x?

[blueyed]

I think this should be covered by 0ec741b since a while, where we're using Django's method for 1.5+ now.

Please re-open in case I'm wrong.