Upgrading the framework your entire website relies on is a hard job, but someone has to do it. It was my turn here at YJ: migrating from Django 1.7.8 to 1.8.2.

Here's the list of things that broke on the way. Many of them are documented in the release notes, some of them are not: they're probably quite niche and specific to our implementation. It might well be the case many of you outside there stumble on the same issues.

Here follows the list of (almost) all the issues I found after running pip install Django==1.8.2 and manage.py test.

settings.TEMPLATE_CONTEXT_PROCESSORS

If you override TEMPLATE_CONTEXT_PROCESSORS in your settings, it's very probably a tuple looking like this:

TEMPLATE_CONTEXT_PROCESSORS = (
    'django.contrib.auth.context_processors.auth',
    'django.contrib.auth.context_processors.auth',
    'django.core.context_processors.debug',
    'django.core.context_processors.i18n',
    'django.core.context_processors.media',
    'django.core.context_processors.static',
    'django.core.context_processors.request',
    'django.core.context_processors.tz',
    'django.contrib.messages.context_processors.messages',
    # and even more ...
)

The django.core.context_processors path has changed in Django 1.8: it's now django.template.context_processors. So, you'd better update that in your settings or your responses will miss quite a lot of context in the best case:

TEMPLATE_CONTEXT_PROCESSORS = (
    'django.contrib.auth.context_processors.auth',
    'django.template.context_processors.debug',
    'django.template.context_processors.i18n',
    'django.template.context_processors.media',
    'django.template.context_processors.static',
    'django.template.context_processors.request',
    'django.template.context_processors.tz',
    'django.contrib.messages.context_processors.messages',
    # etc. etc.
)

Third party apps migrations

Up to 1.7, Django was tolerating missing migrations in third party apps. Since 1.8: no migrations, no party. If an external app with models is unmigrated, running manage.py migrate in the main project won't work out all the migrations in the right order. I read that the order of migrations should be the one given in INSTALLED_APPS, but it wasn't our case.

We use (at least) two apps which models point a ForeighKey to auth.User. Those not having a migrations directory, Django failed in initialising the test database, due to this error: django.db.utils.ProgrammingError: relation "auth_user" does not exist: auth_user was referenced somewhere before it had been created.

Luckily, the unmigrated external apps were both in our control, so I closed the issue providing them with migrations.

To do that, I had to check out into an app repo, create a test project that includes the app, and call manage.py makemigrations from its root.

There's a good post from Twilio that explains the steps to take. On a little side note: it would be nice to have a command to create migrations for an app without need to create a test project around it.

Once the external apps have their migrations directory, it's just a matter of update their version in project requirements and run the test suite again. This time, the db will be well initialised and migrated.

RequestFactory and assertTemplateUsed

This is a peculiar one, and it was probably due to the fact we were using RequestFactory improperly in some places.

We had tests like:

def test_something(self):
    url = reverse('my_view_name')
    request = RequestFactory().get(url)
    response = my_view_foo(request)
    # test response this and that... 
    self.assertTemplateUsed(response, 'my_template.html')

That was raising:

assertTemplateUsed() and assertTemplateNotUsed() are only usable on responses fetched using the Django test Client.

, which is a pretty clear message, but it isn't documented anywhere. These tests were passing in 1.7, so I assume RequestFactory was keeping track of context and templates used in serving the response. It no longer works in Django 1.8. After bumping my head against my desk a couple of times, I came up with an idea, and it did work: replace self.assertTemplateUsed(response, 'my_template.html') calls with a good old context manager: with self.assertTemplateUsed('my_template.html'). So that the code above would be refactored in:

def test_something(self):
    url = reverse('my_view_name')
    request = RequestFactory().get(url)
    with self.assertTemplateUsed(template_name='my_template.html'):
        response = my_view_foo(request)
        # test this and that...

Session Backend User ID

_auth_user_id is stored as a string, not as a number, in Session objects.
We have a function that logs out a user removing their key from Session. It used to look like this:

def clear_user_session(user):
    """Clear out sessions for a given user."""
    for s in Session.objects.all():
        if s.get_decoded().get(settings.SESSION_KEY) == user.id:
            s.delete()

Turns out django.contrib.auth.login now stores the user_id as a string, so that function isn't functioning anymore for our purposes.

I gave a look at Django codebase, and found out this. So the trick was replacing the function above with:

def _clear_user_session(user):
    for s in Session.objects.all():
        _id = s.get_decoded().get(settings.SESSION_KEY, None)
        if _id == user._meta.pk.value_to_string(user):
            s.delete()

Woohoo – our users can be now be logged out when we need to, even using Django 1.8.

Assigning unsaved objects to relations raises an error

This is well documented here and won't linger over the subject too much. Long story short: in many tests, we were passing unsaved objects to others as foreign key. This was quite handy because it spared us a few DB inserts. We're talking milliseconds here. The solution was just a matter of adding a bunch of save() lines here and there.

If you really don't want to save those instances, then use allow_unsaved_instance_assignment at your own risk.

Management commands and the optparse deprecation

Django (just like Python) is discouraging the use of optparse and strongly recommends argparse instead. If all of your management commands are using optparse, don't despair: it's still supported in 1.8. In our codebase, the problem was we were importing make_option from Django: from django.core.management import make_options. Just Replace it with from optparse import make_options and you'll be alright (until Django will drop support to optparse, in 2.0).

ImageField validation

File formats are early validated now, and the list of valid formats is the keys contained in PIL.Image.MIME. A default error message will be given when you try to upload a BMP or a PDF to a ImageField (GIF, JPG, TIFF, PNG are still valid). The code change is here.

 Final thoughts

• Was it worth migrating? It is always worth, especially when you can get new features like this, this, or this.
• Was it a pain? A little bit: it could be worse.

Hope this will help you Djangonauts out there with your migration.