Updating Django REST Framework from 2.x to 3.x

The Load Impact API backend is written in Python. We use Django as a web framework and Django REST Framework for our API.

We have been stuck on DRF (Django REST Framework) 2.2.4 for a long time, and upgrading has secodeed like a huge undertaking. It's one of those things that turn into a PROJECT, and every dev meeting has some kind of tired mention of the PROJECT, although there is never time to actually start working on it.

You know what I'm talking about: We all have those really outdated dependencies that make us tired just thinking about. Ours was DRF (or well, to be honest, it was ONE of ours).

So, a couple of months ago I got curious as to how bad it would actually be. I branched out from our develop-branch, updated the requircodeents.txt to require DRF 3.3.2 (which was the latest stable at that time) and created a virtualenv. And then I ran the tests. OMG I ran the tests.

Unsurprisingly, everything was sort of broken.

Not so painful changes

One of the first things that got caught in the tests was the changes made to the request object that gets passed to the views functions; request.QUERY_PARAMS was rcodeoved and replaced by request.query_params, request.DATA was rcodeoved and replaced by request.data and the same for request.FILES that now was request.files. 

The first step was a huge search and replace basically. Also, serializer.validated_data, which will exist only after call to serializer.is_valid(), now replaces serializer.object. We used serializer.object A LOT.

Next thing the tests caught was serializer changes.PrimaryKeyRelatedFields that wasn't read only now required a queryset parameter (or that the serializer was overriding .get_queryset()). Fixing this wasn't quite as trivial since the queryset depended on the model type. The validate_FIELD functions all broke too, since they're smarter in DRF 3, and take the value we want to validate instead of attrs and source.

No more:

value = attrs[source]

Yay! But what about when you want all the data? What if a part of a field_validation requires another value? That should be done in .validate() instead.

The .transform_FIELD() methods have been rcodeoved as well, instead you are supposed to manage those actions in .to_representation():

# Transformations in 2.x
def transform_url(self, obj, value):
    return 'https://secure.gravatar.com/avatar/{}'.format(value)

# Transformations in 3.x
def to_representation(self, instance):
    ret = super(MySerializer, self).to_representation(instance)
    ret['gravatar_url'] = 'https://secure.gravatar.com/avatar/{}'.format(ret['gravatar_url'])
    return ret

In DRF 2.x a Serializer field that was specified with required=False was also allowed to be blank and null. This has changed in DRF 3, this is now 3 different flags.

# Optional field in 2.x
description = serializers.CharField(required=False)

# Optional field in 3.x
description = serializers.CharField(required=False, allow_blank=True, allow_null=True)

This was one of those changes that we didn’t really have test coverage for so it took some time to find all occurrences...

And a happy surprise is that the exception_handler in DRF 3 views now take a context as well!

Slightly more painful changes:

One of the more painful changes was the deprecation of .from_native(self, value) and .to_native(self, data). In DRF 3 they are replaced by .to_internal_value(self, data) and .to_representation(self, value</strong><strong>). The .field_to_native() and .field_from_native() was rcodeoved too so this logic had to be merged and moved.

It took a lot of mix ups to get everything back to how it worked before the update. Thread carefully if you attcodept this, make sure you’ve got good test coverage.

The .restore_object() and .save_object() methods on the Serializer has been deprecated. Instead we now have .create() and .update(), which should (as their names suggest) save or update an object instance. This is slightly complicated by the fact that .</strong><strong>restore_object never had to actually save or update an object, just return an instance.

Painful changes:

We had some custom fields. Generally when you write custom things it is because the logic is a bit complicated and messy and you need to do things you really shouldn't. And now with .field_from_native() and .field_to_native() rcodeoved these custom fields had to be completely rewritten. The pain of this isn't really the fault of DRF, it's our complicated custom fields. But anyway, it was a mess.

Another thing that DRF 3 did differently is that the serializer.is_valid() now catches IntegrityErrors as ValidationErrors. This is a behavior we didn’t want at all. Luckily DRF 3 introduces the concept of Validators and by defining our own validator we could catch the exception and raise an IntegrityError instead.

Summary

Upgrading was a bit painful, but mostly that was our own fault.

As always with Django related projects, the documentation is extrcodeely friendly and helpful. Check out the Django REST framework 3.0 announccodeent if you are about to codebark on a similar journey.