Mocking dates with Django
It's not as simple as it at first appears - some tips from the frontline.
A lot of what we do at YJ is date-dependent, and amongst our unit tests we have (as I now know) 500+ tests that rely in some sense on the value of python's datetime.date.today()
function.
Since launching YJ we have been running on Django 1.4, and mocking out datetime.date using the technique outlined in this article:
from datetime import date as real_date
import datetime
class FakeDate(datetime.date):
"Mock out the today method, but return a real date instance."
def __new__(cls, *args, **kwargs):
return real_date.__new__(real_date, *args, **kwargs)
@classmethod
def today(cls):
return cls(2000, 1, 1)
Everything worked just fine, until we started preparing to upgrade to Django 1.5+, at which point nearly all of the mocked date tests started to fail.
I've written up the details on this in a StackOverflow question, but the long and short of it is this: if you use Django model DateField you will probably have an issue with mock dates. This is because deep in the django model field innards there is a to_python()
method which attempts to convert whatever value the field currently has to the appropriate date type. The type check that is used within this method is isinstance(value, datetime.date)
, which is where things get complicated.
If you look at that simple statement, and are using mock dates, then there are four possible combinations of value and datetime.date that may occur:
value
is a real date instance,datetime.date
is a real date classvalue
is a mock date instance,datetime.date
is a real date classvalue
is a mock date instance,datetime.date
is a mock date classvalue
is a real date instance,datetime.date
is a mock date class
Of these, 1 and 3 are obvious (isinstance will return True), and as our FakeDate
class inherits from datetime.date
, it's true to say that a FakeDate is an instance of a datetime.date, and so 2 will return True. This is not the case in reverse (a datetime.date is not an instance of a FakeDate), and so 4 will return False.
This arose as a problem in 1.5 as in 1.4 and below, the to_python()
method included logic that said "if this value is neither a date [mocked or otherwise] nor a datetime, then try casting it to a string, and then try parsing that string as a date". Because our mock date class returned a valid date format from the str() method, this code would take a date, format it as a string, and then re-parse it back into a (real) date. Ugly, but effective.
1.5+ this was changed to "if this value is neither a date [mocked or otherwise] nor a datetime, assume it's a string and try parsing that string as a date", effectively unmasking the issue that we had with tests failing. See the StackOverflow question (and the related Django issue, #25213) for more details.
So, how do you get a real date as the value, when you are mocking out datetime.date? Well, for a start, if you are using the FakeDate class referenced at the beginning of this post, then its core design principal is that its constructor should return a real date, rather than a FakeDate - that's the point. So if you are using this method, you will find yourself in this situation.
I tried using a simpler mock date, that inherited from datetime.date, but did not include the clever plumbing to return a real date from its constructor:
>>> import datetime
>>> class FakeDate(datetime.date):
@classmethod
def today(cls):
return cls(2000, 1, 1)
>>> import mock
>>> with mock.patch('datetime.date', FakeDate):
print datetime.date.today()
print type(datetime.date.today())
isinstance(FakeDate.today(), datetime.date)
2000-01-01
<type 'FakeDate'>
true
This gets around the initial problem - in that a call to datetime.date.today()
will now return a FakeDate, as will type(datetime.date)
, so it initially looked like a solution.
However, after running more tests it appeared as if there were real dates slipping through the net - landing us back in the same situation. How was this possible? There are several core python library methods that create dates, including adding (or subtracting) datetime.timedelta
objects from existing dates (mocked or otherwise), and the use of datetime.datetime.date()
. This means that if you have code somewhere that assigns a date value using, for instance, datetime.datetime.now().date()
, then the object you get back will be a real date, and not the mock.At which point you will find yourself back in option 4, with the isinstance check failing, and your tests blowing up.
My initial solution to this situation was to go down the datetime rabbithole, and to mock out everything that could generate a date (the date class operations (add, radd, etc.)), but that way madness lies, and I was saved in the nick of time by someone (aaugustin , a django core dev) on the Django issue, who pointed me in the direction of the __instancecheck__
method. This proved to be a life saver, with the resulting class looking like this:
import datetime
from datetime import date as real_date
class FakeDate(datetime.date):
"""A fake replacement for datetime.date."""
class FakeDateType(type):
"Used to ensure the FakeDate returns True to function calls."
def __instancecheck__(self, instance):
return isinstance(instance, real_date)
# this forces the FakeDate to return True to the isinstance date check
__metaclass__ = FakeDateType
This is all very confusing, and so to help out anyone who has the same issue, I've put a gist together that shows three possible variations of a mock date, along with tests that demonstrate the output.
Making Freelance Work
Posted in: django