Caching in Django absolutely rocks. Much so because of its simplicity which lowers the threshold to the height of a snowflake. However the simplicity can cause some unexpected surprises.
I ran my site and clicked around until I found a bug I wanted to fix. Then I wrote the test and ran the testrunner but I was getting some really weird behavior. The view looked like this:
@never_cache
def club_page(request, clubname):
club = _find_club(clubname)
if club is None:
raise Http404('Could not find the club')
classes = cache.get('club_page_classes')
if classes is None:
classes = ClubClass.objects.filter(club=club).order_by('start_time')
cache.set('club_page_classes', classes, CACHE_TIMEOUT)
...What happened (and what took me a while to figure out) was that the memcache was still active and being used when running the tests since it's only within the hour that I started running the tests with completely different data. Here's how I solved the problem:
class ViewsTestCase(unittest.TestCase):
...
def test_club_page(self):
""" test rendering the club page """
cache.delete('club_page_classes')
...There must be an easier way to patch the testrunner to reset all cache before running the tests. Anybody?
Comments
Post your own comment/etc/init.d/memcached restart
Why not use locmem backend for tests?
Unlike memcache, is locmem reset when you start django with ./manage.py?
Yes its reset, actually its reset whenever the code reloads as well.
I usually do something like this in settings.py
if DEBUG :
CACHE_BACKEND = 'locmem:///'
else :
CACHE_BACKEND = 'memcached://127.0.0.1:11211'
That's actually a pretty good idea because I want the cache to be reset every time the dev server is restarted. Thanks.
You can monkey patch your cache settings in your test module. Stick this on the top of your test module:
from django.conf import settings
settings.CACHE_BACKEND = 'dummy:///'
A better solution is to have a dev settings module that has no cache and a production settings module that does have cache.
When it comes to caching I like abstracting the "if whatever is None" to a decorator so that it can be reused.
For instance that decorator could be like this: http://www.djangosnippets.org/snippets/1237/
That way you don't have to worry about the cache being a factor in your tests when it's disabled because you can set up a unittest to test the decorator in isolation and can assume it's functioning correctly in any dependent code.
Didn't know that was even possible. I'll try that.
I think I want the cache to work in the tests. Otherwise they wouldn't test the cache.
Best would be if I could delete everything from the cache before the tests begin.
Yeah that's why I like to abstract the cache functionality out of the function. That way you can test the functionality without caching and test the caching mechanism in two seperate tests.
That way you can reuse the decorator and rest assured that any function that uses it will cache correctly.
The code above was a simple example, it won't work with dynamic cache keys but I'm sure you could figure out a more complex decorator if you need it.
All good stuff. I'm going to use your little decorator tip too to clean up my code and use locmem (as Stas Shtin pointed out) as the backend. That way cache does come into play in the tests too but reset each time I start the testrunner.
Local memory cache exists only in memory space of python process, so when you terminate it old data is gone.