This took me by surprise today!
If you run this unit test, it actually passes with flying colors:
import unittest
class BadAssError(TypeError):
pass
def foo():
raise BadAssError("d'oh")
class Test(unittest.TestCase):
def test(self):
self.assertRaises(BadAssError, foo)
self.assertRaises(TypeError, foo)
self.assertRaises(Exception, foo)
if __name__ == '__main__':
unittest.main()
Basically, assertRaises doesn't just take the exception that is being raised and accepts it, it also takes any of the raised exceptions' parents.
I've only tested it with Python 2.6 and 2.7. And the same works equally with unittest2.
I don't really know how I feel about this. It did surprise me when I was changing one of the exceptions and expected the old tests to break but they didn't. I mean, if I want to write a test that really makes sure the exception really is BadAssError it means I can't use assertRaises().
Comments
Post your own commentA BadAssError IS A TypeError and also IS A Exception. There is no problem there.
Your tests were probably using a parent to check against an exception instead of the specific type.
For instance, if you change foo() to raise TypeError, the first assertRaises will fail.
It directly parallels the behavior of "except". If an exception would be handled by "except Foo", then "assertRaises(Foo, ...)" will pass. If you changed your exceptions and your unit tests kept passing, consider what that will mean for *application code* already written to handle certain exceptions. Did you want the different exception type to continue to be handled by existing code? Or did you want it to start bypassing certain "except" suites?
I think Python's idea of respecting inheritance hierarchies in the exception handling system itself is questionable, but given that's what we have, this behavior seems to make sense in `assertRaises`. Also, note that `assertRaises` will give you exact exception object if you want it, so you can make additional assertions about it - such as its exact type.
How is this any different than a test like
def testClass(self):
x = ValueError(example)
self.assertTrue(isinstance(x, StandardError))
Why should checking class membership have different semantics for assertRaises than it does for isinstance?
If you really need to validate that the exact class is being raised, and not a subclass (which you shouldn't ever need to do, since try...except clauses will still catch the subclass), you can use the context manager behavior of assertRaises in unittest2 or Python 2.7
with self.assertRaises(BadAssError) as cm:
foo()
self.assertEqual(cm.exception.__class__, BadAssError)
Why? In the same sense that `assertEqual` does an `==` not an `isintance`.
My point is that it's a bit surprising. I'm not pointing out that it's a bug.
I am not sure what you mean. If you want to make sure that the exception “really is” BadAssError, then you test whether assertRaises(BadAssError,…) and you get a real, accurate answer about whether the exception raised will match the pattern “except BadAssError…” in your users' code. It is true that the check assertRaises(BadAssError,…) will *also* succeed if the exception raised is a subclass of BadAssError — but if you were afraid of *that*, then you simply wouldn't create a child-class exception of BadAssError, right?
I know it's a very "silly" issue. I raised this (no pun intended) because I refactored my code to use more explicit exception classes but got baffled that my tests continued to pass even though I was messing around in the code.
I don't see what the problem is: a BadAssError is a TypeError is an Exception, so it makes perfect sense that raising a BadAssError and catching a TypeError will pass: that's exactly what a standard `except` will do.
> I mean, if I want to write a test that really makes sure the exception really is BadAssError
The only case where this could be an issue is if your BadAssError was subtyped (so you had a BadAsserError(BadAssError)) and that subtype was raised and you wanted the base and not the subtype.
The chances of exactly this occuring are pretty low, and in that case you should feel free to use a standard try/except and asserting that `type(e) is BadAssError` (if that snipped does not make you squirm, there might be something wrong with you)
I'm not surprised by this behavior, but sometimes you do want to check the exact error class. When I changed a function in PyMongo to raise a different error in the same hiearchy, I wrote assertRaisesExactly as an alternative to assertRaises:
https://github.com/mongodb/mongo-python-driver/blob/master/test/utils.py#L62
Thank you! Clearly I'm not alone in being anal about testing *exactly* which exception is raised.
For a enlightening metaphor, imagine that in your example, "Exception" is "animal", "TypeError" is "dog", and "BadAssError" is "poodle". You have some function that returns a poodle, which naturally must make the answer to those questions "Yes":
* Did the function return a poodle?
* Did the function return a dog?
* Did the function return an animal?
If you want to test for dog, regardless of breed, test for dog. If you want to test for poodle, and beagle or labrador won't do, test for poodle. But why would you want to test for dog and fail if it is a poodle? If you want a test that would pass for any dog except a poodle, then do two tests:
* assert it is a dog
* assert it is not a poodle
If you were always testing for dogs and suddenly you realize some functions should return a general "dog", but some others should return specifically "poodle" or "labrador", there is no reason why all old tests should not still pass. All functions are still returning dogs. If you want to test some of them for poodle, you can make that specific test for those specific functions.