Assert is the wrong word for tests

Consider the following pseudocode test:


assert person.getAdress is not null

A slightly better test would like something like this:


address1 = new Address
assert address.getZipCode is null
address2 = person.getAddress
assert address2.getZipCode is not null

We don’t want our test to fail if there’s something wrong with a constructor or simple composition.

“Assert” is a bad word to use for tests. An assertion is a sanity check, an assumption that everything is ok and you can proceed with execution. When you write meaningful tests, there are likely several assertions that are doing just that, checking the sanity of your test fixture before you get to your actual, meaningful test assertion.

We should use a different word for assertions and tests, because frankly, we (as testers) don’t care about assertion failures, they’re too hard to track down when we’re really meaning to be testing something else. They’re useful for debugging later, but they don’t accurately tell us which test actually failed.

You could use another key word like verify that wraps an assert (Selenium does this) and trap all other assertions so that a test really only fails if what you’re meaning to test fails. Everything else is an assertion error. it’s be nice if frameworks had this distinction, but it’s still a challenge for the forseeable future.

In C++ assertions cause compile errors, because they’re meant to show that it makes no sense for the code to continue execution if an assertion fails. I know it’s common to use an #ifdef hack to ignore assertions in production, but that’s not (or shouldn’t be) because they were using assert for testing (they often did before xunit style test frameworks existed) but because an assertion failure can throw out ugly errors, sometimes with sensitive or proprietary information about the implementation, because they’re meant for debugging.

So assert still has a place, and I’m arguing it should be treated as a sanity check (and cause a build failure). Another word, like “verify”, should be used for the actual tests. Unfortunately, it comes down to user discipline and personal judgement as to which to use when, but a good rule of thumb is that if an assertion fails, then you did not get a meaningful result for the test.

Advertisements