scenario based literate testing

A more interesting strategic area I’ve been looking into is with literate tests that better describe the requirements.  The goal is something like this PHP example:

$test->verify($user)->with_membership("professional")->can()->access_premium_content();

You are then specifying the requirement as a test, which then knows how to not only navigate, but set up the correct scenario, including registering a user with specific parameters, and a positive or negative outcome.  You can then easily implement

$test->verify($user)->with_membership("basic")->cant()->access_premium_content();

And even better, specify the various scenarios with Fitnesse style tables.

user with membership_type can access_premium_content
+------------------+--------+
| membership_types | expect |
+------------------+--------+
| professional     | true   |
+------------------+--------+
| basic            | false  |  
+------------------+--------+

foreach($membership_types as $type) {
   $test->verify($user)->with_membership($type)->accesses_premium_content();
}

I haven’t really gotten the chance to explore that yet, since the first obstacle to that is breaking out the XUnit style framework, and that will take some work.  But you could then build underlying test methods with selenium, using chain of command.

verify() is a method that essentially creates & returns a test conditions, analogous to an xunit test method.

$test is then equivalent to a TestCase class.  Helper methods can be included at the test level, including setup and teardown, as well as any test domain specific helpers.  You then have a clearer demarcation of what to put in a test — conditions that share the test domain.  Ideally, only tests that are identical except for isolatable scenarios that are easily specified in a table.

$user is the object under test (or principal actor) wich is passed to the verify(), but that’s really just an implementation decision.  It defines the base state and a test condition should pass or fail based on this object’s state.

with_membership($membership_category) is a modifier to the test condition.  It establishes the state which distinguishes a scenario.   There can be multiple modifiers chained together and they should each return the condition.

can()/cant() signifies an expected outcome.  You can create methods with any name that establish this such as should()/shouldnt(), has()/doesnt_have(), is_less_than($number), etc.   It determines the assert.

access_premium_content() is the meat of your test.  It contains the steps needed to get to your assert that are outside the state setup.  It defines the condition that each scenario will test against.

You can also chain expected outcomes with and() or but(), but you shouldn’t use more than 1 or 2 of these.

BDD tools like JBehave and RSPEC/Cucumber try to get here, but they’re more geared towards unit tests, and I haven’t really been satisfied with them.  They seem more geared towards establishing a particular syntax that only developers can write that analysts can read.

I don’t know that this wouldn’t fall down the same path in implementation.  But my goal would be maintainability, and clarity of coverage.  Readability is ancillary and expressiveness shouldn’t be sacrificed for whitespace or any particular methodology’s phrasings.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s