Acceptance Criteria Presentation

A few weeks ago I gave a presentation about acceptance criteria and agile testing to a team of developers I’m working with.

Some of the developers were familiar with agile processes & test driven development, but some were not. I introduced the idea of behavior driven development, with both rspec “it should” and gherkin “given/when/then” style syntax. I stressed that the exact syntax is not important, but consistency helps with understanding and can also help avoid “testers block”.

It’s a Java shop, but I didn’t get into the details of JBehave, Cucumber or any other frameworks.  I pointed out that you can write tests this way without implementing the automation steps and still get value — with the option of completing the automation later.  This is particularly valuable in a system that is difficult to test, or has external dependencies that aren’t easily mocked.

Here are the slides:

Acceptance Criteria Presentation [PDF] or [PPTX]

And a rough approximation below:


Acceptance Criteria

 

how to make it easier to know if what you’re doing is what they want you to do


What are Acceptance Criteria?

Image


By any other name…

● Requirements
● Use Cases
● Features
● Specifications
● User Stories
● Acceptance Tests
● Expected Results
● Tasks, Issues, Bugs, Defects,Tickets…


What are Acceptance Criteria?

Image


…would smell as sweet

 ● A way for business to say what they want
● A way for customers to describe what they need
● A way for developers to know when a feature is done
● A way for testers to know if something is working right


The “Agile” definition


User Stories

As a … [who]
I want to … [action]
So that I can … [result]


Acceptance Criteria

Given … [some precondition]
When … [action is performed]
Then … [expected outcome]

(Gherkin style)


Acceptance Criteria

Describe [the system] … [some context]

It (the system) should … [expected result]

(“should” syntax)


Shh…don’t tell the business guys

it’s programming

Image

but can be compiled by humans…and computers!


Inputs and Outputs

if I enter X + Y
then the result should be Z

f(x,y) = z

 


 Not a proof

or a function
or a test
or a requirement
or …

It’s just a way to help everyone understand


It should

  1. Describe “it”
    (feature/story/task/requirement/issue/defect/whatever)
  2. Give steps to perform
  3. List expected results

Show your work

Image

● Provide examples
● List preconditions
● Specify exceptions


A conversation not a specification

Do

● use plain English
● be precise
● be specific

Don’t…

● worry about covering everything
● include implementation details
● use jargon
● assume system knowledge


Thanks!

If you’re interested in learning how to turn your manual testing process into an agile automated test suite,  I can help.

contact me

Aaron Evans
aarone@one-shore.com

425-242-4304



 

 

Are you suffering from vague specifications?

I was reading a comment on the  QA & Test Management Solutions group on LinkedIn this morning.  There was a question with a pain we’ve all felt as testers.

Are you suffering something similar?

Software specifications with “maybe can do….” or “It coud be located there or maybe there” are not specific at all. If a SQA manager accept that kind of non-specific speficications, He must be going to work to a casino (That is the place for multiple possibilities and unespecific results) 

Here is my response, which got rather wordy, so I posted it here.

I would ask for examples. BDD encourages “specification by example” and while not everyone is a fan of the syntax, it can help to be specific (and somewhat rigid)

As a… [user role]
I want to… [do this thing]
So that I can… [accomplish this goal]

If they can’t describe the user and the goal, then maybe the task isn’t important.

Then, you can describe specific scenarios:

Given… [these pre-conditions]
When… [some action is performed]
[with this specific data]
Then… [some result]

And press a little further (though you don’t want them dictating implementation details) to help describe what they expect the result to achieve

Verified by… [these post-conditions]

You can build your test assertions based on the post-conditions.

It should… [have this state]

Have them describe several scenarios until you think you’ve got it or they’re sure they have all the bases covered. Suggest alternate scenarios as you see them and see if they’re valid or important (at the risk of making more work for you). Then give them a time-frame to implement and negotiate features versus time.

It works rather well if everyone is willing to work together and goes into it with an honest and non-antagonistic approach.

You don’t have to use the exact verbiage. Given/When/Then is called “gherkin” syntax (gherkin is a type of cucumber — http://cukes.info.) I think If/Then/Else is just as suitable (and business types won’t even know it’s programming.)

The important thing is the feedback loop of communication and iteration, and examples are a great way to encourage concrete thinking and clarify misconceptions.

Running NUnit tests programmatically

I’m working on a test framework that needs to be run by less-technical testers. The tests are data driven from a spreadsheet (google docs spreadsheet API + gdata.)

Tests will be run locally (for now at least) since there isn’t a test lab available for remote execution, and no CI. I didn’t want to have to require users to install NUnit to execute tests.

At first I started by writing a main() class and rolling my own assertions. But I decided that the parameterized test features of NUnit were worth the effort of a little research. NUnit can, in fact, be run programmatically, though the execution appears less flexible than with other frameworks.

I created a TestRunner class with main function:


using System;
using NUnit.Core;
using NUnit.Framework.Constraints;
using NLog;

namespace oneshore.qa.testrunner
{
    class TestRunner
    {
        Logger log = NLog.LogManager.GetCurrentClassLogger();

        public static void Main(String[] args)
        {
            //get from command line args
            String pathToTestLibrary = "C:\\dev\\oneshore.Tests.DLL"; 

            DateTime startTime = System.DateTime.Now;
            log.Info("starting test execution...");

            TestRunner runner = new TestRunner();
            runner.run(pathToTestLibrary);

            log.Info("...test execution finished");
            DateTime finishTime = System.DateTime.Now;
            TimeSpan elapsedTime = finishTime.Subtract(startTime);
            log.Info("total elapsed time: " + elapsedTime);
        }

        public void run(String pathToTestLibrary)
        {
            CoreExtensions.Host.InitializeService();
            TestPackage testPackage = new TestPackage(@pathToTestLibrary);
            testPackage.BasePath = Path.GetDirectoryName(pathToTestLibrary);
            TestSuiteBuilder builder = new TestSuiteBuilder();
            TestSuite suite = builder.Build(testPackage);
            TestResult result = suite.Run(new NullListener(), TestFilter.Empty);

            log.Debug("has results? " + result.HasResults);
            log.Debug("results count: " + result.Results.Count);
            log.Debug("success? " + result.IsSuccess);
        }
    }
}

Link to gist of this code.

The advantage to running tests this way is that NUnit does not need to be installed (though DLLs for NUnit — nunit.core.dll & nunit.core.interfaces.dll — and any other dependencies like NLog need to be packaged with the TestRunner.) You can still write and execute your tests from NUnit while under development.

One disadvantage is that you don’t have the full test results by using the TestSuiteBuilder to bundle every test it finds into one suite. I’d like to find a way to improve that. You also can’t run more than one test assembly at the same time — you can create a nunit project xml for that — and at that point you might as well bundle the nunit test framework.

For now, my base test class (that each of my Nunit tests inherit from) reports via catching and counting assertion failures and writing to a log file. It can then use the Quality Center integration tool I described in an earlier post, though I’m planning on wiring it all together eventually, so anyone can run tests automatically by clicking on an icon, using a File Picker dialog to select the test library (see upcoming post) and have test results entered in QC.

This will allow distributed parameterized testing that can be done by anyone. I may try to set up a web UI like fitnesse for data driven tests as well.