Acceptance Criteria Presentation

A few weeks ago I gave a presentation about acceptance criteria and agile testing to a team of developers I’m working with.

Some of the developers were familiar with agile processes & test driven development, but some were not. I introduced the idea of behavior driven development, with both rspec “it should” and gherkin “given/when/then” style syntax. I stressed that the exact syntax is not important, but consistency helps with understanding and can also help avoid “testers block”.

It’s a Java shop, but I didn’t get into the details of JBehave, Cucumber or any other frameworks.  I pointed out that you can write tests this way without implementing the automation steps and still get value — with the option of completing the automation later.  This is particularly valuable in a system that is difficult to test, or has external dependencies that aren’t easily mocked.

Here are the slides:

Acceptance Criteria Presentation [PDF] or [PPTX]

And a rough approximation below:

Acceptance Criteria


how to make it easier to know if what you’re doing is what they want you to do

What are Acceptance Criteria?


By any other name…

● Requirements
● Use Cases
● Features
● Specifications
● User Stories
● Acceptance Tests
● Expected Results
● Tasks, Issues, Bugs, Defects,Tickets…

What are Acceptance Criteria?


…would smell as sweet

 ● A way for business to say what they want
● A way for customers to describe what they need
● A way for developers to know when a feature is done
● A way for testers to know if something is working right

The “Agile” definition

User Stories

As a … [who]
I want to … [action]
So that I can … [result]

Acceptance Criteria

Given … [some precondition]
When … [action is performed]
Then … [expected outcome]

(Gherkin style)

Acceptance Criteria

Describe [the system] … [some context]

It (the system) should … [expected result]

(“should” syntax)

Shh…don’t tell the business guys

it’s programming


but can be compiled by humans…and computers!

Inputs and Outputs

if I enter X + Y
then the result should be Z

f(x,y) = z


 Not a proof

or a function
or a test
or a requirement
or …

It’s just a way to help everyone understand

It should

  1. Describe “it”
  2. Give steps to perform
  3. List expected results

Show your work


● Provide examples
● List preconditions
● Specify exceptions

A conversation not a specification


● use plain English
● be precise
● be specific


● worry about covering everything
● include implementation details
● use jargon
● assume system knowledge


If you’re interested in learning how to turn your manual testing process into an agile automated test suite,  I can help.

contact me

Aaron Evans




Better User-Centric Test Automation

Code-Centric Tests

Most automated testing is code-centric.  Written for developers, by developers.  Unit tests, and some low level integration tests.  This is good and necessary, and it helps developers tremendously, especially when refactoring their code and adding new features to make sure they didn’t break any existing features.

User-Centric Tests

It would be nice to have more user-centric test automation.  Tests that not only act like a user acts, but makes sense to users.  Tests that get used (run frequently) and provide real value (prevent bugs).

User-centric test automation suffers from a number of problems.  They are frequently verbose, brittle, and opaque.  They are slow and hard to maintain.  They don’t make sense to either developers or business.

Let’s look at a few of these issues.

Verbose Tests

Test automation often gets wrapped up in procedural (click here, type this) or implementation (clear cookies, populate test database) details.  This obscures their true purpose and makes them hard to understand and difficult to maintain.

Brittle Tests

Partly because of their verbosity and complexity test automation is often brittle.  Either you have a lot of steps with significant duplication, or you have too much abstraction that means only a  expert can modify them.

Opaque Tests

Tests are often written in a way to make execution easier, or in a shorthand (keyword, data-driven) that makes them not understandable to someone unfamiliar with the syntax.

Testing Silos

People talk about the bad old days when developers would throw the product over the wall to testers, who treated it like a black box.  They’re usually talking about manual testing, but test automation suffers from the same break down.  Developers have their automated (unit) tests and Testers have their automated (functional) tests, and there is no correlation between the two.  And there is often poor correlation between business requirements and automated tests as well.

Wouldn’t it be nice if we could solve these issues…or at least find areas for improvement?

Help me fix it

I’m working on ways to improve this and would love your feedback.   Anyone interested in a virtual round table to talk about these sorts of user-centric test automation issues?

Regression is progress

What would you think of a web framework that advertises itself as:
  • having no session handling
  •  imposes a specific testing framework
  • has an exclusionary “API” that tries to be self contained with the philosophy that it is “all you will never need”
  • using static controllers
  • requiring you to handling requests asynchronously
  • being yet another clone of Rails
  • using Scala for templates
  • written in Java but goes out of it’s way to avoid the existing java ecosystem
It’s called Play.

Not writing much

It’s the end of the year, and I haven’t written much on my blog in 2013, but it’s been a fairly busy year.

I started the year working at Ancestry and then went back to freelancing.  I hired 3 interns, and started working on building a cloud test environment with them. I got a client, DIIO, and ended up working full time for them.  I even got to go to the company party in Mexico.  It was a blast, thanks DIIO!

But Kelsey made me come home from Mexico and she’s talking about moving.  I’m not sure what the new year holds, but I’ve got a couple of interesting ideas and some more projects.

1.) What if logging were just events that a logging service merely subscribed to. You could add logging “instrumentation” and measure performance, coverage, and lots of other things — if it were baked in (or an aspect!)

2) More playing with clouds and PaaS  AWS, Rackspace, Linode, Cloud Foundry, Juju, Docker

3) I’m going to set up my own private “fog” using Open Stack — for building test environments of course.

4) I’m going to be writing PHPUNIT SELENIUM documentation.

5) I’m going to write more and be purposeful.

6) I’m going to find an assistant who can proofread, handle invoices & budgets, do some basic research, handle vendor contacts, marketing & social media, oh yeah, and learn to test.


Writing Every Day

I’ve been following Eric Davis for a long time — ever since I started hacking on Redmine in 2009.  As it’s maintainer, he was extremely helpful to a newbie like me.  Recently I’ve started reading his blog and newsletter about freelancing.  A recent post of his,  “Write Every Day” struck a nerve.   And a comment from Nathan Barry that he linked to hurt even worseslow, consistent progress is the only way to make big things happen.”

It’s something I’ve been intending to do — and procrastinating for years.  I’m not going to set a goal of 1000 words a day.  And I’m not going to publish a blog post every day.  But I am going to write every day for a half hour or so.  I’m also going to take the time, as Eric suggests, to edit every day.  That will hopefully allow me to make progress on a book about testing tools & automation in the cloud.  If I know I’m going to go back and edit, I won’t be as worried about writing garbage — or something that’s wrong, stupid, or cheesy.  It may even improve the quality of blog posts (no guarantees on today’s content.)

Besides writing a book, my goal will be to publish a monthly newsletter.  With at least 1 article about something technical, 1 commentary about a process or business objective, some personal news (including potentially a hurrah for a client), and a few interesting links.  I think of Mike Gunderloy’s blog A Fresh Cup or A Smattering of Selenium by Adam Goucher.

Will anyone read (or subscribe to) my newsletter?  I wouldn’t count on it, not at first.  But it might help me become a better writer, and even more importantly, it might help me be more focused.

Mending Fences

It’s fall, and winter seems to be coming quick this year. I promised to have a big BBQ at the end of summer, but it didn’t happen. Instead, the Saturday after Labor Day there was that crazy storm that rolled up suddenly with strong winds, heavy rain, and nonstop thunder. We’ve had a few good days since but this fall has been dominated by cold and rainy weather.

Well, it looks like this Saturday is likely to be clear and cool, and it might be our last one before winter. So I’ve got to get something done. Living in Seattle has taught me one thing — take advantage of the good weather when you can.

In that wind storm, our back yard fence blew down. Not all the way, but it’ i’s sagging pretty good. About the only things holding it up are our neighbor’s sprinkler heads and apple tree (and maybe a few clinging weeds on our side.)

Our landlord won’t take care of it. But there’s “shoud” and “can” — he should do it, but we can. So, tomorrow morning I’m going to go on a bike ride with my kids, head down to Lowes, and fix the fence. But I’m also going to barbecue.

That big neighborhood BBQ I talked so much about is going to happen. I’ve got to use the last of the charcoal anyway. There will be ribs, spicy sausage, and kabobs. I’m going to ask Kelsey to make rolls. If we can find it, we’ll have corn on the cob. Things will be cooking all day.

And there will be plenty of work to do. I could really use a portable table saw (that can cut 4x4s), a post hole digger, and a portable cement mixer. I’ve got a couple screw guns, but we could use another.

If the neighbors are ok with it, we’ll reuse as much of the old wood as possible. Maybe cut a few inches off the bottom to prevent rot. That means work unscrewing and trimming boards. If we get enough people working on it, it shouldn’t take too long.

Bring the whole family. Or just come with the kids and give your wife a couple hours off. Stay for as long as you like. There’ll be plenty of work to do, and plenty of food. You don’t have to work, and you don’t even have to eat if you don’t want to.

But while tall fences make good neighbors, I’m betting good food makes even better ones.