The purpose of this page is to provide scrum masters, product owner proxies, programmers, and agile testers with tips and techniques for writing acceptance criteria that is useful for the whole team during story estimation, development, and acceptance testing. It is helpful to document acceptance criteria in a central location (for example, on the team's FitNesse wiki) as it is drafted so that the full team can have easy access to the most recent information about current and upcoming stories.

This page is organized into two main sections:

Processes for Defining Acceptance Criteria
Techniques for Documenting Acceptance Criteria


Processes for Defining Acceptance Criteria

In general, project teams have observed that gathering and defining acceptance criteria involve the following activities (not necessarily in the order presented).
  • Hold a discussion with the full team about what the story is meant to accomplish.
  • Walk through the production version of the application, if applicable, to understand the current business process. If possible, sit with clients as they do their work.
  • Identify as many real examples as possible to guide discussion with product owners and the development team.
    • Group examples into similar categories
    • Use examples to extract business rules
    • Conversely, use known business rules to help identify relevant examples
  • Describe what instead of how. Keep the GUI design and other implementation details out of the acceptance criteria as long as possible. They can be discussed during another meeting. Specifications that only cover what will allow developers more freedom to implement good solutions. Also, acceptance tests written as specifications rather than scripts are more stable and easier to maintain over time.
  • Experiment with using different formats for expressing acceptance criteria (see below)
  • Remember that every story and team is unique. Not every format will work with every story. Use what works best for the team to ensure clear communication.
  • If stories are routinely returned as not accepted, retrospect with the team on why this keeps occurring.
  • If an issue is found with a story after it has been accepted, retrospect with the team on why the issue was not caught during development or acceptance, and make appropriate adjustments to the process.
Project leaders and product owners have incorporated the above activities into their respective projects in a way that allowed their teams to settle into a comfortable rhythm. Customized processes reflect project characteristics such as project type (large vs. small, enhancement vs. new development, UI vs. backend), team member skills, and the type and frequency of interaction with end users. The table below outlines the processes for gathering and defining acceptance criteria followed by recent agile projects.
Project
Project Characteristics
Process for Gathering and Defining Acceptance Criteria
SALSA
  • SALSA backend rewrite
  • Considerable coaching from agile consultants
  • Multiple client areas
  • Product owner well versed in client needs and business process and skilled in defining acceptance criteria
  • Dedicated Agile Test Team member constructed Fitnesse tables
  • Project leader worked with management and client area representatives to identify functional themes and release plan.
  • When release initiated, project leader filled release backlog with story cards from intended release functionality and ongoing product backlog. Initial cards may or may not have acceptance criteria written on the back.
  • Iteration planning occurred incrementally in advance of iteration, and all team members participated. In most cases, acceptance criteria written on back of card.
  • When iteration began, stories and acceptance criteria were transferred to Fitnesse by Agile Test Team member, whether Fitnesse test to be written or not.
  • Product owner, Agile Test Team member, and CSC testers met separately from developers to discuss acceptance criteria and flesh out manual tests.
  • Agile Test Team member defined expected Fitnesse tables and collaborated with developer(s) to refine them.
CCOS
  • Rewrite and consolidation of multiple applications
  • Significant UI component
  • Team members and product owner had no previous exposure to agile projects
  • Considerable involvement by both product owner and client (i.e., client highly engaged)
  • Product owner had strong understanding of client's business process
  • Dedicated Agile Test Team member constructed Fitnesse tables
  • Product owner and project leader identified functional themes and release plan.
  • Wiki pages defined for each story as they were encountered in release plan.
  • Product owner defined acceptance criteria as bulleted list in Wiki pages.
  • Team reviewed acceptance criteria during task planning.
  • If story scenario was tested using Fitnesse, Agile Test Team member defined Fitnesse tables and fixture code with guidance from product owner.
Committee Systems
  • Rewrite House committee and coordinator applications
  • Dedicated Agile Tester
  • Dedicated CSC Product Owner Proxy
  • Product Owner and Apps Project Leader meet with clients to discuss features
  • Team discusses features, identifies stories, and defines business rules
  • Agile Tester translates business rules into acceptance criteria and updates wiki
  • Some acceptance criteria are formatted as text, others are formatted as Given/When/Then examples
OnTrack Enhancements
  • Enhancements and bug fixes
  • Dedicated Agile Tester
  • Dedicated CSC Product Owner Proxy (POP)
  • POP and Apps Project Leader meet with clients to discuss upcoming user stories
  • POP and Apps Project Leader provide Agile Tester with requirements, ideally in the form of real examples
  • Agile Tester records acceptance criteria in FitNesse wiki, using Business Rules and supporting examples (Given/When/Then)
  • Agile Tester leads a specifications workshopwith the entire project team
    • Team discusses edge cases and adds more scenarios to the FitNesse wiki, as needed.
    • Sometimes the team uncovers additional edge cases and questions for the clients. In these cases, the POP and/or project leader may follow-up with clients.
    • Team discusses whether an automated test can be created for the business rules. If the answer is yes, then the agile tester creates the FitNesse tables and the developers write the C# test fixtures.
    • Agile tester updates the FitNesse wiki.
  • Team decides if a design session is needed before coding begins
LIS Enhancements
  • Enhancements to multiple small to medium-sized applications
  • Team members had some prior exposure to agile development
  • No Fitnesse tests
  • POP and Apps Project Leader meet to establish a base set of acceptance criteria for each of these feature story and entered in the wiki.
  • Entire Agile Development Team meet to discuss each feature story and add additional acceptance criteria before the story is developed (if necessary).
  • If a feature story is not clear, POP and/or Apps Project Leader meet with client to discuss the feature.
  • If feature is complicate, greater than eight points, team will discusses the feature and document any additional acceptance criteria in the wiki.
Subscriptions
  • Subscriptions System Rewrite
  • Dedicated developer in the Agile Tester role
  • Dedicated CSC Product Owner Proxy (POP)
  • Considerable coaching from agile mentors
  • POP, Agile Tester, and Apps Project Leader meet with clients to discuss upcoming user stories
  • Agile Tester and Project Leader meet to further define stories and come up with questions for the POP and Clients
  • Agile Tester records acceptance criteria in FitNesse wiki, using Business Rules and supporting examples (Given/When/Then)
  • Agile Tester leads a specifications workshop with the entire project team
  • Team discusses edge cases and adds more scenarios to the FitNesse wiki, as needed
  • Team decides if a design session is needed before coding begins

Techniques for Documenting Acceptance Criteria (with examples)
There are various formats that can be used to document acceptance criteria. Project teams have found that there is no one format that works best for every story. The format used will depend on several factors, including how the team communicates best with the product owner (or product owner proxy), whether the team will be writing automated acceptance tests, and the process used to gather the acceptance criteria. The table below lists some techniques used by agile teams to document acceptance criteria and discusses the benefits and challenges of each.
Technique
Pros
Cons
Examples
Document the steps to test the story as part of the acceptance criteria.
  • Acceptance criteria is verifiable because the test to confirm story completion is identified.
  • Business rules aren’t expressly documented and may be hard to easily extract from the test steps. Try listing the business rules or examples along with the steps to verify/test that the story is done.
CMS: Primary Phone numbers
Record examples using the "Given-When-Then" format.
  • Encourages collaboration between all team members (developers, testers, CSC, clients)
  • Helps create a ubiquitous language
  • Exposes business rules
  • Specific scenarios are verifiable and become the tests to verify that the story is complete
  • Can be used to create automated tests
  • Brainstorming technique to help identify as many related scenarios as possible
  • Takes some practice and skill to write well.
  • Time-consuming.
  • Clients may be distracted by "excessive" detail. Try putting the business rule or a scenario title before each Given/When/Then.
House Committee System: MM-1

OnTrack: web service for TREAD

iTrack: Add Delivered Action
Express acceptance as executable requirements using FitNesse.
  • Clear representation of expected outputs for given inputs.
  • Tests are repeatable, which facilitates regression testing.
  • Special skills are required.
  • Developing fixture code must be factored in to all task planning.
  • Tests may be "brittle," where minor changes cause failure even if those changes do not substantively alter the business process.
  • Business rules aren’t expressly documented and may be hard to easily extract from the test
Speaker's TAS: Update Requested Entities

SALSA: Journal Automation

CCOS: Create Supplemental Calendar
Use FitNesse tables to express acceptance criteria, even if acceptance testing will not be automated.
  • Can provide a segue to automated testing, allowing product owners to become comfortable with tabular notation without the rigidity required for automated testing.
  • For some audiences, the tabular format may make criteria difficult to read and interpret.
HBO Subscriptions: Purge Subscription Data
Document acceptance criteria and business rules in bulleted lists.
  • As with any natural language representation, offers the product owner the most flexilibity without being distracted by representational details (e.g., special syntax).
  • Useful for capturing criteria during client meetings.
  • No standard language or format, so different readers may interpret the content differently.
House Committee System: MM2

Speaker's TAS: Expired Appointment Counts


References:

Blogs: