Jan Kremer's blog

Category: All things Agile Page 1 of 2

Product Owner acceptance and testing in a agile delivery model

By

On February 1, 2016

In All things Agile

According to the scrum guide, Sprint Reviews are held at the end of each Sprint with the objective to inspect the software increment and adapt the product backlog if needed. The session is for the Product Owner to explain what product backlog items have been ‘done’ and for the development team to demonstrate the work that it has ‘done’. And in theory, each Sprint delivers a potentially releasable product Increment. So, how do we know when something is ‘done’? How do we assure overall quality? In addition, releasing software to production requires usually rigorous user acceptance testing and approval from various departments and stakeholders.

The Scrum guide provides no answer on when and how to accept a user story. This means that it’s up to the Product Owner and Development team to customise this to their preference, taking organisational constraints or governance processes into consideration. Usually, User Stories could be accepted:

  1. As soon as possible – during a Sprint
    It is completely fine for the Product Owner to accept stories during a Sprint. By doing so, both the Product Owner and Developer validate the product as soon as possible, allowing time for any improvements or defect fixing. If a user story is rejected by the Product Owner then a discussion should clarify whether or not an updated User Story can be completed in the current Sprint or that it should go back to the Product Backlog for a future sprint. Having many individual acceptance or test sessions may not be an efficient use of development time. However, many acceptance discussions between Product Owner and developer might be disruptive to the development process.
  2. At the end of a Sprint – during Sprint review
    User Stories may also be accepted at the end of a Sprint during the Sprint Review meeting. When the development team is demonstrating their User Stories completed, the Product Owner could accept the User Story there and then, or may provide feedback which then results in an amended or new User Story. As this is the end of the Sprint, any User Stories not completed go back into the Product Backlog. During the next Sprint Planning the Product Owner and Development team collaboratively agree on the next Sprints Definition of Done. Note that any story that is rejected by the Product Owner should be discussed at the Sprint Retrospective to determine why the User Story was rejected and what can be done in the future to ensure User Story development meet the Product Owners expectations.

In more complex project it may be more difficult to fully accept functionality. For example, testing integration with legacy systems may be limited by test systems and data availability. In addition, with new cloud based technology the development team may produce such an amount of functionality that the Product Owner cannot keep up. Traditionally, functional and acceptance testing were also used to get business buy in and act as knowledge build up.

  1. Additional testing – outside Sprints
    These key benefits may be achieved by adding a parallel test stream next to the Sprint development cycle. This means that after unit testing (by developer or development QA team) and Product Owner review, a Quality Assurance team could then perform additional testing such as Functional Testing, System Integration Testing, Performance Testing, Security & Penetration Testing, Usability Testing, User Acceptance Testing or Production Readiness Testing.Testing

While testing outside Sprints may not be in line with Scrum, it is in line with agile principles of releasing early and often by testing as early as you can. In addition, testing outside a sprint by another team does provide additional quality assurance and may meet internal governance guidelines. The reason for pulling Functional Testing forward, right after the first Sprint, is to enable the organisation to get organised, start getting into the habit of writing test scripts and get other logistics in place. This then reduces time for the User Acceptance Test as well as de-risks any Go Live delays.

Different project delivery principles

By

On January 23, 2016

In All things Agile

And old European proverb says ‘all roads lead to Rome’ meaning that different paths can take one to the same goal. The same is true for project delivery. There are different project management methodologies with a proven track record. The question though is how do you want to reach your goal? A friend of mine walked to Rome. I prefer a bicycle, and my wife prefers a plane. Depending on what you want to achieve you can choose different means of transport. The same is true for project delivery and there are different project management methodologies to choose from. While comparing every aspect of a waterfall and Agile methodology would be zooming in too much detail, it is interesting exploring the differences in guiding principles.

Waterfall principles
While PMBok is widely used in the US and Canada, in Europe the de facto (waterfall) project management methodology is PRINCE2. PRINCE2 (acronym for PRojects IN Controlled Environments) has a process-based approach for effective project management established in 1989 by the UK government. It moved to the public domain and offers non-proprietorial best practice guidance on project management.

In PRINCE2 much is defined upfront and plan-based, and remains focused on the original Business Case throughout the project life cycle. The 7 key principles are the ‘guiding obligations and good practices which determine whether the project is genuinely being managed using PRINCE2′ and are as follows:

  1. Continued business justification
  2. Learn from experience
  3. Defined Roles and Responsibilities
  4. Manage by stages
  5. Manage by exception
  6. Focus on product
  7. Tailor to suit the project environment

There is a strong emphasis on dividing the project into manageable and controllable stages. ‘Everything’ is ‘defined’ at project start and the project stages are generally progressing from Design to Build, Test, Deploy and Support. A Business Case is created in ‘Starting Up a project’ detailing timescales, costs, benefits and risks. PRINCE2 works with Tolerances, set up front, which allow for permissible deviation.

Agile principles
As a agile delivery framework, Scrum is founded on empirical process control theory, called empiricism. Empiricism asserts that knowledge comes from experience and that decisions could be best based on what is known. Scrum employs an iterative and incremental approach to optimize predictability and control project risk. These three pillars uphold every implementation of empirical process control:

  1. Transparency
  2. Inspection
  3. Adaptation

Defined versus Empirical
Walking to Rome or flying the same distance both have their pro’s and con’s. Likewise, there are different delivery models based on different principles. Some differences between defined and empirical are listed below.

Defined Empirical 
Work and outcomes are understood before execution Frequent inspection and adaptation occurs as work proceeds
Given a well-defined set of inputs, the same outputs are generated every time Processes are accepted as imperfectly defined
Follow the pre-determined steps to get known results Outputs are often unpredictable and unrepeatable

Defined is great for relative simple problems like production lines where you can fix scope and budget, but unlikely time, by documenting all requirements and output upfront. The Project Manager authorises Stage Work Packages based on the Project Initiation Document and creates an End Stage Report summarising the actuals from the current stage and revise project forecast. The Project Manager can do this only because the project team can break down the entire deliverable to a level where it can estimate time required. However, a predictive approach will not work in complex environments.

Where budget and time are fixed, but scope is flexible (the unknown) an empirical approach works best. It creates incremental deliverables in short-term and manageable Sprints, independent of an overarching plan. This means that agile projects are more responsive to changes in project environment and customer / business requirements. The complete Product Backlog is not defined in detail upfront as the Product Owner inspects and adapts requirements throughout the project life cycle. Business value is continuously evaluated as customer requirements may change and technology will evolve. You could argue that a Sprint is a mini predictive project and where scope is fixed at the last possible moment before delivery.

Which one to choose depends on your business characteristics and needs such as customer requirements and time to market. In addition, it also depends on your organisational capabilities; not everyone can work with an unpredictable outcome.

 

References and copyrights:
PMBok: http://www.pmi.org/
PRINCE2: https://www.axelos.com/best-practice-solutions/prince2
Scrum: https://www.scrum.org/
Interesting read PMBok versus PRINCE2: http://www.deltapartners.ca/blog/prince2-vs-pmbok-comparing-apples-and-oranges

How to contract an Agile project?

By

On January 14, 2016

In All things Agile

As mentioned in a previous post (http://www.jankremer.net/category/all-things-agile/) it’s human nature to plan as much ahead as possible. This behavior is seen during project contract negotiations as well. Procurement usually wants to know what value they will get for their money. Value as in ‘which requirements will the project deliver’. More interesting, they usually want a delivery date as well, making the project fixed in time, scope and budget. How does this work with an agile delivery approach, where we promote ‘adaptation’.

Let’s first look at two common contract types:
1) Variable-price, fixed-scope (Time & Material) – Here the price is flexible, usually within a tolerance or a maximum, and during the project Change Orders manage any additional requirements or functionality. If the project is not scoped properly or the world is changing during the project life cycle, you will have a lot of change orders.
2) Fixed-price, fixed-scope (Fixed Price) – Here only time is flexible, although customers usually have a deadline or Go Live expectation as well. This works really well when you know exactly what you want and the supplier is willing to agree to a price upfront. Because there is a risk involved, suppliers will add a risk premium to the price for any unforeseen things. It also means Change Orders for any additional or changed requirements.

One of the key principles for an agile delivery approach is ‘Adaptation’. Agile projects inspect and improve throughout the project life cycle. Prior to an iteration (Sprint) requirements (User Stories) can be added to the product backlog. As long they are not part of a Sprint, they can be updated or deleted. And this will happen throughout the project.

Variable-price, fixed-scope (Time & Material)
A Time & Material contract will work for an Agile project depending on how you define ‘price’ and ‘scope’. The price could be defined as an estimated number of resource days. An agile project is basically Sprint output driven and you could define scope as a number of Sprints outputs, a ‘potentially releasable product increment’ as shown the in the example below.

Example  
Deliverable 1 5 Product Increments
Description Following the incremental software development approach, <supplier> will provide development capacity to deliver 5 software increments based on the items recorded in the agreed Product Backlog.

A software increment will be reviewed by <customer> at the end of each sprint at which point <customer> will provide initial feedback.

Changes or defects will be recorded in the Product Backlog, prioritised and put forward for planning in the next Sprint.
Responsibility <supplier>
Acceptance Criteria Acceptance criteria are defined for each item in the Product Backlog. The software increment shall be accepted for items meeting the acceptance criteria, items not meeting the acceptance criteria will be recorded in the Product Backlog, prioritised and put forward for development in the sprints ahead.
Acceptance Process <Customer> to perform acceptance testing for the product increment the week after Sprint completion, and produce a test report within the same week.

Fixed-price, fixed-scope (Fixed Price)
The second contract type, Fixed Price, is the most difficult to draft for Agile projects. If you fix ‘price’ and ‘scope’ the only variable is your product quality. While development teams use User Stories with acceptance criteria and the way of working is based on inspection and adaptation, a fixed scope will simply not work and will cause frustration and disputes.

Fixed Price contracts usually fix costs based on resource days. They should not contain a list of requirements as these may change. In order to give some direction of what the project is aiming to deliver, and to get the contract signed off by procurement, you could list high level ‘candidate’ Use Cases. However, this is dangerous as it sets customer (and legal) expectations which might not be met. The more detailed scope description, the more quality will be compromised. Remember: the supplier will ‘earn’ money with less effort, while the customer will earn value by postponing acceptance and squeezing in new requirements. There is always room for interpretation, and detailed requirements will not improve trust or collaboration.

Fixed-price, variable scope (Budget box)
The best way to contract an agile project is through a third option:
3) Fixed-price, variable scope (Budget box) – Here the price, based on resource capacity, is fixed and the product owner has the flexibility to influence requirements and its quality. The budget is ‘boxed’ and provides flexibility for the customer. If the product owner wants the development team to paint the walls purple, then that’s what the team will do…

Customers (Product Owners) must now deal with variable scope. They can prioritise product backlog items by business value. From a finance point of view this is also a interesting way to contract as there will never be an overrun. And as each Sprint delivers a releaseable product, the project will always deliver.

However, there is one catch. This way of ‘contracting’ requires a healthy level of trust between supplier and customer. How do you know your supplier will provide you with the best resources and a team that will be delivering the best quality?

Summary
An Agile contract should support the Agile principles transparency, inspection and adaptation. Agile projects usually have fixed development capacity (team) and are time boxed by a number of iterations. Therefore Fixed Price, based on resources and time, and a variable scope, managed through a product backlog, are sound starting points.

It is not a good idea to fix scope in any way. Suggesting candidate product backlog items may be a way to get around procurement, but it sets expectations which may not be met. The team will deliver as per contract, probably by compromising on quality, and it will not deliver best business value.

Page 1 of 2

Next

Powered by WordPress & Theme by Anders Norén