Post-agile Project Delivery, Part I - The Problems

Having run an Agile-Scrum project management method for almost 3 years, Steve Cowie reflects on the challenges of pure agile project management as an agency and the difficulties of engaging with clients for whom it simply is not a good fit

Photo of Steve Cowie
Thu, 2013-12-05 15:07By scowie

Last month I delivered a talk on this subject at a Drupal Camp and that’s available online and embedded at the bottom of this post, as are the slides from the talk. However, I thought it would also be helpful to have the gist of the argument in blog format as it’s sometimes quicker and more convenient than listening to a talk.

As you can infer from the title, we’ve moved beyond Agile in some of our projects because we don’t find it helpful for all cases. There’s a couple of points to be made before getting into the problems we’ve encountered. The first is that we do work on projects where Agile methodology does fit well. In fact, we’ve based much of our ISO 9001 quality certification around an Agile-Scrum approach.

The second point is that it’s arguable that moving away from a proscriptive Agile model, one in which you stick rigidly to a pre-defined set of processes, is actually very agile. Adapting your process in response to experience and circumstances could be seen as closely aligned with the Agile manifesto principle of “responding to change over following a plan”.

With these caveats in mind, here are some of the problems we’ve encountered when seeking to apply Agile-Scrum methods on actual projects.

The product owner

At the heart of Agile-Scrum is the idea that projects should belong to a product owner: i.e. the person within the client’s organisation who acts as a focal point between stakeholders on one side, and the development team on the other. Virtually any software project will have a bunch of stakeholders, all of whom have a business interest in what’s being built. These could include customers, managers, system users, and system maintainers. The product owner is responsible for understanding all the stakeholder requirements, and translating these into a form that a development team can use for planning, building and testing. The owner also has to set business priorities, and manage change as the project proceeds.

This is potentially a huge and onerous role for one person and we find it’s the single most common point of failure when running Agile projects. In some projects, the client simply doesn’t want to be a product owner. He’s hired a team of experts so why has he now got to manage them? Why can’t we just agree a brief and get on with it? Now, we all know that agreeing a brief that accurately and comprehensively captures a complex project is much easier said than done. In fact, maybe clients should be forced to act as product owners. Unfortunately, it may not always be possible for an agency to push customers into roles.

Another product owner problem is that there may be a person who gets what the role is and why it matters, but that doesn’t mean the organisation as a whole agrees. Quite often the nominal product owner has to defer to senior managers who reserve the right to change their mind without consulting the product owner. Again, this shouldn’t be happening, but I doubt if we’re the only development agency that runs into this phenomenon.

Assuming we have a client who is happy to identify a product owner, there are still potential pitfalls. One is that the owner is not given the necessary time to fulfill the role. The new projects justs gets added to her other work because the client hasn’t understood what’s involved.

Even if time is allocated, it might be that the nominated person struggles with the requirements of a product owner. Capturing everything required from a website into user stories or features is not an easy task, never mind the fact all those stories need acceptance tests. An essential part of requirements-gathering is the ability to identify business priorities and use this to say ‘no’ to some features. Also, the product owner has to know when to say ‘stop’ because we’re close enough to the minimum viable product. A product owner who can’t say no to anything will probably also struggle with the idea that estimates are not guarantees, the connection being that if you can never say no then it implies everything has to get done, regardless of time.

Setting priorities

It’s normal in an Agile process to have a backlog of user stories or features, and the product owner picks items from these to make up a defined period of work, often described as a sprint. Normally, the basis of choosing which stories to do will be a combination of business value and resource cost. In other words, the owner will select the highest business items first although she may use the amount of time needed as an extra basis for prioritising. In some projects this works perfectly, particularly where an existing web application is being refined and extended. In other projects, it’s meaningless because it turns out that all the features are required for launch. In that situation, even if requirements are organised into user stories, they are fundamentally a fixed specification.

It’s surprising how often developers view a backlog as a wish list whereas clients see the list as the spec for what’s going to be built.

Scope for iteration

Flexibility and the ability to accommodate change are at the heart of Agile. By avoiding extensive design up front, you get going early. You then develop in relatively short sprints, typically one to two weeks long, and at the end of each sprint you can review and set new goals for the next sprint.

This iterative approach only makes sense when there is actually scope for iterating.  If you don’t have the opportunity to review progress and reset priorities in the light of what’s been achieved, then you might as well agree all your features at the beginning and stick to them. In ‘Get Agile’ (Pieter Jungerius et al) they suggest that it doesn’t make sense to apply Agile for projects with fewer than three sprints, which we’d broadly agree with.

Doing some rough and ready calculations as shown in the table below, we think a likely minimum cost for a sprint would be around £17,500. Allowing there will probably be some extra costs at the start and finish of the project, this suggests the minimum cost at which Agile is relevant is around £55,000. That’s fine, except we’ve read plenty of RFPs where the budget is under £30,000 but the client would like it to be done Agile.

Notional costs for a 10 day sprint

Item Unit cost 10 day cost
Designer £500 £5,000
Developers x 2 £1,000 £10,000
Scrum Master £250 £2,500
Total   £17,500

Sizing the work

Almost everyone accepts the need to do some form of estimate of how much work is involved to build an application. This may sometimes be avoided for in-house teams where an organisation has decided that a given application is essential and simply has to be built. But even in that circumstance, the project managers will probably want to have a feel for how long the team might take so they know how to allocate resources. So, we need to get some sense of the size of the whole project and therefore of how much resource is required to build individual components.

Agile manuals often recommend using scales other than time, such as Fibonacci scale points (1,2,3,5,8, 13) or t-shirt sizes (S,M,L,XL). The reasons for this are partly about speed because we can’t really work out the time involved without making decisions about the approach, in which case we’re fairly far into design-up-front.  Another argument for alternative scales is that they can act as a useful stimulus for discussions about approach. If one team member scores a story low, and another scores high, they need to find out what the basis of the score was.

We find a few problems with this in practice. One is that clients generally want clear indications of price so telling them it looks like a dozen extra large t-shirts won’t work. Another problem is that developers in our experience feel much more confident working in time. They like to work out exactly how they will deliver a given feature and then work out the time required. As with previous points, we have worked on projects where points-based estimating has been helpful, particularly extended projects involving in-house teams. The problem arises when trying to apply Agile estimating to the wrong projects.

Big Design up front (BDUF)

When Agile is being promoted, it often seems to be as an alternative to traditional ‘waterfall’ models where a large amount of resources are spent at the start of the project on planning, specifying and documenting the solution before any development starts. The supposed flaws with this approach is that it’s slow, cumbersome, and inflexible. We’re all aware of software projects, often in the public health area, where there’s an audit report bemoaning the fact that by the time system X was built, it was already obsolete and cost a fortune. It makes much more sense, the argument goes, to rapidly build a minimum viable product (MVP), and get that into production. Then you can iterate versions to improve the MVP.

Again, we have no argument with the principle and can think of projects where this has made sense, but we can also think of quite a few scenarios where this is problematic, or irrelevant. Let’s be honest - our business is creating websites in Drupal. Sure we make sophisticated sites with complex functionality, but we’re not generally building super-complex systems such as the ultimate system for unifying all social security benefits. (On second thoughts, maybe they should have asked some Drupal companies about that one). That being the case, we generally have a fairly clear sense of what needs to be built, at the point when we start building.  So if we know what we’re making, why not do a fairly in-depth design exercise at the start?

The case for this approach is reinforced by the expansion of devices people use for accessing web applications. If we’re going to guarantee that something will work well across all devices, almost all browsers, and is also accessible, that’s really hard to do if you don’t prototype, and a prototype is in effect a design model. 


This week I’ve outlined some of the main problems we encounter as an agency seeking to apply Agile Scrum methods. The recurrent point seems to be that the methods are more valid in some situations than others. In fact, we reckon they can be seriously counter-productive if applied in the wrong projects. Next week I’ll take a look at some of the custom techniques we’ve developed to address these problems.

The Talk