Why Projects Fail (Mastering the Monster) – Part 2.
By Brian Sutton, Director of Learning – QA Training & Consulting.
Towards a more Robust Model for Assessing Project Success
Success is not binary. It is not as simple as succeed or fail – there are degrees of success and failure.
To aid understanding, I would point to four distinct levels of success, each having its own discipline, tools and techniques. Excellence at each level is critical for overall success.
- Level 1 may be termed Project Management Success
- Put simply: did the project produce the desired output?
- Here we are concerned with the management of the work to be done. Key skills are clear scope and deliverable definition, thorough and continuous risk assessment and business case monitoring, planning to an appropriate level of detail, good estimating practices, appropriate scheduling and use of resources, cost and schedule tracking
- Invariably success at this level is measured in terms of the traditional triangle of Cost, Schedule and Performance. Did it cost what we said, did we deliver on time, did it do what we agreed. Some practitioners include Quality but this should not be considered a variable as the unscrupulous will cut corners on quality to maintain performance against the other variables
- Put simply: did the project produce the desired output?
- Level 2 may be termed Repeatable Project Management Success
- Put simply: do our projects consistently produce the desired outputs?
- Here we are concerned with the consistent application of good project management practice. Although we are still predominantly concerned with the management of the work to be done, we are looking for organisational structures that are designed to share and support best practice and to ensure predictability of outcome
- Success at this level can only come when project management is exercised within the auspices of an organisation wide methodology, supported by standards and a mechanism for learning from experience
- Put simply: do our projects consistently produce the desired outputs?
- Level 3 may be termed Project Success
- Put simply: did the project outputs produce the desired outcomes?
- Here we are concerned with an assessment of how well the project has delivered the business benefits that were claimed at the outset. This requires that the outputs of the project are fully and appropriately integrated into the business operation and that process adjustments and people skills are developed in tandem
- Measurement of success requires good metrics and an understanding of baseline conditions. Improvements at this level tend to come from a thorough understanding of benefits logic and from initiatives designed to assign individual responsibility for the realisation of benefits. This needs to be supported by a regime to capture and share lessons learned
- Put simply: did the project outputs produce the desired outcomes?
- Level 4 is inextricably linked with Corporate Success
- Put simply: do the outcomes produced have the intended impact on Business Strategy?
- The primary concern is with project selection and portfolio management, i.e. out of all the possible things in which we could have invested, have we chosen the optimum mix of projects that, when taken together, will most effectively contribute to the advancement of our stated business strategy?
- Measurement of this level of contribution requires a programmatic approach to organisational initiatives and probably a balanced score card approach to organisational performance measurement
- Put simply: do the outcomes produced have the intended impact on Business Strategy?
This model shows graphically that, when we talk of project failure, it could occur at any one of the four levels. What should not be surprising is that each independent level of failure has a corresponding different level of corrective action. Too often I see organisations that only know that they are failing, they do not stop to analyse why before they adopt corrective measures, and then invariably wonder why they make no progress.
The message is simple: understand where and how you are failing and then target the measures that produce the greatest likelihood of success.
Matching Corrective Interventions to Problem Levels
It is worth reminding ourselves that we are dealing with two types of project. Type 1 projects have shown themselves to be amenable to a range of measures that improve successful completion. By contrast, Type 2 projects remain largely impervious to our efforts at improvement. This is particularly the case when dealing with level 1 success.
Level 1 Success
For Level 1 success we need to answer the question: “did the project produce the desired output?” There are a number of very positive things we can do at this level:
- Product based planning will help produce consensus on the nature of the output required and the quality criteria to be applied in its assessment
- Robust risk management practices have the biggest impact upon on-time delivery.
- Rigorous change control procedures impact directly on cost performance
- Good project reporting practices, such as earned value analysis, have a direct impact upon cost performance but also ensure that the project is truly in control
- A project culture that rewards, rather than penalises, fault reporting will affect both cost and schedule. For if there is anything more damaging to a project than rework – it is undiscovered rework. Undiscovered rework is a project killer and it is most prevalent where deadlines are tight and project teams live in fear of their manager
If you are suffering Level 1 failures use the above list to help target the practice that will impact your particular problem. If you are setting up a new project and are concerned about Level 1 performance in your organisation dwell upon this statistic from Standish Group regarding challenged US IT projects: on average they were 189% over budget when challenged! There is only one way to avoid this horrifying possibility and that is to get the project steering committee to set tolerances on the project – typically 15%. Then institute a rigorous earned value reporting regime that insists both on open reporting of progress and routinely reforecasting time and cost to complete, based upon trends observed from actual performance. This doesn’t stop your project being challenged but it at least ensures that you are no more than 15% over when the challenge happens.
Finally if you are engaged in a Type 2 project, the first and foremost concern should be to take actions that change elements of its character so that it behaves more like a Type 1 project. For example:
- Select a development cycle that helps remove ambiguity about outputs / success criteria. For example, a discovery prototyping strategy to identify essential functionality or time-boxing using facilitated workshops of stakeholders to agree scope limitation or targeted functionality
- Use simulations or pilot implementations to better understand the likely benefits and the time profile in which they can be realised
- Establish project control boards and proper project steering to ensure continuity of purpose, should significant stakeholders change or move on
- Identify interim deliverables or phase the introduction of functionality so that early benefit can be delivered to those powerful stakeholders who are least supportive of the project
Once the above are in place the project will be more amenable to Level 1 success criteria.
Level 2 Success
For Level 2 success we need to answer the question: “do our projects consistently produce the desired outputs?” Level 2 success is independent of the nature of the project. Therefore the guidance below is equally valid for Type 1 and 2 projects. The emphasis is to move from success based on individual excellence to success based upon consistent and repeatable processes. We are therefore looking to implement structures that promote and sustain organisational learning.
The sorts of things that impact upon Level 2 success are:
- A simple, widely accepted and practiced standard vocabulary for project management
- This is most likely to be achieved by the implementation of some form of methodology, or by a collection of best practices
- This is most likely to be achieved by the implementation of some form of methodology, or by a collection of best practices
- The consistent application of best practice
- This is most likely to be achieved through the auspices of some form of “project support office”. Whether it is physical or virtual is of less importance than its ability to influence the consistent application of procedures relating to:
- Project initiation
- Risk management
- Change control
- Estimating – through the provision of accurate historical data
- Project planning (at the detail level)
- Project reporting, including the capture of actual performance
- This is most likely to be achieved through the auspices of some form of “project support office”. Whether it is physical or virtual is of less importance than its ability to influence the consistent application of procedures relating to:
- A mechanism for the capture and sharing of experience and lessons learned from projects and project reviews (not the same as post implementation reviews)
- The fostering of a community of practice amongst project managers from across the organisation
Level 3 Success
For Level 3 success we need to answer the question: “did the project outputs produce the desired outcomes?” Many project managers consider this to be beyond their remit, in that generally benefits are delivered or taken during operation after the project output has been delivered. The reality is that the ability to reap the benefits is often inextricably linked to actions that are, or should have been, taken before the end product was delivered. Benefits measurement of Type 1 projects is relatively straightforward as generally they rest on easily traceable revenue and maintenance streams.
However, Type 2 projects have far more complex benefits models, as there is often ambiguity about the desired outcomes that were to be produced. We are also often faced with the “serendipity effect” in that Type 2 projects are more likely to generate unpredicted side effects that could contribute to either cost or benefit.
The organisational initiative that impacts upon Level 3 success is:
- Proactive Benefits Management
- This requires rigorous benefits logic to be applied to the identification of benefits in the business case and regular reviews of the business case whenever a major change request is applied to the project
- Clear ownership for the realisation of benefits
- Clear metrics agreed at the project outset about how benefits will be assessed
- Rigorous post implementation review processes, that hold people accountable for the delivery of benefits
- A clear mechanism for the capture and enculturation of lessons learned from ongoing projects
Level 4 Success
For Level 4 success we need to answer the question: “do the outcomes produced have the intended affect on the Business Strategy?” Here we have moved beyond the realms of a typology for projects as at this level all projects exhibit essentially the same behaviour.
If project managers consider delivering benefit to be beyond their remit, then the idea that they are engaged in the realisation of business strategy is so alien as not to be worth consideration. Whilst it is true that the mechanisms that I will recommend below are generally outside the working remit of the project manager, the reality is that programmes and projects are commissioned specifically to deliver strategy and unless everyone thinks along these lines we should not be surprised if we create a culture whereby projects within a programme succeed at the expense of other, perhaps more important, programmatic efforts.
The sorts of organisational practices that can be put in place to impact upon Level 4 success are:
- Portfolio Management
- This means adopting a proactive portfolio approach to the selection and initiation of projects. This requires attention at the highest levels to ensure that we only approve those projects that complement the desired strategic direction. This inevitably requires a broader base for selection than just financial measures such as ROI. Indeed, I would expect consideration on the benefit side to be given to such things as: contribution to competitive advantage (or competitive response), fit with the business strategy, fit with the architecture etc. On the cost side, we should see considerations such as organisational risk, technical risk
- This means adopting a proactive portfolio approach to the selection and initiation of projects. This requires attention at the highest levels to ensure that we only approve those projects that complement the desired strategic direction. This inevitably requires a broader base for selection than just financial measures such as ROI. Indeed, I would expect consideration on the benefit side to be given to such things as: contribution to competitive advantage (or competitive response), fit with the business strategy, fit with the architecture etc. On the cost side, we should see considerations such as organisational risk, technical risk
- Programme Management
- If we are managing projects as a financial portfolio it will be necessary to embrace strong programme management practices. We cannot rely on individual project managers to instinctively know or act in accordance with the greater good – we need to manage to ensure that this happens
- Organisational Performance Measurement
- Most organisations have poor mechanisms for monitoring progress against business strategy; indeed most have incomplete or inaccurate ideas about what actions lead to what consequences. In order to manage programmes effectively we need effective measures of performance – evidence seams to suggest that these are best achieved through the application of some form of “balanced scorecard”. Furthermore measuring is not enough, we must learn from what we measure and adjust actions and belief structures accordingly – this requires the balanced score card to be used as an enabler of organisational learning
Final Thoughts
In this article I have provided a simple model of the multi-dimensional nature of project management success. It is neither prescriptive nor entirely robust; its value lies as a lens through which to view the problem. In the final analysis it does not matter whether there are three, four or five layers of success in your model – the important thing is that there is more than one. For once you admit to layers of success, there must be corresponding layers of appropriate corrective actions. If you are serious about improving the success of projects in your organisation use this model to help you target the practices that are likely to make a difference and match your project control structure to the nature of the project in hand.
Resources:-
Access Free White Papers on Business Process Management
Free Articles and Guides about IT Governance
Social Bookmarks