Agile project managers do not try to answer the question “how much will the project cost?”

As a Project Manager, you are repeatedly asked to reply to one simple yet tricky question: "how much will the project cost?".

Show

In a waterfall environment, the process is fairly straightforward, although not necessarily more accurate. The cost of the project is usually the result of an estimation process of each phase, with some buffer and a good deal of hope in the team being able to deliver within the available budget. Surprisingly the supplier is the one forecasting and paying for any mistakes in the forecast.

If Agile is the preferred approach, you will be likely asked a second question "how long it will take?". Both questions in an Agile scenario seems to defeat the very purpose of this methodology. But trying to explain the benefits of Agile won't make clients and sales less concerned about what will be delivered and by when.

If your projects are similar, and size and skillset of the team pretty levelled, historical data will probably provide some hints about the velocity you can expect. But how common is that scenario? Not very, and that is exactly why we resort to less traditional forecasting methods.

A Monte Carlo Analysis could be the ace upon your sleeve, as it allows project managers to use the uncertainty in the estimate of a project to determine the possible outcomes and their likelihood. At the end of the process, you will be able to express the potential cost/duration of a project with a certain level of confidence.

Requirements

The only requirement to run a Monte Carlo simulation is actually what will upset many Agile purists, as you need to estimate the effort for every task in your project - even if it doesn't prevent you from working with story points.

The team is not requested to provide accurate estimates; they just need to be accurate in their inaccuracy. Instead of providing a precise value for every task they shall provide ranges that they think the real effort will probably fall into. For instance:

  • Task #1: 3 to 5 days
  • Task #2: 8 to 13 days
  • Task #3: 5 to 8 days

The simulation

That list gave us a best case and a worst case scenario, respectively 16 and 26 days. But the chances for either of them to happen are only two on 72 (3 × 6 × 4). As well as any other sequence of numbers taken from those ranges has one possibility out of 72 to be accurate.

Now imagine to pick random numbers within the ranges, calculate the total and repeat the operation hundreds, thousands of times. That would give us thousands of possible outcomes for the project, but even if the sequence of numbers were different, many of these totals would be the same.

The outcome

Each of the totals occurred a specific number of times, in our simulation, and each of them represents a successful scenario compared to any other result smaller than or equal to that total. If, for example, we considered an estimate of 26 days and the project actually ended in 24 days, we would rate our forecast successful.

In the given example, after running the simulation thousands of time we might get the following outcomes (considering cumulative probability):

  • 16 days: successful 2% of times
  • 17 days: 6% of times
  • 18 days: 14% of times
  • 19 days: 27% of times
  • 20 days: 42% of times
  • 21 days: 59% of times
  • 22 days: 74% of times
  • 23 days: 87% of times
  • 24 days: 95% of times
  • 25 days: 99% of times
  • 26 days: 100% of times

that tells us that 23 days grants 87% of confidence (we usually pick the closest value to 90%) on a positive outcome, and thus we might feel comfortable with this number rather than just going for the worst case scenario.

Real world scenario

Ok, you understood how it works. Now what?

You can create your own Excel spreadsheet to run your simulation, following this advice. Most of the suggestions in the document also apply to Google Spreadsheet, in case you wanted more people to work on the same file in real time. They are effective but dramatically slow if you have tens of tasks and more than a few hundred iterations.

Conclusions

A Monte Carlo simulation might turn to be your best chance to get some reasonable estimates to provide your sales team with - maybe just don't spread the fact it's all about random numbers, they might get a bit nervous if they are not really into statistic...

  • Agile project managers do not try to answer the question how much will the project cost?”
    Access through your institution

Agile project managers do not try to answer the question how much will the project cost?”

Agile project managers do not try to answer the question how much will the project cost?”

Abstract

Context

The processes of estimating, planning and managing are crucial for software development projects, since the results must be related to several business strategies. The broad expansion of the Internet and the global and interconnected economy make Web development projects be often characterized by expressions like delivering as soon as possible, reducing time to market and adapting to undefined requirements. In this kind of environment, traditional methodologies based on predictive techniques sometimes do not offer very satisfactory results. The rise of Agile methodologies and practices has provided some useful tools that, combined with Web Engineering techniques, can help to establish a framework to estimate, manage and plan Web development projects.

Objective

This paper presents a proposal for estimating, planning and managing Web projects, by combining some existing Agile techniques with Web Engineering principles, presenting them as an unified framework which uses the business value to guide the delivery of features.

Method

The proposal is analyzed by means of a case study, including a real-life project, in order to obtain relevant conclusions.

Results

The results achieved after using the framework in a development project are presented, including interesting results on project planning and estimation, as well as on team productivity throughout the project.

Conclusion

It is concluded that the framework can be useful in order to better manage Web-based projects, through a continuous value-based estimation and management process.

Graphical abstract

This paper tries to answer the following question by identifying Agile practices and adapting them for being integrated into a coherent framework “Is it possible to define an Agile approach to estimate, plan and manage Web projects guided by business value”. The paper includes the results obtained from a real experience dealing with applying the proposed framework. To finish, it states relevant conclusion.

Agile project managers do not try to answer the question how much will the project cost?”
  1. Download : Download full-size image

Introduction

Starting a professional software development project soon raises some critical questions such as: How much will the project cost? When will it finish? How much effort must be invested in it? Will the investment be returned soon? What are the features our customers really need?

Being able to answer these questions and some others related to them is crucial for designing business strategies (e.g. financial or commercial, among others) from project results. The responses to the aforementioned questions must condition all decisions like starting one project or another, the type of product to develop and the money that must be invested in it.

It is well known that estimating and planning a development project is a compulsory and complex process [7], [18], [45]. To face this challenge, traditional estimation techniques focus on a predictive approach [1], [12], [13], [37], which requires a stable and familiar environment. Essentially, these techniques begin with a strong initial requirements gathering phase to freeze user needs [53]. This approach makes these methods especially sensitive to uncertainties and changes of customer needs.

Nowadays, the rise of the Internet and the actual global and interconnected economy has increased the needs for quickly adaptation to changing customer needs. These events have emerged in parallel with the acceptance of Web Engineering as a discipline in Software Engineering [24]. Web Engineering can be defined as a set of methods, techniques and tools in Software Engineering that helps a development team build up systems on the Web. There are several characteristics that differentiate Web projects from the rest of software development projects [24], [52]:

Complex navigational structure.

Critical interface requirements (such as unknown users or availability, among others).

Security aspects.

Increase on maintenance efficiency, avoiding downtimes.

Delivery as soon as possible.

Reduction of “time-to-market”.

Adaptation to quick-changing requirements.

It is important to highlight that some of the aforesaid characteristics are not exclusive of Web development projects and can also appear in non-Web projects. Nevertheless, the concurrence of all of them together at the same time can be identified as a Web project specificity.

In such environments, Agile software development methodologies, with constant monitoring and measurement, and frequent intervention mainly based on the use of empirical processes [61], are turning into a solid alternative for organizations developing software to plan and estimate Web projects [4]. These methodologies offer a suitable framework for the exposed Web development characteristics [55], like quick response to changes, adaptability and reduction of development time [31], [50]. In addition, as it has been mentioned, the classical approach regarding up-front requirements gathering demands a stable environment, not being the case of Web projects, where requirements change quicker. The incremental and iterative way of processing Agile methods requirements [18], [27] may better fit this particular case.

In contrast, the project management classical approach states that a project succeeds when it combines achieving the goals established on variables such as cost, schedule and scope [53]. Following these criteria, the Standish Group conducts the well-known CHAOS surveys to test the projects success [72]. They define a successful project as the one that is carried out on time, on budget and includes the originally specified features. Fig. 1 shows the projects success level depending on the type of methodology used [73].

As it can be noticed, the percentage of successful projects using Agile methods is significantly higher than that of projects using traditional approaches. These results can be associated with the improvements that Agile techniques bring to project management, for example, “just-in-time” planning, iterative requirements gathering or frequent collaboration. However, it has to be added that, in terms of the above definition concerning a project success, the classical approach leaves behind crucial aspects such as quality and delivered value to customers [33]. They are main issues to address on projects, since they are related to the functionality developed and the kind of process used. Thus, using techniques that allow us to better identify and measure the value delivered to users will improve the results of our projects. For this purpose, we suggest some techniques to take into account these variables.

Lastly, it must be kept in mind that the number of unused functionality represents expended resources that rarely return to the development organization. A survey conducted by the Standish Group [35], covering 2000 projects carried out by 1000 organizations, showed that more than half of the functionality developed on a project is hardly ever or never used. Fig. 2 shows the results of that survey.

As before stated, the Agile iterative and incremental approach can better fit the special needs of Web projects in order to particularly identify what should be built and when it should be built. This approach will earlier identify these changes and will cope with them more properly to avoid designing unneeded features. It will allow a higher return on the projects investments. Based on the foregoing, this work aims to cover the following objectives:

Proposing a framework, based on existing Agile methods, to estimate, plan and manage Web projects that, guided by the business value, will help to select what to build, estimating cost and adapting plans to a changing environment.

Presenting the results obtained from a real experience dealing with applying the proposed framework to a project developed for a Spanish public administration office.

Taking out the main lessons learned after applying the proposed framework, which will generalize successes and avoid failures as well as will present future lines of work.

This paper is organized into the following sections. Following this introduction, Section 2 presents the research scope, including also the research questions and methodology. Section 3 presents the related work, describing different approaches to estimate, plan and manage Web projects. Then, Section 4 provides an approach to the suggested framework to estimate, plan and manage Web projects based on Agile techniques, whereas Section 5 describes the experience of applying the proposed framework to a real project. To conclude, Section 6 states the conclusions taken out, interprets results and consequences and advances possible future lines of work.

Section snippets

Research questions, scope and method

The main question we will try to answer in order to achieve the objectives presented in the previous section is: “Is it possible to define an Agile approach to estimate, plan and manage Web projects guided by business value?” As it is very generic, we have tried to decompose it into the following research questions:

RQ1: What are the suitable existing techniques for estimating, planning and managing Agile Web projects?

RQ2: Can business value help estimation, plan and management of Agile Web

As mentioned in the preceding sections, this work integrates several research areas that have to be analyzed in order to present a right related work section. In fact, our approach is similar to other approaches regarding estimation models in Web Engineering paradigm under an Agile perspective. As this general context is too young, this section includes an overview of five contexts: estimation models in Software Engineering, solutions in Web Engineering for project estimations, solutions for

An approach to Web Engineering estimation, plan and management with Agile methods

As put forward in the previous section, there are several approaches to software development projects estimation and plan, coming from different fields (classical project management, Agile context or algorithm-based), but none of them individually considers all the special needs Web development projects entail together with a value-based perspective. In the introduction, we advanced that there is a set of characteristics that differentiate Web projects from classical software development

A practical example

This section describes a practical sample of the proposed framework, being our first empirical experience regarding the application of this framework. A single experience can or cannot validate it, although it can help to provide the first insights for our further research. Some of the results described in this section have been already presented in our previous paper [75]. That was focused on assessing the suitability of Agile practices in Public Administration and its adaptability to

Conclusions and future work

This section states the research conclusions, stressing the strengths and weaknesses of the proposed Agile framework, including relevant information resulting from applying it to a real project. These conclusions can provide valuable insights to guide the future work, although further research is needed.

As previously stated in Section 2, the general research question is structured into 3 research questions. Based on the results, some arguments can be provided to answer them, as shown below:

RQ1:

Acknowledgements

This research has been supported by the MeGUS Project (TIN2013-46928-C3-3-R) of the Ministry of Economy and Competitiveness, Spain. We would like to thank the Regional Ministry of Culture and Sports of Junta de Andalucía for allowing us to issue this data

References (78)

  • et al.

    The treatment of navigation in web engineering

    Adv. Eng. Softw.

    (2007)

  • T. Dybå et al.

    Empirical studies of agile software development: a systematic review

    InfSoftwTechnol

    (2008)

  • S. Ceri et al.

    Web modelling language (WebML): a modelling language for designing web sites

    Comput. Networks

    (2000)

  • A.J. Albrecht et al.

    Software function, source lines of codes, and development effort prediction: a software science validation

    IEEE TSE

    (1983)

  • G. Alleman, et al., Making agile development work in a government contracting environment – using earned value to...
  • Agile COCOMO II. <http://sunset.usc.edu/cse/pub/research/AgileCOCOMO/AgileCOCOMOII/Main.html> (accessed...
  • S.W. Ambler

    Lessons in Agility from Internet-Based Development

    (2002)

  • S. Ambler, Generalizing Specialists: Improving Your IT Career Skills....
  • J. Appelo

    Management 3.0. Leading Agile Developers, Developing Agile Leaders

    (2009)

  • J. Armario, J.J. Gutiérrez, M. Alba, J.A. García-García, J. Vitorio, M.J. Escalona, Project estimation with NDT, in:,...

  • K. Beck, et al., Manifesto for Agile Software Development. <http://www.agilemanifesto.org>, 2001 (accessed...
  • K. Beck

    Test-Driven Development by Example

    (2002)

  • K. Beck et al.

    Extreme Programming Explained: Embrace Change

    (2004)

  • A. Begel, N. Nagappan, Usage and perceptions of agile software development in an industrial context: an exploratory...
  • B.W. Boehm

    Software Engineering Economics

    (1981)

  • B.W. Boehm et al.

    Software Cost Estimation with COCOMO II

    (2000)

  • A. Cockburn

    Crystal Clear: A Human-Powered Methodology for Small Teams

    (2004)

  • A. Cockburn, Balancing Lightness with Sufficiency. <http://alistair.cockburn.us/Balancing+lightness+with+sufficiency>,...
  • M. Cohn

    User Stories Applied: For Agile Software Development

    (2004)

  • M. Cohn

    Agile Estimating and Planning

    (2005)

  • M. Cohn

    Succeeding with Agile Using Scrum

    (2009)

  • R. Corral, Gestión ágil de proyectos: experiencias prácticas y resultados obtenidos....
  • J.W. Creswell, Research Design: Qualitative, Quantitative, and Mixed Method Approaches, second ed., SAGE,...
  • Decreto 1/1979, de 30 de julio, sobre creación y publicación del Boletín Oficial de la Junta de Andalucía....
  • Decreto 205/1983, de 5 de octubre, por el que se aprueba el Reglamento del Boletín Oficial de la Junta de Andalucía....
  • Y. Deshpande et al.

    Web engineering

    J. Web Eng.

    (2002)

  • E. Derby et al.

    Agile Retrospectives. Making Good Teams Great

    (2006)

  • S. Downey, J. Sutherland, Scrum metrics for hyperproductive teams: how they fly like fighter aircraft, in: HICSS 2012,...
  • M.J. Escalona et al.

    NDT: a model-driven approach for web requirements

    IEEE Trans. Softw. Eng.

    (2008)

  • R. Fewster, E. Mendes, Measurement, Prediction and Risk Analysis for Web Applications, in: Proceedings of Metrics ’01,...
  • H. Glazer, J. Dalton, D. Anderson, M. Konrad, S. Dhrum, CMMI or agile: why not embrace both!, in: CMU/SEI-2008-TN-003,...
  • D. Gray

    Gamestorming. A Playbook for Innovators, Rulebreakers and Changemakers

    (2010)

  • J. Highsmith

    Agile Project Management: Creating Innovative Products

    (2009)

  • JIRA Website. <https://www.atlassian.com/es/software/jira> (accessed...
  • J. Johnson (Ed.), XP2002, Report to the Third International Conference on Extreme Programming and Agile Processes in...
  • M. Jorgensen

    Practical Guidelines for Expert Judgment-Based Software Effort Estimation

    (2005)

  • Gustav Karner, Resource Estimation for Objectory Projects, Objective Systems SF AB,...
  • N. Koch et al.

    UML-Based Web Engineering: An Approach Based on Standards

    (2008)

  • N. Koch, A. Knapp, S. Kozuruba, Assessment of effort reduction due to model-to-model transformations in the web domain,...
  • Cited by (45)

    View full text

    Copyright © 2015 Elsevier B.V. All rights reserved.

    Does the project manager still exist in agile software development projects?

    Agile teams are not meant to have project managers. Instead, agile methods such as Scrum and XP define roles such as product owner, scrum master, and coach.

    Which of the following is a basic value of agile software development?

    The four core values of Agile software development as stated by the Agile Manifesto are: individuals and interactions over processes and tools; working software over comprehensive documentation; customer collaboration over contract negotiation; and.

    What technique is used to control the scope in an agile project?

    What technique is used to control the scope in an Agile project? Keep a prioritized list of required functions. Project Quality Management is different for Agile projects because they place heavy emphasis in what additional area? The quality of the project processes.

    Which two of the following questions decision points are most important when monitoring project progress?

    Which two of the following questions (decision points) are important when monitoring project progress? Is the task on target? ​Is the variance significant?