Mapping metrics for agile project performance
systematic literature review
DOI:
https://doi.org/10.23925/2179-3565.2023v14i4p132-162Keywords:
Software Development, Agile and Traditional Metrics, Scrum, KanbanAbstract
In a competitive context where organizations need to deliver products or services through agile project management, a process in which metrics become important inputs for project control and monitoring. What is sought in this article is to understand whether there are organizations that, despite conducting agile projects, use project control through the use of traditional metrics. In addition, it seeks to understand whether the use of predictive project metrics hinder the vision and work of corporate leadership to make decisions in agile projects. The objective of this article was to map which are the metrics used to measure the performance of traditional and agile projects. As a methodological strategy, a Systematic Literature Review was adopted to assist in mapping and evaluating a specific intellectual structure to develop a body of knowledge. For data collection, the Web of Science and Scopus databases were used. 83 articles were found and the results showed that traditional projects continue to control projects with traditional metrics, such as Earned Value Management (EVM). In agile projects, the most common metrics are metrics associated with product backlog, delivery and product quality. This study contributes to other studies that wish to identify the metrics used between the different project management approaches and guide new research for future work.
References
Abdelrahman Aljemabi, M. et al. (2020). Mining social collaboration patterns in developer social networks. IET Software, 14(7), 839-849.
Ahmed, A. R. et al. (2017). Impact of story point estimation on product using metrics in scrum development process. Intern. Journal of Advanced Computer Science and Applications, 8(4).
Aljedaibi, W. & Khamis, S.(2019).Towards Measuring the Project Management Process During Large Scale Software System Implementation Phase.The ISC Intern. Journal of Info. Security,11(3),161-172.
Antinyan, V. (2014). Monitoring evolution of code complexity and magnitude of changes. Acta Cybernetica, 21(3), 367-382.
Asha, N. & Mani, P. (2018). Knowledge-based Acceptance Test driven agile Approach for Quality Software Development. International Journal of Resent Tech. and Engineering, 7.
Athanasiou, D. et al. (2014). Test code quality and its relation to issue handling performance. IEEE Transac. on Software Eng, 40(11),1100-1125.
Azzeh, M.& Nassif, A. B. (2016). A hybrid model for estimating software project effort from Use Case Points. Applied Soft Computing, 49, 981-989.
Bakhtiary, V. et al. (2020).The effectiveness of test-driven development approach on software projects: A multi-case study.Bulletin of Electrical Eng and Info.,9(5),2030-2037.
Batselier, J.& Vanhoucke, M. (2015). Evaluation of deterministic state-of-the-art forecasting approaches for project duration based on earned value management. International Journal of Project Management, 33(7), 1588-1596.
Budacu, E. N.& Pocatilu, P. (2018). Real Time Agile Metrics for Measuring Team Performance. Informatica Economica, 22(4), 70-79.
Chang, C. P. (2015). Software risk modeling by clustering project metrics. International Journal of Software Engineering and Knowledge Engineering, 25(06), 1053-1076.
Chen, H. L. (2018). Early prediction of project duration: A longitudinal study. Engineering Management Journal, 30(3), 191-202.
Chen, H. L. (2021). Early identification of distressed capital projects: A longitudinal approach. International Journal of Managing Projects in Business.
Chen, H. L. (2014). Improving forecasting accuracy of project earned value metrics: Linear modeling approach. Journal of Management in Engineering, 30(2), 135-145.
Chen, H. L., Chen, W. T.& Lin, Y. L. (2016). Earned value project management: Improving the predictive power of planned value. International Journal of Project Management, 34(1), 22-29.
Chen, Z. et al. (2020).A Bayesian approach to set the tolerance limits for a statistical project control method. International Journal of Production Research,58(10), 3150-3163.
Choraś, M.et al. (2020). Measuring and improving agile processes in a small-size software development company. IEEE access, 8, 78452-78466.
Choudhury, I. (2019). Agile Methods for Engineering. In Management for Scientists. Emerald Publishing Limited.
Costa, J., Vasconcelos, A.& Fragoso, B. (2020). An Enterprise Architecture Approach for Assessing the Alignment Between Projects and Goals. International Journal of Information Technology Project Management (IJITPM), 11(3), 55-76.
Colin, J. et al. (2015). A multivariate approach for top-down project control using earned value management. Decision Support Systems, 79, 65-76.
Colin, J.& Vanhoucke, M. (2014). Setting tolerance limits for statistical project control using earned value management. Omega, 49, 107-122.
Colin, J. et al. (2015). A comparison of the performance of various project control methods using earned value management systems.Expert Systems with Applications,42(6),3159-3175.
Concas, G. et al. (2012). An empirical study of software metrics for assessing the phases of an agile project. International Journal of Software Engineering and Knowledge Engineering, 22(04), 525-548.
Day, F. C. et al. (2019). Feasibility study of an EHR-integrated mobile shared decision making application. International journal of medical informatics, 124, 24-30.
Dennehy, D.& Conboy, K. (2018). Identifying challenges and a research agenda for flow in software project management. Project Management Journal, 49(6), 103-118.
Dias, K. R. S. et al. (2021).Hybrid Project Management Method for managing ICT project’s scope: a case study in a Brazilian company. Brazilian Journal of Develop.,7(8),75984-76014.
El Asmar, M. et al. (2013). Quantifying performance for the integrated project delivery system as compared to established delivery systems. Journal of construction engineering and management, 139(11), 04013012.
Fleche, D. et al. (2017). Collaborative project: Evolution of computer-aided design data completeness as management information. Concurrent Engineering, 25(3), 212-228.
Gao, K. et al. (2011).Choosing software metrics for defect prediction: an investigation on feature selection techniques.Software:Practice and Exp.,41(5),579-606.
Geng, S. et al. (2018). Knowledge contribution as a factor in project selection. Project Management Journal, 49(1), 25-41.
Glenwright, E. (2007). A survey of the 30 most serious flaws in scheduling. In PMICoS 2007 Annual Conference.
Grau, D.& Back, W. E. (2015). Predictability index: Novel metric to assess cost and schedule performance. Journal of Construction Eng. and Management, 141(12), 04015043.
Gren, L., Goldman, A.& Jacobsson, C. (2020). Agile ways of working: a team maturity perspective. Journal of Software: Evolution and Process, 32(6), e2244.
Grimaldi, P. et al. (2016). An agile, measurable and scalable approach to deliver software applications in a large enterprise. Intern. Journal of Agile Systems and Manag. 9(4),326-339.
Han, S., Choi, J. O.& O’Connor, J. T. (2017). Quality of baseline schedules: Lessons from higher education capital facility projects. Journal of Professional Issues in Engineering Education and Practice, 143(1), 04016017.
Hanna, A. S. (2016). Benchmark performance metrics for integrated project delivery. Journal of Construction Engineering and Management, 142(9), 04016040.
Hanna, A. S. et al. (2012). Request for information: benchmarks and metrics for major highway projects. Journal of construction engineering and management, 138(12), 1347-1352.
Hazır, Ö. (2015). A review of analytical models, approaches and decision support tools in project monitoring and control. International Journal of Project Manag. 33(4), 808-815.
Hernández, G. et al. (2019). Productivity Metrics for an Agile Software Develop. Team: A Systematic Review. TecnoLógicas, 22(SPE), 63-81.
Ibba, S. et al. (2018). Initial coin offerings and agile practices. Future Internet, 10(11), 103.
Ibrahim, M. W. et al. (2020). Quantitative comparison of project performance between project delivery systems. Journal of management in engineering, 36(6), 04020082.
Iqbal, S. et al. (2012). Yet another set of requirement metrics for software projects.
Isong, B.& Obeten, E. (2013). A systematic review of the empirical validation of object-oriented metrics towards fault-proneness prediction. International Journal of Software Engineering and Knowledge Engineering, 23(10), 1513-1540.
Jethani, K. (2013). Software metrics for effective project management. International Journal of System Assurance Engineering and Management, 4(4), 335-340.
Junior, V. B. et al. (2019).Elaboração de um Plano de Gerenciamento de Projeto para Implantação de Laboratório de Metrologia Utilizando o Guia PMBOK 5°Edição.Cippus,7(2),127-145.
Jyothi, V. E.& Rao, K. N. (2017). Effective Implementation of Agile Software Development with a Framework, Metric Tool, and in Association with Cloud and Lean Kanban. International Journal of Advanced Eng. Research and Science, 4(3), 237085.
Kamulegeya, G. et al. (2018). Measurements in the early stage software start-ups: a multiple case study in a nascent ecosystem. Found. of Computing and Decision Sciences,43(4),251-280.
Kayes, I. et al. (2016). Product backlog rating: a case study on measuring test quality in scrum. Innovations in Systems and Software Eng. 12(4), 303-317.
Kerkhove,L.P. & Vanhoucke, M.(2017).Extensions of earned value management: Using the earned incentive metric to improve signal quality. Intern. Journal of Project Management,35(2),148-168.
Kristiansen, J. N.& Ritala, P. (2018). Measuring radical innovation project success: typical metrics don’t work. Journal of Business Strategy.
Kim, T., Kim, Y. W.& Cho, H. (2016). Customer earned value: performance indicator from flow and value generation view. Journal of management in Eng. 32(1),04015017.
Ko, B. S.& Cho, M. S.(2020).Evaluating the relative efficiency of defense R&D projects in a multi-project environment using EVM and CAIV measures. ICIC Express Letters, Part B: Applications,11(7), 675-682.
Kock, A.& Gemünden, H. G. (2019). Project lineage management and project portfolio success. Project Management Journal, 50(5), 587-601.
Yamashita, A.& Counsell, S. (2013). Code smells as system-level indicators of maintainability: An empirical study. Journal of Systems and Software, 86(10), 2639-2653.
Yousefi, N. et al. (2019). Using statistical control charts to monitor duration-based performance of project. arXiv preprint arXiv:1902.02270.
Lappi, T. M. et al. (2019). Project governance and portfolio management in government digitalization. Transforming Government: People, Process and Policy, 13(2), 159-196.
Mas, A. et al. (2020). Supporting the deployment of ISO-based project manag. processes with agile metrics. Computer Standards & Interfaces, 70, 103405.
Mascarenhas Hornos da Costa, J., Oehmen, J., Rebentisch, E.& Nightingale, D. (2014). Toward a better comprehension of Lean metrics for research and product development management. R&D Management, 44(4), 370-383.
Mäkiaho, P. et al. (2022). MMT: a tool for observing metrics in software projects. In Research Anthology on Agile Software, Software Develop and Testing (pp. 1077-1089). IGI Global.
Meidan, A. et al. (2018). Measuring software process: a systematic mapping study. ACM Computing Surveys (CSUR), 51(3), 1-32.
Narbaev, T. et al. (2014). Combination of growth model and earned schedule to forecast project cost at completion. Journal of Construction engineering and management, 140(1), 04013038.
Nidagundi, P.& Novickis, L. (2016). Introduction to Lean Canvas Transformation Models and Metrics in Software Testing. Appl. Comput. Syst., 19(1), 30.
Orgut, R. E. et al. (2018). Metrics that matter: Core predictive and diagnostic metrics for improved project controls and analytics. Journal of Construction Engineering and Management, 144(11), 04018100.
Pajares, J.& Lopez-Paredes, A. (2011). An extension of the EVM analysis for project monitoring: The Cost Control Index and the Schedule Control Index. International Journal of Project Management, 29(5), 615-621.
Parrend, P. et al. (2014, May). Swarm Projects: Beyond the Metaphor. In International Conference on Swarm Intelligence Based Optimization (pp. 131-138). Springer, Cham.
Pavlova, N. et al. (2021). Creating the Agile Model to Manage the Activities of Project Oriented Transport Companies. Eastern-European Journal of Enterprise Tech, 1(3), 109.
Perkusich, M. et al. (2017). Assisting the continuous improvement of Scrum projects using metrics and Bayesian networks. Journal of Software: Evolution and Process, 29(6), e1835.
Petersen, K.& Wohlin, C. (2011). Measuring the flow in lean software development. Software: Practice and experience, 41(9), 975-996.
Pradhan, S.& Nanniyur, V. (2021). Large scale quality transformation in hybrid development organizations–A case study. Journal of Systems and Software, 171, 110836.
Prakash B. et al. (2018). Ontology based risks management model for agile software development. Journal of Advanced Research in Dynamical and Control Systems, 10(11 Special Issue):319–330.
Pressman, R. S.& Maxim, B. R. (2021). Engenharia de software-9. McGraw Hill Brasil.
Project Management Institute (PMI) (2021): A Guide to the Project Management Body of Knowledge (PMBOK Guide), PMI, Newtown Square, PA, 2021.
Pozzana, I. et al. (2021). Spreading of performance fluctuations on real-world project networks. Applied Network Science, 6(1), 1-15.
Rajablu, M. et al. (2017). Managing for stakeholders: Introducing stakeholder metrics-integrated model to lead project ethics and success. International Journal of Project Organisation and Management, 9(1), 31-56.
Rajagopalan, J. et al. (2018). Introduction of a new metric “Project Health Index”(PHI) to successfully manage IT projects. Journal of Organizational Change Management.
Ries, E. (2011). The Lean Startup: How today's entrepreneurs use continuous innovation to create radically successful businesses. New York: Crown Business.
Royce, W. (1970). The software lifecycle model(waterfall model). In Proc. Westcon, V.314.
Sampaio, R. F.& Mancini, M. C. (2007). Systematic review studies: a guide for careful synthesis of the scientific evidence. Brazilian Journal of Physical Therapy, 11, 83-89.
Sato, T.& Hirao, M. (2013). Optimum budget allocation method for projects with critical risks. International Journal of Project Management, 31(1), 126-135.
Savola, R. et al. (2012). Risk-driven security metrics in agile software development: An industrial pilot study. Journal of Universal Computer Science, 18(12), 1679-1702.
Sadler, H. (2020). ER2C SDMLC: enterprise release risk-centric systems development and maintenance life cycle. Software Quality Journal, 28(4), 1755-1787.
Sajedi‐Badashian,A. & Stroulia,E.(2020).Investigating the info. value of different sources of evidence of developers’ expertise for bug assignment in open‐source projects. IET Software,14(7),748-758.
Sharma, V. et al. (2021). Development of metrics and an external benchmarking program for healthcare facilities. International Journal of Construction Management, 21(6), 615-630.
Shawky, D. M.& Abd‐El‐Hafiz, S. K. (2016). Characterizing software development method using metrics. Journal of Software: Evolution and Process, 28(2), 82-96.
Staron, M. & Meding, W. (2016). Mesram–a method for assessing robustness of measurement programs in large software development organizations and its industrial evaluation. Journal of Systems and Software, 113, 76-100.
Singh, J., Singh, K.& Singh, J. (2019). Reengineering framework to enhance the performance of existing software. System, 1139, 120-3.
Simpson, J. R. et al. (2021). Optimizing the electronic health record: An inpatient sprint addresses provider burnout and improves electronic health record satisfaction. Journal of the American Medical Informatics Association, 28(3), 628-631.
Spijkman, T. et al. (2021). Alignment and granularity of requirements and architecture in agile development: A functional perspective. Information and Software Technology, 133, 106535.
Sommerville, I.(2007).Eng. de Software, 9. Ed. Pearson, Addison Wesley,8(9),10,p.44-45.
Song, J., Martens, A.& Vanhoucke, M. (2021). Using schedule risk analysis with resource constraints for project control. European Journal of Operational Research, 288(3), 736-752.
Sutherland, J. (2019). Scrum: A arte de fazer o dobro do trabalho na metade do tempo. Rio de Janeiro: Sextante, 2019.
Tahir, T. et al. (2018). A systematic mapping study on software measurement programs in SMEs. E-Informatica Software Engineering Journal, 12(1).
Tanveer, B. et al. (2019). An evaluation of effort estimation supported by change impact analysis in agile software develop. Journal of Software: Evolution and Process, 31(5), e2165.
Tranfield,D. et al. (2003). Towards a methodology for developing evidence‐informed management knowledge by means of systematic review.British journal of management,14(3),207-222.
Wallace, L. G.& Sheetz, S. D. (2014). The adoption of software measures: A technology acceptance model (TAM) perspective. Information & Management, 51(2), 249-259.
Wauters, M.& Vanhoucke, M. (2015). Study of the stability of earned value management forecasting. Journal of Construction Engineering and Management, 141(4), 04014086.
Van Casteren, W. (2017). The Waterfall Model and the Agile Methodologies: A comparison by project characteristics. Research Gate, 1-6.
Vanhoucke, M. (2011). On the dynamic use of project performance and schedule risk information during projecttracking. Omega, 39(4), 416-426.
Vanhoucke, M. (2019). Tolerance limits for project control: An overview of different approaches. Computers & Industrial Engineering, 127, 467-479.
Üsfekes, Ç. et al. (2019). Auction‐based serious game for bug tracking. IET Software, 13(5), 386-392.
Zheng, M. et al. (2021). Key performance indicators for the integration of the service-oriented architecture and scrum process model for IOT. Scientific Programming, 2021.
Downloads
Published
Issue
Section
License
This Journal is licensed under a Creative Commons Attribution-Non Commercial-No Derivers 4.0 International license.
1.The author (s) authorize the publication of the article in the journal;
2.The author (s) warrant that the contribution is original and unpublished and is not in the process of being evaluated in other journal (s);
3. The journal is not responsible for the opinions, ideas and concepts emitted in the texts, as they are the sole responsibility of its author (s);
4. The editors are entitled to make textual adjustments and to adapt the articles to the standards of publication.
