THE SOFTWARE PROCESS ASSESSMENT AND CERTIFICATION: APPLICATION OF THE ANALYTIC HIERARCHY PROCESS FOR PRIORITY DETERMINATION
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Software certification involves assessing and certifying the quality of the software process based on multiple evaluation criteria where each criterion has different importance values on the quality of the software. However, the different importance values of the evaluation criteria have not been addressed in the existing software process certification models. A systematic technique is needed to ensure that the certification results are consistent, accurate and not made arbitrarily. To address this issue, the Extended Software Process Certification (ESPAC) model was introduced by adopting the Analytic Hierarchy Process (AHP) technique to determine the priorities of the evaluation criteria. There were three main phases in this study: (a) theoretical study, (b) expert review and (c) focus group discussion. Ultimately, a mutual agreement was achieved about the evaluation criteria and the AHP was shown to be a suitable technique to be employed in software process assessment and certification. Furthermore, the acquired priorities were used as the ideal priorities for the ESPAC model, which can be used by assessors during the assessment and certification process. The outcome of this study benefits researchers in the AHP and software process assessment fields.
How to Cite
Downloads
##plugins.themes.bootstrap3.article.details##
software engineering, software certification, ESPAC Model, weight values, Analytic Hierarchy Process
Acuna, S. T., Antonio, A. D., Ferre, X., Lopez, M., & Mate, L. (2000). The software process: Modeling, evaluation and improvement. In Chang, S. K. Handbook of software engineering and knowledge engineering (pp. 193-237). River Edge: World Scientific Publishing Co. Pte. Ltd. Doi: https://doi.org/10.1142/9789812389718_0011
Agile Manifesto. (2001). Retrieved from www.Agilemanifesto.org.
Ahimbisibwe, A., Daellenbach, U., & Cavana, R. Y. (2017). Empirical comparison of traditional plan-based and agile methodologies. Journal of Enterprise Information Management, 30(3), 400-454. Doi: 10.1108/JEIM-06-2015-0056
Akbar, M. A., Sang, J., Khan, A. A., Shafiq, M., Hussain, S., Hu, H., & Xiang, H. (2017). Improving the quality of software development process by introducing a new methodology–AZ-model. IEEE Access, 6, 4811-4823. Doi: 10.1109/ACCESS.2017.2787981
Alaidaros, H., & Omar, M. (2017). Software project management approaches for monitoring work-in-progress: A review. Journal of Engineering and Applied Sciences, 12(15), 3851-3857.
Aldahmash, A., Gravell, A. M., & Howard, Y. (2017). A review on the critical success factors of agile software development. European Conference on Software Process Improvement, 504-512. Springer, Cham. Doi: 10.1007/978-3-319-64218-5_41
Ali, Y., Butt, M., Sabir, M., Mumtaz, U., & Salman, A. (2018). Selection of suitable site in Pakistan for wind power plant installation using analytic hierarchy process (AHP). Journal of Control and Decision, 5(2), 117-128. Doi: 10.1080/23307706.2017.1346490
Al-Tarawneh, F. H. (2014). A framework for COTS software evaluation and selection for COTS mismatches handling and non-functional requirements. (Unpublished doctoral dissertation). Universiti Utara Malaysia, Kedah, Malaysia.
Animah, I., & Shafiee, M. (2019). Maintenance strategy selection for critical shipboard machinery systems using a hybrid AHP-PROMETHEE and cost benefit analysis: A case study. Journal of Marine Engineering & Technology, 1-12. Doi: 10.1080/20464177.2019.1572705
Ansari, M. T. J., Pandey, D., & Alenezi, M. (2018). Store: Security threat oriented requirements engineering methodology. Journal of King Saud University-Computer and Information Sciences, 34(2), 191-203. Doi: 10.1016/j.jksuci.2018.12.005
Baharom, F., Yahya, J., Deraman, A., & Hamdan, A. R. (2011). SPQF: Software Process Quality Factor for software process assessment and certification. International Conference on Electrical Engineering and Informatics, 1-7. Doi: 10.1109/ICEEI.2011.6021526
Baharom, F. (2008). A software certification model based on development process quality assessment. (Unpublished doctoral dissertation). Universiti Kebangsaan Malaysia, Selangor, Malaysia.
Baharom, F., Deraman, A., & Hamdan, A. R. (2005). A survey on the current practices of software development process in Malaysia. Journal of ICT, 4, 57-76.
Baidya, R., Dey, P. K., Ghosh, S. K., & Petridis, K. (2018). Strategic maintenance technique selection using combined quality function deployment, the analytic hierarchy process and the benefit of doubt approach. The International Journal of Advanced Manufacturing Technology, 94(1-4), 31-44. Doi: 10.1007/s00170-016-9540-1
Behkamal, B., Kahani, M., & Akbari, M. K. (2009). Customizing ISO 9126 quality model for evaluation of B2B applications. Information and Software Technology, 51(3), 599-609. Doi: 10.1016/j.infsof.2008.08.001
Chen, J. K., Pham, V. K., & Yuan, B. J. (2013). Adopting AHP approach on evaluation and selection of outsourcing destination in East and Southeast Asia. Technology Management in the IT-Driven Services (PICMET), 528-537. Retrieved from http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6641715
Chow, T., & Cao, D. B. (2008). A survey study of critical success factors in agile software projects. Journal of Systems and Software, 81(6), 961-971. Doi: 10.1016/j.jss.2007.08.020Get
Ciancarini, P., Missiroli, M., & Sillitti, A. (2019). Preferred tools for agile development: A sociocultural perspective. International Conference on Objects, Components, Models and Patterns, 43-58. Springer, Cham. Doi: 10.1007/978-3-030-29852-4_3
CMMI Institute. (2018). CMMI Adoption and Transition Guidance V2.0. Retrieved from https://cmmiinstitute.com/getattachment/5868888b-5f37-4715-bc8b-c43250ec0abc/attachment.aspx
ComputerWorldUKStaff. (2020, Feb 17). Top software failures in recent history. ComputerWorld. Retrieved from https://www.computerworld.com/article/3412197/top-software-failures-in-recent-history.html#slide1
Crostack, H. A., Hackenbroich, I., Refflinghaus, R., & Winter, D. (2007). Investigations into more exact weightings of customer demands in QFD. Asian Journal on Quality, 8(3), 71-80. Doi: http://dx.doi:org/10.1108/15982688200700026
Daneva, M., & Ahituv, N. (2011). What practitioners think of inter-organizational ERP requirements engineering practices: Focus group results. International Journal of Information System Modeling and Design, 2(3), 49-74. Doi: 10.4018/jismd.2011070103
Darwish S. M. (2016). Software test quality rating: A paradigm shift in swarm computing for software certification. Knowledge-Based Systems, 10, 167-175. Doi: 10.1016/j.knosys.2016.07.022
Deming, W. (1982). Out of the crisis. Cambridge, MA: MIT Center for Advanced Engineering Study.
Destefanis, G., Ortu, M., Counsell, S., Swift, S., Marchesi, M., & Tonelli, R. (2016). Software development: Do good manners matter?. PeerJ Computer Science, 2, e73. Doi: 10.7717/peerj-cs.73.
Dyba, T., Dingsoyr, T., & Moe, N. B. (2014). Agile project management. In Ruhe, G. & Wohlin, C. (Eds.), Software Project Management in a Changing World (pp. 277-300). Springer Berlin Heidelberg.
Ferreira, G., Kästner, C., Sunshine, J., Apel, S., & Scherlis, W. (2019). Design Dimensions for Software Certification: A Grounded Analysis. arXiv preprint arXiv:1905.09760.
Galin, D. (2004). Software quality assurance. England: Pearson Education Limited.
Gasston, J. L. (1996). Process improvement: An alternative to BPR for software development organizations. Software Quality Journal, 5, 171-183. Doi: https://doi.org/10.1007/bf00678582
Gualo, F., Caballero, I., & Rodriguez, M. (2020). Towards a software quality certification of master data-based applications. Software Quality Journal, 28, 1-24. Doi: 10.1007/s11219-019-09495-w
Hallowell, M. R., & Gambatese, J. A. (2010). Qualitative research: Application of the Delphi method to CEM research. Journal of Construction Engineering and Management, 136(1), 99-107. Doi: 10.1061/_ASCE_CO.1943-7862.0000137
Hambali, A., & Rahman, M. A. (2017). Application of integrated AHP and TOPSIS techniques for determining the best Fresh Fruit Bunches (FFB). Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 9(3), 145-149.
Heck, P., Klabbers, M., & Eekelen, M. (2010). A software product certification model. Software Quality Journal, 18(1)37-55. Doi: 10.1007/s11219-009-9080-0
Heikkilä, V. T., Damian, D., Lassenius, C., & Paasivaara, M. (2015). A mapping study on requirements engineering in agile software development. 41st Euromicro Conference on Software Engineering and Advanced Applications, 199-207. IEEE.
Hsiao, S. W. (2002). Concurrent design method for developing a new product. International Journal of Industrial Ergonomics, 29(1), 41-55. Doi: 10.1016/S0169-8141(01)00048-8
Humphrey, W. (1989). Managing the software process. Mass: Addison-Wesley.
Ishizaka, A., & Labib, A. (2011). Review of the main developments in the Analytic Hierarchy Process. Expert Systems with Applications, 38(11), 14336-14345. Doi: 10.1016/j.eswa.2011.04.143
Jones, C., & Bonsignour, O. (2012). The economics of software quality. Boston: Pearson Education.
Jung, H. W. (2001). Rating the process attribute utilizing AHP in SPICE‐based process assessments. Software Process: Improvement and Practice, 6(2), 111-122. Doi: 10.1002/spip.139
Komuro, M., & Komoda, N. (2008). An explanation model for quality improvement effect of peer reviews. International Conference on Computational Intelligence for Modelling Control & Automation, 1159-1164. Doi: 10.1109/CIMCA.2008.187
Kontio, J., Bragge, J., & Lehtola, L. (2008). The focus group method as an empirical tool in software engineering. In Shull, F. et al. (Eds.) Guide to advanced empirical software engineering (pp. 93-116). Springer, London.
Kumar, U., Kumar, N., Mishra, V. N., & Jena, R. K. (2019). Soil quality assessment using Analytic Hierarchy Process (AHP): A case study. In Sugumaran, V. (Ed.), Interdisciplinary approaches to information systems and software engineering (pp. 1-18). IGI Global. Doi: 10.4018/978-1-5225-7784-3.ch001
Kumari, N. D., & Shylaja, B. S. (2019). AMGRP: AHP-based multimetric geographical routing protocol for urban environment of VANETs. Journal of King Saud University-Computer and Information Sciences, 31(1), 72-81. Doi: 10.1016/j.jksuci.2017.01.001
Kunda, D. (2003). STACE: Social technical approach to COTS software evaluation. In Cechich, A., Piayyini, M., & Vallecillo (Eds.), A. component-based software quality (pp. 64-84). Berlin Heidelberg: Springer-Verlag.
Lai, V. S., Wong, B. K., & Cheung, W. (2002). Group decision making in a multiple criteria environment: A case using AHP in software selection. European Journal of Operational Research, 137, 134-144. Doi: 10.1016/S0377-2217(01)00084-4
Lee, G., & Xia, W. (2010). Toward agile: An integrated analysis of quantitative and qualitative field data on software development agility. MIS Quarterly, 34(1), 87-114. Doi: 10.2307/20721416
Liamputtong, P. (2011). Focus group methodology principles and practices. London: Sage Publication.
Machuca-Villegas, L., Gasca-Hurtado, G. P., Tamayo, L. M. R., & Puente, S. M. (2020). Social and human factor classification of influence in productivity in software development teams. European Conference on Software Process Improvement, 717-729. Springer, Cham. Doi: 10.1007/978-3-030-56441-4_54
Marjani, M. E., Soh, K. G., Majid, M., Mohd Sofian, O. F., Nur Surayyah, M. A., & Mohd Rizam, A. B. (2012). Usage of group decision making approach in karate agility test selection. International Symposium on the Analytic Hierarchy Process, 1-11. Retrieved from http://www.isahp.org/uploads/59.pdf
Martakis, A., & Daneva, M. (2013). Handling requirements dependencies in Agile projects: A focus group with Agile software development practitioners. Seventh International Conference on Research Challenges in Information Science, 1-11. Doi: 10.1109/RCIS.2013.6577679
Mas, A., Fluxa, B., & Amengual, E. (2012). Lessons learned from an ISO/IEC 15504 SPI programme in a company. Journal of Software: Evolution and Process, 24(5), 493-500. Doi: 10.1002/smr.501
Mazza, R., & Berre, A. (2007). Focus group methodology for evaluating information visualization techniques and tools. 11th International Conference Information Visualization, 74-80. Doi: 10.1109/IV.2007.51
Meier, S. R. (2017). Technology Portfolio Management for Project Managers. Retrieved from https://www.pmiwdc.org/sites/default/files/presentations/201703/PMIW_LocalCommunity_Tysons_presentation_2017-02.pdf
Mu, E., & Pereyra-Rojas, M. (2017). Understanding the analytic hierarchy process. In Mu and Pereya-Rojas (Eds.), Practical decision making (pp. 7-22). Springer, Cham.
Mokhtar, M. R., Abdullah, M. P., Hassan, M. Y., & Hussin, F. (2017). Comparative study of multiple criteria decision making methods for selecting the best demand side management options. ELEKTRIKA-Journal of Electrical Engineering, 16(1), 11-16. Doi: 10.11113/elektrika.v16n1.15
Moradi, N. (2022). Performance evaluation of university faculty by combining BSC, AHP, and TOPSIS: from the students’perspective. International Journal of the Analytic Hierarchy Process, 14(2), 1-29. Doi: https://ijahp.org/index.php/IJAHP/article/view/915
Morgan, D. L. (1998). Planning focus groups. Thousand Oaks: Sage Publications.
Nath, U. K., Jagadev, A. K., & Pattnaik, P. K. (2020). Achieving target schedule time faster through Agile simulation model. Journal of Critical Reviews, 7(19), 4080-4088. Doi: 10.31838/jcr.07.19.476
Packeer Mohamed, S. F., Baharom, F., & Deraman, A. (2015). ESPAC model: Extended Software Process Assessment and Certification model for Agile and secure software processes. ARPN Journal of Engineering and Applied Sciences, 10(3), 1364-1373.
Padumadasa, E. U., Colombo, S., & Rehan, S. (2009). Investigation into decision support systems and multiple criteria decision making to develop a web-based tender management system. International Symposium on the Analytic Hierarchy Process, 1-17. Retrieved from http://www.isahp.org/2009Proceedings/Final_Papers/66_Padumadasa_EvaluatingTenderOffers_REV_FIN.pdf
Pietrantuono, R., & Russo, S. (2018). Robotics software engineering and certification: Issues and challenges. IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), 308-312. IEEE. Doi: 10.1109/ISSREW.2018.00023
Poth, A., Kottke, M., & Riel, A. (2020). Evaluation of agile team work quality. International Conference on Agile Software Development, 101-110. Springer, Cham. Doi: 10.1007/978-3-030-58858-8_11
Pressman, R. S. (2010). Software engineering: A practitioner's approach. (7th ed). New York: McGraw-Hill Higher Education.
Rae, A., Robert, P., & Hausen, H. L. (1995). Software evaluation for certification principles, practice and legal liability. England: McGraw-Hill.
Saaty, T. L., & De Paola, P. (2017). Rethinking design and urban planning for the cities of the future. Buildings, 7(3), 76. Doi: 10.3390/buildings7030076
Saaty, T. L. (2008). Decision making with the Analytic Hierarchy Process. International Journal of Services Sciences, 1(1), 83-98. Doi: 10.1504/IJSSci.2008.01759
Saaty, T. L. (1990). How to make a decision: The Analytic Hierarchy Process, European Journal of Operation Research, 48(1), 9–26. Doi: 10.1016/0377-2217(90)90057-I
Sambinelli, F., & Borges, M. A. F. (2019). Survey on strategies to increase customer value in Brazilian Agile software development companies. 14th Iberian Conference on Information Systems and Technologies (CISTI), 1-7. IEEE. Doi: 10.23919/CISTI.2019.8760653
Salo, O., & Abrahamsson, P. (2008). Agile methods in European embedded software development organizations: A survey study of extreme programming and scrum, IET Software, 2(1), 58-64. Doi: 10.1049/iet-sen:20070038
Sjøberg, D. I. (2018). Teamwork quality and team performance: Exploring differences between small and large Agile projects. Agile Processes in Software Engineering and Extreme Programming, 267.
Sommerville, I. (2007). Software engineering (8th ed). Harlow: Pearson Education Limited.
Sun-Jen, H., & Wen-Ming, H. (2006). Selection priority of process areas based on CMMI continuous representation. Information & Management, 43(3), 297-307. Doi: 10.1016/j.im.2005.08.003
Triantaphyllou, E., & Mann, S. H. (1995). Using the Analytic Hierarchy Process for decision making in engineering applications: Some challenges. International Journal of Industrial Engineering: Applications and Practice, 2(1), 35-44. Retrieved from http://bit.csc.lsu.edu/trianta/Journal_PAPERS1/AHPapls1.pdf
Voas, J., & Laplante, P. A. (2018). IoT’s certification quagmire. Computer, 51(4), 86-89. Doi: 10.1109/MC.2018.2141036
Voas, J. (1998). The software quality certification triangle: Crosstalk. The Journal of Defense Software Engineering, 11(11), 12-14.
Wagenaar, G., Overbeek, S., Lucassen, G., Brinkkemper, S., & Schneider, K. (2018). Working software over comprehensive documentation–Rationales of Agile teams for artefacts usage. Journal of Software Engineering Research and Development, 6(1), 7. Doi: 10.1186/s40411-018-0051-7
Wang, Y., & Leung, H. K. (2001). A benchmark-based adaptable software process model. Proceedings 27th EUROMICRO Conference. 2001: A Net Odyssey, 216-224.
Yaghoobi, T. (2018). Prioritizing key success factors of software projects using fuzzy AHP. Journal of Software: Evolution and Process, 30(1), e1891. Doi: 10.1002/smr.1891
Yahya, J. (2007). The development of software certification model based on product quality approach. (Unpublished doctoral dissertation). Universiti Kebangsaan Malaysia, Selangor, Malaysia.
Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Momani, F., & Zughoul, O. (2020). Novel multiperspective hiring framework for the selection of software programmer applicants based on AHP and group TOPSIS techniques. International Journal of Information Technology & Decision Making, 19(3), 775-847. Doi: 10.1142/S0219622020500121
Zhou, Z., & Liang, K. (2013). Network course evaluation system based on AHP theory. In Wenjiang, Du. Informatics and Management Science II (pp. 569-575). London: Springer London. Doi: 10.1007/978-1-4471-4811-1_73
Copyright of all articles published in IJAHP is transferred to Creative Decisions Foundation (CDF). However, the author(s) reserve the following:
- All proprietary rights other than copyright, such as patent rights.
- The right to grant or refuse permission to third parties to republish all or part of the article or translations thereof. In case of whole articles, such third parties must obtain permission from CDF as well. However, CDF may grant rights with respect to journal issues as a whole.
- The right to use all or parts of this article in future works of their own, such as lectures, press releases, reviews, textbooks, or reprint books.
- The authors affirm that the article has been neither copyrighted nor published, that it is not being submitted for publication elsewhere, and that if the work is officially sponsored, it has been released for open publication.
The only exception to the statements in the paragraph above is the following: If an article published in IJAHP contains copyrighted material, such as a teaching case, as an appendix, then the copyright (and all commercial rights) of such material remains with the original copyright holder.
CDF will receive permission for publication of copyrighted material in IJAHP. This permission is not transferable to third parties. Permission to make electronic and paper copies of part or all of the articles, including all computer files that are linked to the articles, for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage.
This permission does not apply to previously copyrighted material, such as teaching cases. In paper copies of the article, the copyright notice and the title of the publication and its date should be visible. To copy otherwise is permitted provided that a per-copy fee is paid.
To republish, to post on servers, or redistribute to lists requires that you post a link to the IJAHP article, which is available in open access delivery mode. Do not upload the article itself.
Authors are permitted to present a talk, based on a paper submitted to or accepted by IJAHP, at a conference where the paper would not be published in a copyrighted publication either before or after the conference and where the author did not assign copyright to the conference or related publisher.