Multi-Criteria Decision-Making in Software Production

Abstract

Decision making is an inevitable part of software engineering. Software engineers make a considerable number of decisions during the software development life cycle. Thus, as a subset of software engineering, software production can be considered a continuous decision-making process. The decision process refers to the steps involved in choosing and evaluating the best fitting alternative solution(s) for software engineers, as decision-makers, according to their preferences and requirements.

In order to make informed decisions, the decision-makers around a software product should either acquire knowledge themselves or hire external experts to support them with their decision-making process. The process gets more complicated as the number of decision-makers, alternatives, and criteria increases. Therefore, software production is a suitable domain to deploy decision support systems that intelligently support these decision-makers in the decision-making process. A decision model for each decision-making problem is required to externalize and organize knowledge regarding the selection context. In this dissertation, we focus on pragmatically selected decision-making problems that software engineers face in software production. The following categories of software production decisions are discussed: (1) decision-making regarding COTS components for inclusion into software products. (2) decision problems related to software development technologies that deal with finding the best fitting technologies for developing a software product. (3) architectural design decisions concerning pattern-driven software design. We developed a theoretical framework to assist software engineers with a set of Multi-Criteria Decision-Making (MCDM) problems in software production. The framework provides a guideline for software engineers to systematically capture knowledge from different knowledge sources to build decision models for MCDM problems in software production.

The framework and the decision support system have been used to model and support software engineers with the following decision-making problems:

1. COTS component selection problems: Database Technology Selection, Cloud Service Provider Selection, Blockchain Platform Selection 2. Software development technology selection problems: Programming Language Ecosystem Selection, Model-Driven Software Development Platform selection 3. Decision-Making in Pattern-Driven Design: Software Architecture Pattern Selection

A broad study has been carried out based on qualitative and quantitative research to evaluate the DSS’s efficiency and effectiveness and the decision models inside its knowledge base to support software engineers with their decision-making process in software production. The DSS and the decision models have been evaluated through 21 real-world case studies at different software-producing organizations located in the Netherlands and Iran. The case study participants asserted that the approach and tooling provide significantly more insight into their selection process, provide a richer prioritized option list than if they had done their research independently, and reduce the time and cost of the decision-making process.

The dissertation concludes that software production decisions are best made with decision support systems but that the steps towards full adoption of such systems are hampered. First, gathering and maintaining appropriate knowledge in a centralized manner is relatively costly and requires more time investment than traditional decision methods. Secondly, software engineers are not used to using such technologies and find it challenging to adopt it into their daily practice.