Process Improvement through Software Operation Knowledge : If the SOK Fits, Wear It!

Abstract

Although the software industry is flourishing, and software-producing organizations strive for high levels of end-user satisfaction, these organizations do only limitedly recognize and use knowledge of the in-the-field operation of their software (e.g. software operation knowledge or SOK). Less than one-third of these organizations makes use of crash and usage feedback reports to acquire knowledge of the in-the-field behavior of their software and end-users. Furthermore, the concept of software operation knowledge has only been vaguely described, and is underexposed till date. For example, virtually no techniques or methods exist for structurally using such knowledge in processes implemented at software-producing organizations. Consequently, an emerging need is observed for a framework that guides software-producing organizations in improving their software processes through knowledge of the in-the-field behavior of their software and end-users. In this dissertation research, a framework is presented that provides and structures directions for identification, acquisition, integration, presentation and utilization of software operation knowledge. The framework, as well as the tools, techniques and methods that are presented in this dissertation, aid software-producing organizations in increasing the efficiency of their software processes. This dissertation is divided in four parts. After the introductory part, which describes research triggers, questions and methods, the SOK concept is defined. Also, the SOK framework is presented: a structure that describes parties, perspectives and life cycle processes related to knowledge of in-the-field software operation. The SOK concept is then further established by identification and classification of operational software operation knowledge practices of software-producing organizations in software ecosystems. The third part of this dissertation is focused on process improvement through knowledge of in-the-field software operation. A novel technique for generic recording and visualization of in-the-field software operation is presented. We show that this technique enables software vendors to obtain a uniform insight in the operation of their software in the field, and contributes to reduction of software maintenance efforts. Also, we present a template method for situational integration of software operation information in software processes. We show that by using the template method, software-producing organizations can improve their particular software processes with acquired software operation information. Another artifact that is presented in this dissertation is the software operation summary (SOS), a medium for presentation of software operation information. We show that the summary increases awareness of in-the-field software operation throughout software-producing organizations, and demonstrate that it contributes to reaching consensus on software maintenance task prioritization. Finally, service knowledge utilization (SKU) is introduced as an approach to increase a software vendor's flexibility, as well as its responsiveness to changes in performance and usage of its service-based, online software. The last part of this dissertation provides answers to the formulated research questions, as well as a reflection on the hermeneutic process we experienced throughout the construction of the artifacts that are presented in this dissertation.