Studying the cost and effectiveness of oss quality assessment models: An experience report of fujitsu QNET

Yasutaka Kamei, Takahiro Matsumoto, Kazuhiro Yamashita, Naoyasu Ubayashi, Takashi Iwasaki, Shuichi Takayama

Research output: Contribution to journalArticle

Abstract

Nowadays, open source software (OSS) systems are adopted by proprietary software projects. To reduce the risk of using problematic OSS systems (e.g., causing system crashes), it is important for proprietary software projects to assess OSS systems in advance. Therefore, OSS quality assessment models are studied to obtain information regarding the quality of OSS systems. Although the OSS quality assessment models are partially validated using a small number of case studies, to the best of our knowledge, there are few studies that empirically report how industrial projects actually use OSS quality assessment models in their own development process. In this study, we empirically evaluate the cost and effectiveness of OSS quality assessment models at Fujitsu Kyushu Network Technologies Limited (Fujitsu QNET). To conduct the empirical study, we collect datasets from (a) 120 OSS projects that Fujitsu QNET's projects actually used and (b) 10 problematic OSS projects that caused major problems in the projects. We find that (1) it takes average and median times of 51 and 49 minutes, respectively, to gather all assessment metrics per OSS project and (2) there is a possibility that we can filter problematic OSS systems by using the threshold derived from a pool of assessment metrics. Fujitsu QNET's developers agree that our results lead to improvements in Fujitsu QNET's OSS assessment process. We believe that our work significantly contributes to the empirical knowledge about applying OSS assessment techniques to industrial projects.

Original languageEnglish
Pages (from-to)2744-2753
Number of pages10
JournalIEICE Transactions on Information and Systems
VolumeE101D
Issue number11
DOIs
Publication statusPublished - Nov 1 2018

Fingerprint

Costs
Computer systems
Open source software

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Cite this

Studying the cost and effectiveness of oss quality assessment models : An experience report of fujitsu QNET. / Kamei, Yasutaka; Matsumoto, Takahiro; Yamashita, Kazuhiro; Ubayashi, Naoyasu; Iwasaki, Takashi; Takayama, Shuichi.

In: IEICE Transactions on Information and Systems, Vol. E101D, No. 11, 01.11.2018, p. 2744-2753.

Research output: Contribution to journalArticle

Kamei, Yasutaka ; Matsumoto, Takahiro ; Yamashita, Kazuhiro ; Ubayashi, Naoyasu ; Iwasaki, Takashi ; Takayama, Shuichi. / Studying the cost and effectiveness of oss quality assessment models : An experience report of fujitsu QNET. In: IEICE Transactions on Information and Systems. 2018 ; Vol. E101D, No. 11. pp. 2744-2753.
@article{103ee844a2174600b66ffde7228cae03,
title = "Studying the cost and effectiveness of oss quality assessment models: An experience report of fujitsu QNET",
abstract = "Nowadays, open source software (OSS) systems are adopted by proprietary software projects. To reduce the risk of using problematic OSS systems (e.g., causing system crashes), it is important for proprietary software projects to assess OSS systems in advance. Therefore, OSS quality assessment models are studied to obtain information regarding the quality of OSS systems. Although the OSS quality assessment models are partially validated using a small number of case studies, to the best of our knowledge, there are few studies that empirically report how industrial projects actually use OSS quality assessment models in their own development process. In this study, we empirically evaluate the cost and effectiveness of OSS quality assessment models at Fujitsu Kyushu Network Technologies Limited (Fujitsu QNET). To conduct the empirical study, we collect datasets from (a) 120 OSS projects that Fujitsu QNET's projects actually used and (b) 10 problematic OSS projects that caused major problems in the projects. We find that (1) it takes average and median times of 51 and 49 minutes, respectively, to gather all assessment metrics per OSS project and (2) there is a possibility that we can filter problematic OSS systems by using the threshold derived from a pool of assessment metrics. Fujitsu QNET's developers agree that our results lead to improvements in Fujitsu QNET's OSS assessment process. We believe that our work significantly contributes to the empirical knowledge about applying OSS assessment techniques to industrial projects.",
author = "Yasutaka Kamei and Takahiro Matsumoto and Kazuhiro Yamashita and Naoyasu Ubayashi and Takashi Iwasaki and Shuichi Takayama",
year = "2018",
month = "11",
day = "1",
doi = "10.1587/transinf.2018EDP7163",
language = "English",
volume = "E101D",
pages = "2744--2753",
journal = "IEICE Transactions on Information and Systems",
issn = "0916-8532",
publisher = "一般社団法人電子情報通信学会",
number = "11",

}

TY - JOUR

T1 - Studying the cost and effectiveness of oss quality assessment models

T2 - An experience report of fujitsu QNET

AU - Kamei, Yasutaka

AU - Matsumoto, Takahiro

AU - Yamashita, Kazuhiro

AU - Ubayashi, Naoyasu

AU - Iwasaki, Takashi

AU - Takayama, Shuichi

PY - 2018/11/1

Y1 - 2018/11/1

N2 - Nowadays, open source software (OSS) systems are adopted by proprietary software projects. To reduce the risk of using problematic OSS systems (e.g., causing system crashes), it is important for proprietary software projects to assess OSS systems in advance. Therefore, OSS quality assessment models are studied to obtain information regarding the quality of OSS systems. Although the OSS quality assessment models are partially validated using a small number of case studies, to the best of our knowledge, there are few studies that empirically report how industrial projects actually use OSS quality assessment models in their own development process. In this study, we empirically evaluate the cost and effectiveness of OSS quality assessment models at Fujitsu Kyushu Network Technologies Limited (Fujitsu QNET). To conduct the empirical study, we collect datasets from (a) 120 OSS projects that Fujitsu QNET's projects actually used and (b) 10 problematic OSS projects that caused major problems in the projects. We find that (1) it takes average and median times of 51 and 49 minutes, respectively, to gather all assessment metrics per OSS project and (2) there is a possibility that we can filter problematic OSS systems by using the threshold derived from a pool of assessment metrics. Fujitsu QNET's developers agree that our results lead to improvements in Fujitsu QNET's OSS assessment process. We believe that our work significantly contributes to the empirical knowledge about applying OSS assessment techniques to industrial projects.

AB - Nowadays, open source software (OSS) systems are adopted by proprietary software projects. To reduce the risk of using problematic OSS systems (e.g., causing system crashes), it is important for proprietary software projects to assess OSS systems in advance. Therefore, OSS quality assessment models are studied to obtain information regarding the quality of OSS systems. Although the OSS quality assessment models are partially validated using a small number of case studies, to the best of our knowledge, there are few studies that empirically report how industrial projects actually use OSS quality assessment models in their own development process. In this study, we empirically evaluate the cost and effectiveness of OSS quality assessment models at Fujitsu Kyushu Network Technologies Limited (Fujitsu QNET). To conduct the empirical study, we collect datasets from (a) 120 OSS projects that Fujitsu QNET's projects actually used and (b) 10 problematic OSS projects that caused major problems in the projects. We find that (1) it takes average and median times of 51 and 49 minutes, respectively, to gather all assessment metrics per OSS project and (2) there is a possibility that we can filter problematic OSS systems by using the threshold derived from a pool of assessment metrics. Fujitsu QNET's developers agree that our results lead to improvements in Fujitsu QNET's OSS assessment process. We believe that our work significantly contributes to the empirical knowledge about applying OSS assessment techniques to industrial projects.

UR - http://www.scopus.com/inward/record.url?scp=85056145494&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056145494&partnerID=8YFLogxK

U2 - 10.1587/transinf.2018EDP7163

DO - 10.1587/transinf.2018EDP7163

M3 - Article

AN - SCOPUS:85056145494

VL - E101D

SP - 2744

EP - 2753

JO - IEICE Transactions on Information and Systems

JF - IEICE Transactions on Information and Systems

SN - 0916-8532

IS - 11

ER -