An Empirical Study of Common Challenges in Developing Deep Learning Applications

Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael Lyu, Miryung Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Recent advances in deep learning promote the innovation of many intelligent systems and applications such as autonomous driving and image recognition. Despite enormous efforts and investments in this field, a fundamental question remains under-investigated - what challenges do developers commonly face when building deep learning applications? To seek an answer, this paper presents a large-scale empirical study of deep learning questions in a popular Q&A website, Stack Overflow. We manually inspect a sample of 715 questions and identify seven kinds of frequently asked questions. We further build a classification model to quantify the distribution of different kinds of deep learning questions in the entire set of 39,628 deep learning questions. We find that program crashes, model migration, and implementation questions are the top three most frequently asked questions. After carefully examining accepted answers of these questions, we summarize five main root causes that may deserve attention from the research community, including API misuse, incorrect hyperparameter selection, GPU computation, static graph computation, and limited debugging and profiling support. Our results highlight the need for new techniques such as cross-framework differential testing to improve software development productivity and software reliability in deep learning.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE 30th International Symposium on Software Reliability Engineering, ISSRE 2019
EditorsKatinka Wolter, Ina Schieferdecker, Barbara Gallina, Michel Cukier, Roberto Natella, Naghmeh Ivaki, Nuno Laranjeiro
PublisherIEEE Computer Society
Pages104-115
Number of pages12
ISBN (Electronic)9781728149813
DOIs
Publication statusPublished - Oct 2019
Event30th IEEE International Symposium on Software Reliability Engineering, ISSRE 2019 - Berlin, Germany
Duration: Oct 28 2019Oct 31 2019

Publication series

NameProceedings - International Symposium on Software Reliability Engineering, ISSRE
Volume2019-October
ISSN (Print)1071-9458

Conference

Conference30th IEEE International Symposium on Software Reliability Engineering, ISSRE 2019
CountryGermany
CityBerlin
Period10/28/1910/31/19

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Software
  • Safety, Risk, Reliability and Quality

Cite this

Zhang, T., Gao, C., Ma, L., Lyu, M., & Kim, M. (2019). An Empirical Study of Common Challenges in Developing Deep Learning Applications. In K. Wolter, I. Schieferdecker, B. Gallina, M. Cukier, R. Natella, N. Ivaki, & N. Laranjeiro (Eds.), Proceedings - 2019 IEEE 30th International Symposium on Software Reliability Engineering, ISSRE 2019 (pp. 104-115). [8987482] (Proceedings - International Symposium on Software Reliability Engineering, ISSRE; Vol. 2019-October). IEEE Computer Society. https://doi.org/10.1109/ISSRE.2019.00020