Evaluating Automated Program Repair Techniques using Introductory Programming Course Datasets

研究成果: 書籍/レポート タイプへの寄稿会議への寄与

抄録

Debugging erroneous programs requires a great deal of human effort. To reduce human effort, automating debugging processes has been actively studied so far. One of such automation is automated program repair techniques for syntactic errors in programs. Researchers intend to support novice programmers such as students with these techniques because fixing syntactic errors is a difficult task for novice programmers. However, there exist few datasets that consist of programs written by novice programmers in universities and can be used to evaluate these techniques. Also, it is difficult to prepare such datasets from scratch. Indeed, prior studies usually utilized the Indian Institute of Technology Kanpur (IITK) dataset only. This limitation restricts the findings and implications in prior studies as a case study in the university. In this study, we intend to clarify which findings and implications in prior studies remain the same and which ones change in another university by a case study. We prepare three datasets that consist of over 21k programs collected from an introductory programming course in different divisions at our university. We compare the state-of-the-art automated program repair techniques, DeepFix, RLAssist, and Dr Repair, in these datasets. We found that (1) the best technique remains the same in all the datasets, (2) these techniques fix 8.3 % to 54.5 % syntactic errors in our datasets, which are 6.7 % to 32.4 % lower than those in the IITK dataset, (3) the error types that are fixed by them change in different datasets. Hence, the main finding in the IITK dataset remains the same; however, each technique fixes different errors.

本文言語英語
ホスト出版物のタイトルProceedings - 2022 IEEE 46th Annual Computers, Software, and Applications Conference, COMPSAC 2022
編集者Hong Va Leong, Sahra Sedigh Sarvestani, Yuuichi Teranishi, Alfredo Cuzzocrea, Hiroki Kashiwazaki, Dave Towey, Ji-Jiang Yang, Hossain Shahriar
出版社Institute of Electrical and Electronics Engineers Inc.
ページ569-574
ページ数6
ISBN(電子版)9781665488105
DOI
出版ステータス出版済み - 2022
イベント46th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2022 - Virtual, Online, 米国
継続期間: 6月 27 20227月 1 2022

出版物シリーズ

名前Proceedings - 2022 IEEE 46th Annual Computers, Software, and Applications Conference, COMPSAC 2022

会議

会議46th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2022
国/地域米国
CityVirtual, Online
Period6/27/227/1/22

!!!All Science Journal Classification (ASJC) codes

  • コンピュータ サイエンスの応用
  • ハードウェアとアーキテクチャ
  • ソフトウェア
  • メディア記述
  • 教育

フィンガープリント

「Evaluating Automated Program Repair Techniques using Introductory Programming Course Datasets」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル