Detecting non-natural language artifacts for de-noising bug reports

Publikation: Beitrag in einer FachzeitschriftArtikelBegutachtung

Abstract

Textual documents produced in the software engineering process are a popular target for natural language processing (NLP) and information retrieval (IR) approaches. However, issue tickets often contain artifacts such as code snippets, log outputs and stack traces. These artifacts not only inflate the issue ticket sizes, but also can this noise constitute a real problem for some NLP approaches, and therefore has to be removed in the pre-processing of some approaches. In this paper, we present a machine learning based approach to classify textual content into natural language and non-natural language artifacts at line level. We show how data from GitHub issue trackers can be used for automated training set generation, and present a custom preprocessing approach for the task of artifact removal. The training sets are automatically created from Markdown annotated issue tickets and project documentation files. We use these generated training sets to train a Markdown agnostic model that is able to classify un-annotated content. We evaluate our approach on issue tickets from projects written in C++, Java, JavaScript, PHP, and Python. Our approach achieves ROC-AUC scores between 0.92 and 0.96 for language-specific models. A multi-language model trained on the issue tickets of all languages achieves ROC-AUC scores between 0.92 and 0.95. The provided models are intended to be used as noise reduction pre-processing steps for NLP and IR approaches working on issue tickets.
Originalspracheenglisch
Aufsatznummer52
Seitenumfang29
FachzeitschriftAutomated Software Engineering
Jahrgang29
Ausgabenummer2
DOIs
PublikationsstatusVeröffentlicht - 24 Aug. 2022

ASJC Scopus subject areas

  • Software

Fields of Expertise

  • Information, Communication & Computing

Treatment code (Nähere Zuordnung)

  • Basic - Fundamental (Grundlagenforschung)

Fingerprint

Untersuchen Sie die Forschungsthemen von „Detecting non-natural language artifacts for de-noising bug reports“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren