MSR 2020
Mon 29 - Tue 30 June 2020
co-located with ICSE 2020
Mon 29 Jun 2020 16:30 - 16:37 at MSR:Zoom2 - Platforms & Datasets Chair(s): Moritz Beller

The software engineering practice of automated testing helps programmers find defects earlier during development. With growing software projects and longer-running test suites, frequency and immediacy of feedback decline, thereby making defects harder to repair. Regression test prioritization (RTP) is concerned with running relevant tests earlier to lower the costs of defect localization and to improve feedback.

Finding representative data to evaluate RTP techniques is non-trivial, as most software is published without failing tests. In this work, we systematically survey a wide range of RTP literature regarding whether their dataset uses real or synthetic defects or tests, whether they are publicly available, and whether datasets are reused. We observed that some datasets are reused, however, many projects study only few projects and these rarely resemble real-world development activity.

In light of these threats to ecological validity, we describe the construction and characteristics of a new dataset based on 20 open-source Java programs.

Our dataset allows researchers to evaluate prioritization heuristics based on version control meta-data, source code, and test results from fine-grained, automated builds over 9 years of development history. We provide reproducible baselines for initial comparisons and make all data publicly available.

We see this as a step towards better reproducibility, ecological validity, and long-term availability of studied software in the field of test prioritization.

Mon 29 Jun
Times are displayed in time zone: (UTC) Coordinated Universal Time change

16:30 - 17:00: Technical Papers - Platforms & Datasets at MSR:Zoom2
Chair(s): Moritz BellerDelft University of Technology

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

msr-2020-papers16:30 - 16:37
Live Q&A
Toni MattisHasso Plattner Institute, University of Potsdam, Patrick ReinHasso Plattner Institute, Falco Dürsch, Robert HirschfeldHasso-Plattner-Institut (HPI), Germany
DOI Pre-print Media Attached
msr-2020-papers16:37 - 16:45
Live Q&A
Konstantinos Barmpis , Patrick NeubauerUniversity of York, UK, Jonathan Co, Dimitris KolovosUniversity of York, Nicholas Matragkas, Richard PaigeMcMaster University
Media Attached
msr-2020-papers16:45 - 16:52
Live Q&A
Che Shian Hung, Robert DyerUniversity of Nebraska - Lincoln
Pre-print Media Attached
msr-2020-Registered-Reports16:52 - 17:00
Live Q&A
Antoine PietriInria, Guillaume RousseauUniversité de Paris and Inria, Stefano ZacchiroliUniversité de Paris and Inria
Pre-print Media Attached