MSR 2020
Mon 29 - Tue 30 June 2020
co-located with ICSE 2020
Mon 29 Jun 2020 16:30 - 16:37 at MSR:Zoom2 - Platforms & Datasets Chair(s): Moritz Beller

The software engineering practice of automated testing helps programmers find defects earlier during development. With growing software projects and longer-running test suites, frequency and immediacy of feedback decline, thereby making defects harder to repair. Regression test prioritization (RTP) is concerned with running relevant tests earlier to lower the costs of defect localization and to improve feedback.

Finding representative data to evaluate RTP techniques is non-trivial, as most software is published without failing tests. In this work, we systematically survey a wide range of RTP literature regarding whether their dataset uses real or synthetic defects or tests, whether they are publicly available, and whether datasets are reused. We observed that some datasets are reused, however, many projects study only few projects and these rarely resemble real-world development activity.

In light of these threats to ecological validity, we describe the construction and characteristics of a new dataset based on 20 open-source Java programs.

Our dataset allows researchers to evaluate prioritization heuristics based on version control meta-data, source code, and test results from fine-grained, automated builds over 9 years of development history. We provide reproducible baselines for initial comparisons and make all data publicly available.

We see this as a step towards better reproducibility, ecological validity, and long-term availability of studied software in the field of test prioritization.

Conference Day
Mon 29 Jun

Displayed time zone: (UTC) Coordinated Universal Time change

16:30 - 17:00
Platforms & DatasetsTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom2
Chair(s): Moritz BellerFacebook, USA

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

Live Q&A
RTPTorrent: An Open-source Dataset for Evaluating Regression Test PrioritizationMSR - Technical Paper
Technical Papers
Toni MattisHasso Plattner Institute, University of Potsdam, Patrick ReinHasso Plattner Institute, Falco Dürsch, Robert HirschfeldHasso-Plattner-Institut (HPI), Germany
DOI Pre-print Media Attached
Live Q&A
Polyglot and Distributed Software Repository Mining with CROSSFLOWMSR - Technical Paper
Technical Papers
Konstantinos Barmpis , Patrick NeubauerUniversity of York, UK, Jonathan Co, Dimitris KolovosUniversity of York, Nicholas Matragkas, Richard PaigeMcMaster University
Media Attached
Live Q&A
Boa Views: Easy Modularization and Sharing of MSR AnalysesMSR - Technical Paper
Technical Papers
Che Shian Hung, Robert DyerUniversity of Nebraska - Lincoln
Pre-print Media Attached
Live Q&A
Determining the Intrinsic Structure of Public Software Development HistoryMSR - Registered Reports
Registered Reports
A: Antoine PietriInria, A: Guillaume RousseauUniversité de Paris and Inria, A: Stefano ZacchiroliUniversité de Paris and Inria
Pre-print Media Attached