MSR 2020
Mon 29 - Tue 30 June 2020
co-located with ICSE 2020
Mon 29 Jun 2020 11:36 - 11:48 at MSR:Zoom - Build, CI, & Dependencies Chair(s): Raula Gaikovina Kula

Continuous Integration (CI) is a development practice where developers frequently integrate code into a common codebase. After the code is integrated, the CI server runs a test suite and other tools to produce a set of reports (e.g., output of linters and tests). If the result of a CI test run is unexpected, developers have the option to manually restart the build, re-running the same test suite on the same code; this can reveal build flakiness, if the restarted build outcome differs from the original build. In this study, we analyze restarted builds, flaky builds, and their impact on the development workflow. We observe that developers restart at least 1.72% of builds, amounting to 56,522 restarted builds in our Travis CI dataset. We observe that more mature and more complex projects are more likely to include restarted builds. The restarted builds are mostly builds that are initially failing due to a test, network problem, or a Travis CI limitations such as execution timeout. Finally, we observe that restarted builds have a major impact on development workflow. Indeed, in 54.42% of the restarted builds, the developers analyze and restart a build within an hour of the initial failure. This suggests that developers wait for CI results, interrupting their workflow to address the issue. Restarted builds also slow down the merging of pull requests by a factor of three, bringing median merging time from 16h to 48h.

Mon 29 Jun
Times are displayed in time zone: (UTC) Coordinated Universal Time change

msr-2020-papers
11:00 - 12:00: Technical Papers - Build, CI, & Dependencies at MSR:Zoom
Chair(s): Raula Gaikovina KulaNAIST

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

msr-2020-papers11:00 - 11:12
Live Q&A
Yiwen WuNational University of Defense Technology, Yang ZhangNational University of Defense Technology, China, Tao WangNational University of Defense Technology, Huaimin Wang
Pre-print Media Attached
msr-2020-papers11:12 - 11:24
Live Q&A
Suhaib MujahidConcordia University, Rabe AbdalkareemConcordia University, Montreal, Canada, Emad ShihabConcordia University, Shane McIntoshMcGill University
Pre-print Media Attached
msr-2020-Data-showcase11:24 - 11:36
Live Q&A
Jordan HenkelUniversity of Wisconsin–Madison, Christian Bird, Shuvendu K. LahiriMicrosoft Research, Thomas RepsUniversity of Wisconsin-Madison, USA
Media Attached
msr-2020-papers11:36 - 11:48
Live Q&A
Thomas DurieuxINESC-ID, University of Lisbon, Claire Le GouesCarnegie Mellon University, Michael HiltonCarnegie Mellon University, USA, Rui AbreuInstituto Superior Técnico, U. Lisboa & INESC-ID
DOI Pre-print Media Attached
msr-2020-Data-showcase11:48 - 12:00
Live Q&A
Carolin BrandtDelft University of Technology, Annibale PanichellaDelft University of Technology, Andy ZaidmanTU Delft, Moritz BellerDelft University of Technology
Pre-print Media Attached