What is the Vocabulary of Flaky Tests?
MSR - Technical Paper
Flaky tests are tests whose outcomes are non-deterministic. Despite the recent research activity on this topic, no effort has been made on understanding the vocabulary of flaky tests (e.g., networking or concurrency identifiers). This work proposes to automatically classify tests as flaky or not. Classification of flaky tests is important,for example, to detect the introduction of flaky test and to search for flaky tests after they are introduced in test suites. We evaluated performance of various machine learning algorithms to solve this problem. We constructed a dataset of flaky and non-flaky tests by running more than 50k test cases, 100 times each. We then used machine learning techniques on the resulting data set to predict which tests are flaky from their source. Based on features, such as counting stemmed tokens extracted from source code identifiers, we achieved an F-measure of 0.95 for the identification of flaky tests. The best performance was achieved when using Random Forest and Support Vector Machines for the prediction. In terms of the code identifiers that are most strongly associated with test flakiness, we noted that job, action, and services are commonly associated with flaky tests.
Tue 30 Jun Times are displayed in time zone: (UTC) Coordinated Universal Time change
|14:00 - 14:12|
A Machine Learning Approach for Vulnerability CurationACM SIGSOFT Distinguished Paper AwardMSR - Technical Paper
Chen YangVeracode, Inc., Andrew SantosaVeracode, Inc., Ang Ming Yi, Abhishek Sharma Singapore Management University, Singapore, Asankhaya SharmaVeracode, Inc., David LoSingapore Management UniversityPre-print Media Attached
|14:12 - 14:24|
Rhys ComptonUniversity of Waikato, Eibe FrankDepartment of Computer Science, University of Waikato, Panos Patros, Abigail KoayUniversity of WaikatoDOI Pre-print Media Attached
|14:24 - 14:36|
A Study on the Accuracy of OCR Engines for Source Code Transcription from Programming ScreencastsMSR - Technical Paper
Abdulkarim KhormiFlorida State University, USA - Jazan University, KSA, Mohammad AlahmadiFlorida State University, Sonia HaiducFlorida State UniversityPre-print Media Attached
|14:36 - 14:48|
Gustavo PintoUFPA, Breno MirandaFederal University of Pernambuco, Supun DissanayakeThe University of Adelaide, Marcelo d'AmorimFederal University of Pernambuco, Christoph TreudeThe University of Adelaide, Antonia BertolinoCNR-ISTIPre-print Media Attached
|14:48 - 15:00|
Sakib HaqueUniversity of Notre Dame, Alexander LeClairUniversity Of Notre Dame, Lingfei WuIBM Research, Collin McMillanUniversity of Notre DamePre-print Media Attached