MSR 2020
Mon 29 - Tue 30 June 2020
co-located with ICSE 2020
Tue 30 Jun 2020 11:24 - 11:36 at MSR:Zoom2 - Security Chair(s): Dimitris Mitropoulos

Security is an increasing concern in software development. Developer Question and Answer (Q&A) websites provide a large amount of security discussion. Existing studies have used human-defined rules to mine security discussions, but these works still miss many posts, which may lead to an incomplete analysis of the security practices reported on Q&A websites. Traditional supervised Machine Learning methods can automate the mining process; however, the required negative (non-security) class is too expensive to obtain. We propose a novel learning framework, PUMiner, to automatically mine security posts from Q&A websites. PUMiner builds a context-aware embedding model to extract features of the posts, and then develops a two-stage PU model to identify security content using the labelled Positive and Unlabelled posts. We evaluate PUMiner on more than 17.2 million posts on Stack Overflow and 52,611 posts on Security StackExchange. We show that PUMiner is effective with the validation performance of at least 85% across all model configurations. Moreover, Matthews Correlation Coefficient (MCC) of PUMiner is 0.906 points, 148% and 10.4% higher than one-class SVM, positive-similarity filtering, and one-stage PU models on unseen testing posts, respectively. PUMiner also performs well with an MCC of 0.745 for scenarios where string matching totally fails. Even when the ratio of the labelled positive posts to the unlabelled ones is only 1:100, PUMiner still achieves a strong MCC of 0.65, which is 160% better than fully-supervised learning. Using PUMiner, we provide the largest and up-to-date security content on Q&A websites for practitioners and researchers.

Tue 30 Jun
Times are displayed in time zone: (UTC) Coordinated Universal Time change

11:00 - 12:00: SecurityTechnical Papers / Data Showcase at MSR:Zoom2
Chair(s): Dimitris MitropoulosAthens University of Economics and Business

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

11:00 - 11:12
Live Q&A
Technical Papers
Danielle GonzalezRochester Institute of Technology, USA, Michael RathTechnische Universit├Ąt Ilmenau, Mehdi MirakhorliRochester Institute of Technology
DOI Pre-print Media Attached
11:12 - 11:24
Live Q&A
Technical Papers
Paolo Calciati IMDEA Software Institute, Konstantin KuznetsovSaarland University, CISPA, Alessandra GorlaIMDEA Software Institute, Andreas ZellerCISPA Helmholtz Center for Information Security
Media Attached
11:24 - 11:36
Live Q&A
Technical Papers
Triet Le Huynh MinhThe University of Adelaide, David Hin, Roland Croft, Muhammad Ali BabarThe University of Adelaide
DOI Pre-print Media Attached
11:36 - 11:48
Live Q&A
Data Showcase
A: Jiahao FanNew Jersey Institute of Technology, USA, A: Yi LiNew Jersey Institute of Technology, USA, A: Shaohua WangNew Jersey Institute of Technology, USA, A: Tien N. NguyenUniversity of Texas at Dallas
Media Attached
11:48 - 12:00
Live Q&A
Technical Papers
James WaldenNorthern Kentucky University
Pre-print Media Attached