MSR 2020
Mon 29 - Tue 30 June 2020
co-located with ICSE 2020

To increase transparency and replicability of empirical research, other disciplines have started to offer preregistration and registered reports for studies. With preregistration, authors can submit an experimental plan, including hypotheses and expected outcome, and get feedback before data is collected. More information on preregistration can be found with the Open Science Initiative (https://cos.io/rr/). Due to its success, we are piloting a registered reports track at this year’s MSR.

Dates
Tracks
Plenary
You're viewing the program in a time zone which is different from your device's time zone - change time zone

Mon 29 Jun
Times are displayed in time zone: (UTC) Coordinated Universal Time change

10:30 - 11:00: Programming Languages & ModelsTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Dimitris KolovosUniversity of York

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

10:30 - 10:36
Live Q&A
Registered Reports
A: Jürgen CitoMIT, A: Jiasi ShenMassachusetts Institute of Technology, A: Martin RinardMIT
Pre-print Media Attached
10:36 - 10:42
Live Q&A
Technical Papers
Jason TsayIBM Research, Alan BrazIBM Research, Martin HirzelIBM Research, Avraham ShinnarIBM Research, Todd Mummert
Pre-print Media Attached
10:42 - 10:48
Live Q&A
Technical Papers
Timofey BryksinJetBrains Research, Saint Petersburg State University, Victor PetukhovJetBrains, ITMO University, Ilya Alexin, Stanislav Prikhodko, Alexey Shpilman, Vladimir KovalenkoTU Delft, Nikita PovarovJetBrains
Pre-print Media Attached
10:48 - 10:54
Live Q&A
Technical Papers
Tomoki NakamaruGraduate School of Information Science and Technology, The University of Tokyo, Tomomasa Matsunaga, Tetsuro YamazakiGraduate School of Information Science and Technology, The University of Tokyo, Soramichi AkiyamaDepartment of Creative Informatics, The University of Tokyo, Shigeru ChibaThe University of Tokyo
Pre-print Media Attached
10:54 - 11:00
Live Q&A
Technical Papers
Nan YangEindhoven University of Technology, The Netherlands, Pieter Cuijpers, Ramon SchiffelersEindhoven University of Technology and ASML, the Netherlands, Johan Lukkien, Alexander SerebrenikEindhoven University of Technology
Media Attached
10:30 - 11:00: Refactoring & TestingTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom2
Chair(s): Mauricio AnicheDelft University of Technology, Netherlands

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

10:30 - 10:37
Live Q&A
Technical Papers
Leonardo Da Silva SousaCarnegie Mellon University, USA, Diego CedrimPontifical Catholic University of Rio de Janeiro, Alessandro GarciaPUC-Rio, Willian OizumiPUC-Rio, Ana Carla BibianoPUC-Rio, Daniel OliveiraPUC-Rio, Miryung KimUniversity of California, Los Angeles, Anderson OliveiraPUC-Rio
Pre-print Media Attached
10:37 - 10:45
Live Q&A
Technical Papers
Matheus PaixaoUniversity of Fortaleza, Anderson UchôaPontifical Catholic University of Rio de Janeiro (PUC-Rio), Ana Carla BibianoPUC-Rio, Daniel OliveiraPUC-Rio, Alessandro GarciaPUC-Rio, Jens KrinkeUniversity College London, Emilio Arvonio
Pre-print Media Attached
10:45 - 10:52
Live Q&A
Data Showcase
Federico Corò, A: Roberto VerdecchiaVrije Universiteit Amsterdam, A: Emilio Cruciani, A: Breno MirandaFederal University of Pernambuco, A: Antonia BertolinoCNR-ISTI
Pre-print Media Attached
10:52 - 11:00
Live Q&A
Data Showcase
A: András Kicsi, A: László VidácsUniversity of Szeged, Hungary, A: Tibor Gyimothy
Pre-print Media Attached
11:00 - 12:00: Build, CI, & DependenciesTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Raula Gaikovina KulaNAIST

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

11:00 - 11:12
Live Q&A
Technical Papers
Yiwen WuNational University of Defense Technology, Yang ZhangNational University of Defense Technology, China, Tao WangNational University of Defense Technology, Huaimin Wang
Pre-print Media Attached
11:12 - 11:24
Live Q&A
Technical Papers
Suhaib MujahidConcordia University, Rabe AbdalkareemConcordia University, Montreal, Canada, Emad ShihabConcordia University, Shane McIntoshMcGill University
Pre-print Media Attached
11:24 - 11:36
Live Q&A
Data Showcase
A: Jordan HenkelUniversity of Wisconsin–Madison, A: Christian Bird, A: Shuvendu K. LahiriMicrosoft Research, A: Thomas RepsUniversity of Wisconsin-Madison, USA
Media Attached
11:36 - 11:48
Live Q&A
Technical Papers
Thomas DurieuxKTH Royal Institute of Technology, Sweden, Claire Le GouesCarnegie Mellon University, Michael HiltonCarnegie Mellon University, USA, Rui AbreuInstituto Superior Técnico, U. Lisboa & INESC-ID
DOI Pre-print Media Attached
11:48 - 12:00
Live Q&A
Data Showcase
A: Carolin BrandtDelft University of Technology, A: Annibale PanichellaDelft University of Technology, A: Andy ZaidmanTU Delft, A: Moritz BellerFacebook, USA
Pre-print Media Attached
11:30 - 12:00: Registered Reports Track DiscussionRegistered Reports at MSR:Zoom2
Chair(s): Neil ErnstUniversity of Victoria, Janet SiegmundTU Chemnitz

Open Discussion over Zoom (Joining info available on Slack)

11:30 - 12:00
Other
Registered Reports
Neil ErnstUniversity of Victoria, Janet SiegmundUniversity of Passau
12:00 - 13:00: Code SmellsTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Alessandro GarciaPUC-Rio

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

12:00 - 12:10
Live Q&A
Technical Papers
Pre-print Media Attached
12:10 - 12:20
Live Q&A
Technical Papers
Davide SpadiniDelft University of Technology, Netherlands, Martin Schvarcbacher, Ana Maria Oprescu, Magiel BruntinkSoftware Improvement Group, Alberto BacchelliUniversity of Zurich
DOI Pre-print Media Attached
12:20 - 12:30
Live Q&A
Technical Papers
Biruk Asmare Muse, Masud RahmanDalhousie University, Csaba NagySoftware Institute - USI, Lugano, Anthony CleveUniversity of Namur, Foutse KhomhPolytechnique Montréal, Giuliano AntoniolPolytechnique Montréal
Pre-print Media Attached
12:30 - 12:40
Live Q&A
Registered Reports
A: Mouna Abidi, A: Moses Openja, A: Foutse KhomhPolytechnique Montréal
Pre-print Media Attached
12:40 - 12:50
Live Q&A
Technical Papers
Hadhemi Jebnoun, Masud RahmanDalhousie University, Foutse KhomhPolytechnique Montréal, Houssem Ben Braiek
Pre-print Media Attached
12:50 - 13:00
Live Q&A
Technical Papers
Fabiano PecorelliUniversity of Salerno, Fabio PalombaUniversity of Salerno, Foutse KhomhPolytechnique Montréal, Andrea De LuciaUniversity of Salerno
Pre-print Media Attached
12:00 - 13:00: MSR Mining ChallengeMining Challenge / Technical Papers at MSR:Zoom2
Chair(s): Antoine PietriInria, Stefano ZacchiroliUniversité de Paris and Inria, Diomidis SpinellisAthens University of Economics and Business

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

12:00 - 12:20
Live Q&A
Mining Challenge
Pre-print Media Attached
12:20 - 12:40
Live Q&A
Mining Challenge
A: Avijit BhattacharjeeUniversity of Saskatchewan, Canada, A: Sristy Sumana Nath, A: Shurui ZhouCarnegie Mellon University, USA / University of Toronto, CA, A: Debasish Chakroborti, A: Banani RoyUniversity of Saskatchewan, A: Chanchal K. RoyUniversity of Saskatchewan, A: Kevin SchneiderUniversity of Saskatchewan
DOI Pre-print Media Attached
12:40 - 13:00
Live Q&A
Mining Challenge
Gabor Antal, Márton Keleti, A: Peter HegedusUniversity of Szeged
Pre-print Media Attached
13:00 - 13:15: "Opening" & AwardsMSR Plenary at MSR:Zoom
Chair(s): Sunghun KimHong Kong University of Science and Technology, Georgios GousiosDelft University of Technology, Sarah NadiUniversity of Alberta

Live on YouTube: https://www.youtube.com/watch?v=Qvf7mHa-YYs

13:00 - 13:15
Day opening
MSR Plenary
Sunghun KimHong Kong University of Science and Technology, Sarah NadiUniversity of Alberta, Georgios GousiosDelft University of Technology
Media Attached
14:30 - 15:30: Bugs & IssuesTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Francisco ServantVirginia Tech

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

14:30 - 14:40
Live Q&A
Data Showcase
A: Cristiano PolitowskiConcordia University, Canada, A: Fabio PetrilloUniversity of Quebec at Chicoutimi, A: Yann-Gaël GuéhéneucConcordia University and Polytechnique Montréal, A: Gabriel Cavalheiro UllmannUNIJUI - Universidade Regional do Noroeste do Estado do Rio Grande do Sul, A: Josias De Andrade Werly
Media Attached
14:40 - 14:50
Live Q&A
Technical Papers
Omar El Zarif, Daniel Alencar Da CostaUniversity of Otago, Safwat HassanQueens University, Kingston, Canada, Ying ZouQueen's University, Kingston, Ontario
Pre-print Media Attached
14:50 - 15:00
Live Q&A
Technical Papers
Pre-print Media Attached
15:00 - 15:10
Live Q&A
Technical Papers
Media Attached
15:10 - 15:20
Live Q&A
Data Showcase
A: Rafael-Michael KarampatsisThe University of Edinburgh, A: Charles SuttonGoogle Research
Pre-print Media Attached
15:20 - 15:30
Live Q&A
Registered Reports
A: Steffen HerboldUniversity of Göttingen, A: Alexander TrautschUniversity of Göttingen, A: Benjamin Ledel
Pre-print Media Attached
14:30 - 15:00: Tutorial 1: GDPR ConsiderationsEducation / Technical Papers at MSR:Zoom2
Chair(s): Alexander SerebrenikEindhoven University of Technology, Abram HindleUniversity of Alberta

Q/A for tutorial (Joining info available on Slack)

14:30 - 15:00
Tutorial
Education
A: Jesus M. Gonzalez-BarahonaUniversidad Rey Juan Carlos
Pre-print Media Attached
16:30 - 17:30: Github & OSS DatasetsTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Olga BaysalCarleton University

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

16:30 - 16:38
Live Q&A
Data Showcase
A: Xunhui ZhangNational University of Defense Technology, China, A: Ayushi RastogiPostdoctoral researcher at TU Delft, A: Yue YuCollege of Computer, National University of Defense Technology, Changsha 410073, China
Pre-print Media Attached
16:38 - 16:47
Live Q&A
Data Showcase
A: Usman Ashraf, A: Christoph Mayr-DornJohannes Kepler University Linz, A: Alexander EgyedJohannes Kepler University, Linz, A: Sebastiano Panichella
Media Attached
16:47 - 16:55
Live Q&A
Data Showcase
A: Audris Mockus, A: Zoe KottiAthens University of Economics and Business, A: Diomidis SpinellisAthens University of Economics and Business, A: Gabriel Dusing
Media Attached
16:55 - 17:04
Live Q&A
Data Showcase
A: Diomidis SpinellisAthens University of Economics and Business, A: Zoe KottiAthens University of Economics and Business, A: Konstantinos Kravvaritis, A: Georgios Theodorou, A: Panos Louridas Athens University of Economics and Business
DOI Pre-print Media Attached
17:04 - 17:12
Live Q&A
Data Showcase
A: Diomidis SpinellisAthens University of Economics and Business, A: Zoe KottiAthens University of Economics and Business, A: Audris Mockus
DOI Pre-print Media Attached
17:12 - 17:21
Live Q&A
Data Showcase
A: Tanner Fry, A: Tapajit Dey, A: Andrey KarnauchUniversity of Tennessee Knoxville, A: Audris Mockus
Pre-print Media Attached
17:21 - 17:30
Live Q&A
Data Showcase
A: Maëlick Claes University of Oulu, A: Mika MäntyläUniversity of Oulu
Media Attached
16:30 - 17:00: Platforms & DatasetsTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom2
Chair(s): Moritz BellerFacebook, USA

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

16:30 - 16:37
Live Q&A
Technical Papers
Toni MattisHasso Plattner Institute, University of Potsdam, Patrick ReinHasso Plattner Institute, Falco Dürsch, Robert HirschfeldHasso-Plattner-Institut (HPI), Germany
DOI Pre-print Media Attached
16:37 - 16:45
Live Q&A
Technical Papers
Konstantinos Barmpis , Patrick NeubauerUniversity of York, UK, Jonathan Co, Dimitris KolovosUniversity of York, Nicholas Matragkas, Richard PaigeMcMaster University
Media Attached
16:45 - 16:52
Live Q&A
Technical Papers
Che Shian Hung, Robert DyerUniversity of Nebraska - Lincoln
Pre-print Media Attached
16:52 - 17:00
Live Q&A
Registered Reports
A: Antoine PietriInria, A: Guillaume RousseauUniversité de Paris and Inria, A: Stefano ZacchiroliUniversité de Paris and Inria
Pre-print Media Attached

Tue 30 Jun
Times are displayed in time zone: (UTC) Coordinated Universal Time change

10:30 - 11:00: EvolutionTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Jürgen CitoMIT

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

10:30 - 10:37
Live Q&A
Technical Papers
Jens MeinickeCarnegie Mellon University, Juan HoyosUniversidad Nacional de Colombia, Bogdan VasilescuCarnegie Mellon University, Christian KästnerCarnegie Mellon University
Pre-print Media Attached
10:37 - 10:45
Live Q&A
Technical Papers
Antoine PietriInria, Guillaume RousseauUniversité de Paris and Inria, Stefano ZacchiroliUniversité de Paris and Inria
Pre-print Media Attached
10:45 - 10:52
Live Q&A
Technical Papers
Sergey Svitkov, Timofey BryksinJetBrains Research, Saint Petersburg State University
Pre-print Media Attached
10:52 - 11:00
Live Q&A
Data Showcase
A: Themistoklis DiamantopoulosElectrical and Computer Engineering Dept, Aristotle University of Thessaloniki, A: Michail Papamichail , A: Thomas Karanikiotis, A: Kyriakos Chatzidimitriou Aristotle University of Thessaloniki, A: Andreas SymeonidisAristotle University of Thessaloniki
Pre-print Media Attached
10:30 - 11:00: Apps & BotsTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom2
Chair(s): Ivano MalavoltaVrije Universiteit Amsterdam

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

10:30 - 10:37
Live Q&A
Data Showcase
A: Pei Liu, A: Li LiMonash University, Australia, A: Yanjie Zhao, A: Xiaoyu Sun, A: John GrundyMonash University
Media Attached
10:37 - 10:45
Live Q&A
Data Showcase
Media Attached
10:45 - 10:52
Live Q&A
Technical Papers
Tapajit Dey, Sara Mousavi, Eduardo PonceUniversity of Tennessee - Knoxville, Tanner Fry, Bogdan VasilescuCarnegie Mellon University, Anna Filippova, Audris MockusUniversity of Tennessee - Knoxville
Pre-print Media Attached
10:52 - 11:00
Live Q&A
Technical Papers
Ahmad AbdellatifConcordia University, Diego CostaConcordia University, Canada, Khaled BadranConcordia University, Rabe AbdalkareemConcordia University, Montreal, Canada, Emad ShihabConcordia University
Pre-print Media Attached
11:00 - 12:00: QualityTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Jens KrinkeUniversity College London

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

11:00 - 11:12
Live Q&A
Technical Papers
Laerte XavierUniversidade Federal de Minas Gerais (UFMG), Fabio da Silva Ferreira, Rodrigo Brito, Marco Tulio ValenteFederal University of Minas Gerais, Brazil
Pre-print Media Attached
11:12 - 11:24
Live Q&A
Technical Papers
Peipei WangNorth Carolina State University, USA, Chris BrownNorth Carolina State University, Jamie JenningsNorth Carolina State University, Kathryn StoleeNorth Carolina State University
Pre-print Media Attached
11:24 - 11:36
Live Q&A
Registered Reports
A: Pavlína Wurzel Gonçalves, A: Enrico Fregnan, A: Tobias Baum, A: Kurt SchneiderLeibniz Universität Hannover, Software Engineering Group, A: Alberto BacchelliUniversity of Zurich
Pre-print Media Attached
11:36 - 11:48
Live Q&A
Technical Papers
Pre-print Media Attached
11:48 - 12:00
Live Q&A
Technical Papers
Yaroslav GolubevJetBrains Research, ITMO University, Maria Eliseeva, Nikita PovarovJetBrains, Timofey BryksinJetBrains Research, Saint Petersburg State University
Pre-print Media Attached
11:00 - 12:00: SecurityTechnical Papers / Data Showcase at MSR:Zoom2
Chair(s): Dimitris MitropoulosAthens University of Economics and Business

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

11:00 - 11:12
Live Q&A
Technical Papers
Danielle GonzalezRochester Institute of Technology, USA, Michael RathTechnische Universität Ilmenau, Mehdi MirakhorliRochester Institute of Technology
DOI Pre-print Media Attached
11:12 - 11:24
Live Q&A
Technical Papers
Paolo Calciati IMDEA Software Institute, Konstantin KuznetsovSaarland University, CISPA, Alessandra GorlaIMDEA Software Institute, Andreas ZellerCISPA Helmholtz Center for Information Security
Media Attached
11:24 - 11:36
Live Q&A
Technical Papers
Triet Le Huynh MinhThe University of Adelaide, David Hin, Roland Croft, Muhammad Ali BabarThe University of Adelaide
DOI Pre-print Media Attached
11:36 - 11:48
Live Q&A
Data Showcase
A: Jiahao FanNew Jersey Institute of Technology, USA, A: Yi LiNew Jersey Institute of Technology, USA, A: Shaohua WangNew Jersey Institute of Technology, USA, A: Tien N. NguyenUniversity of Texas at Dallas
Media Attached
11:48 - 12:00
Live Q&A
Technical Papers
James WaldenNorthern Kentucky University
Pre-print Media Attached
14:00 - 15:00: ML4SETechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Kevin MoranWilliam & Mary/George Mason University

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

14:00 - 14:12
Live Q&A
Technical Papers
Chen YangVeracode, Inc., Andrew SantosaVeracode, Inc., Ang Ming Yi, Abhishek Sharma Singapore Management University, Singapore, Asankhaya SharmaVeracode, Inc., David LoSingapore Management University
Pre-print Media Attached
14:12 - 14:24
Live Q&A
Technical Papers
Rhys ComptonUniversity of Waikato, Eibe FrankDepartment of Computer Science, University of Waikato, Panos Patros, Abigail KoayUniversity of Waikato
DOI Pre-print Media Attached
14:24 - 14:36
Live Q&A
Technical Papers
Abdulkarim KhormiFlorida State University, USA - Jazan University, KSA, Mohammad AlahmadiFlorida State University, Sonia HaiducFlorida State University
Pre-print Media Attached
14:36 - 14:48
Live Q&A
Technical Papers
Gustavo PintoUFPA, Breno MirandaFederal University of Pernambuco, Supun DissanayakeThe University of Adelaide, Marcelo d'AmorimFederal University of Pernambuco, Christoph TreudeThe University of Adelaide, Antonia BertolinoCNR-ISTI
Pre-print Media Attached
14:48 - 15:00
Live Q&A
Technical Papers
Sakib HaqueUniversity of Notre Dame, Alexander LeClairUniversity Of Notre Dame, Lingfei WuIBM Research, Collin McMillanUniversity of Notre Dame
Pre-print Media Attached
16:00 - 17:00: Developer CollaborationTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom
Chair(s): Bogdan VasilescuCarnegie Mellon University

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

16:00 - 16:10
Live Q&A
Technical Papers
Pre-print Media Attached
16:10 - 16:20
Live Q&A
Technical Papers
Nicole NovielliUniversity of Bari, Fabio CalefatoUniversity of Bari, Davide DongiovanniUniversity of Bari, Daniela GirardiUniversity of Bari, Filippo LanubileUniversity of Bari
DOI Pre-print Media Attached
16:20 - 16:30
Live Q&A
Data Showcase
A: Esteban ParraFlorida State University, A: Ashley Ellis, A: Sonia HaiducFlorida State University
Pre-print Media Attached
16:30 - 16:40
Live Q&A
Registered Reports
A: Ingrid NunesUniversidade Federal do Rio Grande do Sul (UFRGS), Brazil, A: Christoph TreudeThe University of Adelaide, A: Fabio CalefatoUniversity of Bari
Pre-print Media Attached
16:40 - 16:50
Live Q&A
Data Showcase
A: Preetha ChatterjeeUniversity of Delaware, USA, A: Kostadin DamevskiVirginia Commonwealth University, A: Nicholas A. KraftUserVoice, A: Lori Pollock
Pre-print Media Attached
16:50 - 17:00
Live Q&A
Technical Papers
Yalin LiuUniversity of Notre Dame, Jinfeng LinUniversity of Notre Dame, Jane Cleland-HuangUniversity of Notre Dame
Media Attached
16:00 - 17:00: Visions & ReflectionsTechnical Papers / Registered Reports / Keynote / MSR Awards / FOSS Award / Education / Data Showcase / Mining Challenge / MSR Challenge Proposals / Ask Me Anything at MSR:Zoom2
Chair(s): Venera ArnaoudovaWashington State University

Q/A & Discussion of Session Papers over Zoom (Joining info available on Slack)

16:00 - 16:15
Live Q&A
Technical Papers
Danielle GonzalezRochester Institute of Technology, USA, Thomas ZimmermannMicrosoft Research, Nachiappan NagappanMicrosoft Research
DOI Pre-print Media Attached
16:15 - 16:30
Live Q&A
Technical Papers
Nicolas GoldUniversity College London, Jens KrinkeUniversity College London
DOI Pre-print Media Attached
16:30 - 16:45
Live Q&A
Technical Papers
Ang JiaXi'an Jiaotong University, Ming FanXi'an Jiaotong University, Xi Xu, Di CuiXi'an Jiaotong University, Wenying Wei, Zijiang YangWestern Michigan University, Kai Ye, Ting LiuXi'an Jiaotong University
DOI Pre-print Media Attached
16:45 - 17:00
Live Q&A
Technical Papers
Pre-print Media Attached

Call for Registrations

Call for Registrations: MSR/EMSE Registered Reports

EMSE, in conjunction with the conference on Mining Software Repositories (MSR), is conducting a pilot RR track.

See the associated Author’s Guide. Please email the MSR track chairs - Neil Ernst or Janet Siegmund - for any questions, clarifications, or comments.

What are Registered Reports

  • Methods and proposed analyses are pre-registered and reviewed prior to research being conducted.
  • Reduce/eliminate under-powered, selectively reported, researcher-biased studies.

Two Phase Review

  • (MSR 2020) Phase 1: Introduction, Methods (including proposed analyses), and Pilot Data (where applicable). In Principle Acceptance.
  • (EMSE) Phase 2: full study, after data collection and analysis. Results may be negative!

Additional exploratory analyses in Phase 2 may be conducted, if they are justified. Any deviation from the protocol must be justified and logged in detail to ensure replicability. EMSE J. Editors reserve the right to tighten eligibility criteria if necessary.

Phase 1 Criteria

  • Importance of the research question(s).
  • Logic, rationale, and plausibility of the proposed hypotheses.
  • Soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
  • Clarity and degree of methodological detail for replication.
  • Will results obtained test the stated hypotheses?

Phase 2 Criteria (via https://osf.io/pukzy/)

  • Whether the data are able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls)
  • Whether the Introduction, rationale and stated hypotheses are the same as the approved Stage 1submission (required)
  • Whether the authors adhered precisely to the registered experimental procedures
  • Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative
  • Whether the authors’ conclusions are justified given the data

Qualitative Research and RR

  • No reason to assume pre-registration cannot be for qualitative methods such as card-sorting, grounded theory, coding, member checking etc.
  • E.g. phase 1 may include details on survey respondents, survey instrument design, data collection techniques.
  • OSF Qualitative Pre-Registration

Organizers

Publicity

  • Norman Peitek

Program Committee

See sidebar.

Timeline

DateMilestone
January 10, 2020 study protocols and plans due
January 31, 2020 initial protocol reviews
February 14, 2020 rebuttals/clarifications due
March 2, 2020 In Principle Acceptance (IPA) decision notifications
March 16, 2020 summary plan / camera-ready
March 31, 2020Phase 1 Reports registered with OSF registry
tbd Phase 2 submitted to EMSE (deadline to be determined)

Submissions

Submit via EasyChair. EasyChair will be used to handle Phase 1 reviews and feedback/rebuttal. EMSE’s EditorialManager system will be used for the Phase 2 submissions, with OSF managing the registration. Reviews from Phase 1 will be shared with the Phase 2 reviewers.

Submission Details

Submissions should follow the ACM Conference Proceedings Formatting Guidelines. LaTeX users must use the provided acmart.cls and ACM-Reference-Format.bst without modification, enable the conference format in the preamble of the document (i.e., \documentclass[sigconf,review]{acmart}), and use the ACM reference format for the bibliography (i.e., \bibliographystyle{ACM-Reference-Format}). The review option adds line numbers, thereby allowing referees to refer to specific lines in their comments.

Follow the template requested in the author’s guide to MSR RR submissions.

Page limit for MSR is 4 pages including references.

Review will be unblinded or single blind. There will be a light-weight rebuttal phase, in which authors have the opportunity to clarify unclear parts of the report. However, the rebuttal is not there to make changes to the experimental design.

FAQ

Q. How will self-plagiarism be handled?
A. Self-plagiarism is where an author includes verbatim text from other, already published work. We expect this to be managed using the existing workshop/extension model; there will be sufficient new content in Phase 2 to clearly indicate this is a new piece of work.

Q. What if I publish my Phase 1 proposal, and then someone scoops me by following the protocol?
A. In practice, this seems quite uncommon. However, OSF has mechanisms to manage embargo periods, so this might be something we also consider in the future. Currently the MSR/EMSE model makes embargos impractical. However, tracks such as “new ideas” already pose this potential risk, so we don’t anticipate extensive problems.

Q. How does this process deal with exploratory studies, where there is no well-defined hypothesis?
A. For now, we strongly suggest such studies target the New Ideas and Emerging Results Track of MSR: . We will focus on studies that have a clear, well-formulated hypothesis.

Q. What if my study changes as I gather data?
A. RR have flexibility to deviate from the analysis plan. However, authors will need to provide solid reasons as to why they deviated from the plan.

Other FAQs on RR in general are at the bottom of the OSF page.

Links

  1. https://cos.io/prereg/
  2. See these links

NB: Please contact the MSR RR track chairs with any questions, feedback, or requests for clarification. Specific analysis approaches mentioned below are intended as examples, not mandatory components.

Title (required)

Provide the working title of your study. It may be the same title that you submit for publication of your final manuscript, but it is not a requirement.

  • Example: Should your family travel with you on the enterprise?
  • Subtitle (optional): Effect of accompanying families on the work habits of crew members

Authors (required)

At this stage, we believe that an unblinded/single blind review is most productive

Structured Abstract (required)

The abstract should describe in 200 words or so:

Background/Context

What is your research about? Why are you doing this research, why is it interesting?

Example: The enterprise is the flag ship of the federation, and it allows families to travel onboard. However, there are no studies that evaluate how this affects the crew members.”

Objective/Aim

What exactly are you studying/investigating/evaluating? What are the objects of the study?

We welcome both confirmatory and exploratory types of studies.

Example: We evaluate whether the frequency of sick days, the work effectiveness and efficiency differ between science officers who bring their family with them, compared to science officers who are serving without their family.

Example: We investigate the problem of frequent Holodeck use on interpersonal relationships with an ethnographic study using participant observation, in order to derive specific hypotheses about Holodeck usage.

Method

How are you addressing your objective? What data sources are you using.

Example: We conduct an observational study and use a between subject design. To analyze the data, we use a t test or Wilcoxon test, depending on the underlying distribution. Our data come computer monitoring of Enterprise crew members.

Limitations

Hypotheses / research questions (required)

Clearly state the research hypotheses that you want to test with your study, and a rationalization for the hypotheses.

  • Example: Science officers with their family on board have more sick days than science officers without their family

  • Rationale: Since toddlers are often sick, we can expect that crew members with their family onboard need to take sick days more often.

Introduction

Give more details on the bigger picture of your study and how it contributes to this bigger picture. An important component pf phase 1 review is assessing the importance and relevance of the study questions, so be sure to explain this.

Variables (required)

  • Independent Variable(s) and their operationalization
  • Dependent Variable(s) and their operationalization (e.g., time to solve a specified task)
  • Confounding Variable(s) and how their effect will be controlled (e.g., species type (Vulcan, Human, Tribble) might be a confounding factor; we control for it by separating our sample additionally into Human/Non-Human and using an ANOVA (normal distribution) or Friedman (non-normal distribution) to distill its effect).

For each variable, you should give: - name (e.g., presence of family) - abbreviation (if you intend to use one) - description (whether the family of the crew members travels on board) - scale type (nominal: either the family is present or not) - operationalization (crew members without family on board vs. crew members with family onboard)

Material/objects (required)

Describe any material that you plan to use, be specific on whether you developed it (and how) or whether it is already defined (e.g., a standard myers-briggs-type indicator)

Example: For sick days, we recruit the medical records from sick bay (ethics approval pending). For efficiency, we conduct standard interviews with the superior officer and crew members. The questions are the following: / can be found on the Web site / Appendix. Furthermore, we observe their performance during a simulated task.

Tasks (optional)

If you use tasks, describe them, how they were designed or from where they are taken and why they are suitable to evaluate the hypotheses / research question

Example: For effectiveness of the crew members, we ask them to sweep a class 2 nebula. We simulate an error in the primary sensory array. Crew members should then run a level 3 diagnostic to spot the error, fix the error, and complete the sweep of the nebular. We measure the time to (i) spot that there is an error, (ii) decide on the correct diagnostic protocol, (iii) fix the error, and (iv) complete the sweep.

Participants/Subjects/sample (required)

Describe how and why you select the sample. When you conduct a meta analysis, describe the primary studies / work on which you base your meta analysis.

Example: We recruit crew members from the science department on a voluntary basis. They are our targeted population.

Execution Plan (required)

Describe the experimental protocol.

Example: Each crew member needs to sign the informed consent and agreement to process their data according to GDPR. Then, we conduct the interviews. Afterwards, participants need to complete the simulated task.

Analysis Plan (required)

Descriptive statistics

How do you describe the data? How do you handle outliers?

Example: To represent the number of sickdays, we use histograms. Dependending on the distribution, we remove values that are 2 standard deviations above the mean as outliers (normal distribution). If the data are non-normal, we use the median and values below the 10th/above the 90th percentile.

How do you evaluate the practical significance of the hypotheses?

How are you testing the significance of your results? Be specific about the epistemological paradigm and statistical paradigm you are using. This will help us assign reviewers familiar with the relevant research strategies. See Neto et al. for more information.

  • Example: (Frequentist) To test for normality, we use a Shapiro-Wilk test. For efficiency, we use a t test / Wilcoxon test depending on the distribution. To evaluate the effect of species type, we use a two-way ANOVA / Friedman test, depending on the distribution.
  • Example: (Bayesian) We derive a posterior predictive distribution by choosing a weakly informative prior with sickdays modeled using a Poisson distribution, and likelihood of species influence modeled using a normal distribution with mean 0 and s.d. \sigma. We then calculate the 95% and 99% uncertainty intervals and median m and mean μ of the posterior.

Examples