Call for Papers (PDF)
With the steep increase of empirical studies and the advent of secondary studies (aggregation or meta-study), one main concern raised by recent studies, is the quality of the primary studies (e.g., a single experiment). Concerns about different quality issues have been raised for many years, however, due to the complexity of empirical studies, not all of them seem to be solved. In particular, problems with regard to the appropriate design of experiments and the correct usage of statistical tests are crucial for the trustworthiness of the results. Consequently, we need to better understand the issues and how to mitigate, if not remove them. The workshop focuses on the empirical process, which can be seen as one measure to ensure repeatable quality. It of six consecutive steps: Definition, Design, Implementation, Execution, Analysis, and Reporting. We also address research with regard to methodological issues arising during the execution of the stages of the process. The workshop’s aim is to increase awareness on the importance of the quality and appropriateness of the methods applied during the different stages of empirical studies to get a reliable answer from an empirical study and to disseminate this knowledge to the broader SE community.
Topics of interest include, but are not restricted to:
- Issues with regard to the motivation and definition of appropriate research questions.
- Issues with regard to the operationalization, e.g., missing standard metrics.
- Issues in the selection of appropriate experimental design and its consequences
- Issues in the systematic identification of threats to validity; and how to address them.
- Dealing with perceptions and biases.
- Drawing the appropriate conclusions from the findings and their generalization.
- Issues with replication, families of studies and their aggregation.
- Issues with quantitative, qualitative approaches and combined approaches.
Experience reports (8 pages max) describe the experience with conducting studies over a period of time and lessons learned that would be useful to be added to the methodological body of knowledge.
Technical papers (8 pages max) describe either studies conducted highlighting concrete results and the way they were designed and conducted to overcome challenges. Also expected in these papers are comparisons of the advances made against the state-of-the-art.
Vision / Position papers (4 pages max) are expected to describe new directions to follow in conducting empirical studies. Convincing arguments supported by clear rationale need to be included.
Papers must describe original work not submitted or presented at other forums. Accepted papers will be published in ICSE workshop proceedings. The official publication date of the workshop proceedings is the date the ICSE proceedings are made available in the ACM Digital Library. This date may be up to two weeks prior to the first day of ICSE 2018. The official publication date affects the deadline for any patent filings related to published work.
Keynote speaker and invited talks
Keynote by Per Runeson (U Lund) on “Studying the Human Side of Software Engineering”
Invited talks by Natalia Juristo (UP Madrid) and Dieter Rombach (Uni Kaiserslautern).