Main content



Loading wiki pages...

Wiki Version:
Structured Abstract ------------------- **Background.** User studies in cyber security in the widest sense often rely on statistical inference to show that effects seen in (quasi-)experiments and surveys are significant, and that the null hypothesis can be rejected. In such Null Hypothesis Significance Testing (NHST), the community relies on sound reporting to ascertain the credibility of the results. **Aim.** We investigate the prevalence of statistical misreporting as well as the relation to publication venue and year. **Method.** Based on a systematic literature review of user studies in cyber security from selected venues in the 10 years 2006-2016, we will evaluate that prevalence of statistical misreporting using the **R** package **statcheck**. We will offer a systematic quantification of insufficient reporting, reporting inconsistencies and decision errors. We further conduct correlational ordinal/multinomial logistic regressions to establish the relation to publication venue and year. **Anticipated Results.** We anticipate descriptive statistics and graphs of the prevalence of statistical misreporting. We further intend to obtain logistic-regression models on the relation of predictors Venue and Year on coded **statcheck** outcomes in an correlational study. **Anticipated Conclusions.** We anticipate a systematic overview of statistical misreporting over time and venues, yielding an evidence-based estimate how how we are performing as a field as well as what our trajectory is. **Available Publications.** This work is published as "Fidelity of Statistical Reporting in 10 Years of Cyber Security User Studies." to appear in proceedings of the 9th International Workshop on Socio-Technical Aspects in Security (STAST'2019), LNCS 11739, Springer Verlag, 2019, pp. 1-24. An extended technical report is available on ArXiv: [Fidelity of Statistical Reporting in 10 Years of Cyber Security User Studies, 2004.06672][1] **Acknowledgement.** This work was supported by the ERC Starting Grant Confidentiality-Preserving Security Assurance (CASCAde, GA no 716980). [1]: http://arXiv:2004.06672
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.