Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
Background Null hypothesis significance testing (NHST) causes researchers to draw erroneous conclusions from their data. NHST is still ubiquitously used as a default procedure to make inferences about population effects. Despite it’s popularity, NHST has received much criticism in the past decades. The core of this criticism is that NHST is a form of indirect inference; a conclusion about a population effect (the null hypothesis) is drawn from the p-value that provides the probability of the data or more extreme data, given the null hypothesis. However, the probability of a null population effect depends greatly on the prior probability of a null-hypothesis being ‘true’. This is an important underlying reason that many conclusions from research are false and it makes NHST unsuitable as the default procedure to draw conclusions from empirical data. A number of alternatives have been developed that overcome this pitfall, such as Bayesian inference methods, informative hypothesis testing and a-priori inferential statistics. However the uptake of these methods has been limited. Therefore, the questions are why many scientists keep on using NHST and how this information can be used to create a paradigm shift in the scientific community. Aims 1. To explore views on NHST among all relevant stakeholders in the scientific system (first 14 months). 2. To develop strategies to implement use of alternative methods for drawing conclusions from empirical data (last 10 months). Methods Theoretical framework The theoretical framework for this study is the diffusion of innovation theory of Rogers. In this proposal the innovation consists of the idea that our alternatives provide more valid and more responsible methods for drawing conclusions from empirical data than NHST. An important feature of Rogers’ theory is that it distinguishes between phases in the adoption process, which coincide with different types of people. In this project we will specifically focus on early adopters. They form the second group to adopt an innovation, after the innovators themselves. This group involves a great deal of opinion leadership and serves as role models for the rest of the system. Strategy The project consists of two phases, largely coinciding with the two research aims. Phase I (months 1-14): At the start of the study we will form an interest group that will provide input for a list of alternatives for drawing conclusions from empirical data. Next, we will conduct interviews with all relevant stakeholders from the science system. We aim to enroll 5-10 editors, 8-10 senior researchers, 8-10 junior researchers, 8-10 lecturers in statistics and 5-7 program leaders from funding agencies. The topic list for the semi-structured interviews will contain key characteristics from Rogers’ theory as well as topics derived from the theory of planned behavior. This will result in information on the perceived barriers and facilitators and will also allow us to identify early adopters. Subsequently we will organize four focus groups (one per stakeholder group) to validate and also to enrich and further explore the data gathered during the interviews. Phase II (months 15-24): We will organize three search conferences with the ‘early adopters’ as identified in the first phase of the project complemented by other early adopters if needed. A search conference is a participative planning method that enables people to create a strategy for the most desirable future, in our case, with respect to the way conclusions are drawn from empirical data. The participants take responsibility for carrying out the strategy themselves. The potential barriers and facilitators identified from the first phase of the study serve as input for these search conferences. A total of 20-30 representatives of all four groups of stakeholders will be included. In the first search conference we will discuss the outline of the new paradigm for drawing conclusions in empirical sciences. Information from the interviews and focus groups will be used for each of the alternatives from the perspective of the different stakeholder groups. The second search conference will address a strategy to achieve the desirable paradigm shift in research practice, the result of which will become visible in future published material. The third search conference will address the way in which educational programs need to be adapted in order to teach students the desirable alternative way(s) of drawing conclusions from empirical data. The deliverable of the search conferences is a proof of concept for the strategy, ready to be incorporated in a specific implementation plan. The final months of the project will be spent on processing the input form the search conferences into specific implementation plans and for other implementation activities.
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.