Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
1. Replication Research in Special Education by Bryan G. Cook, Suzanne McClain, Hannah Robinson, Danielle Waterfield, Latesha Watson, Joseph Boyle, Jesse Fleming, William Therrien, & Jason Travers Background and Purpose<br/> Lemons et al. (2016) and Makel et al. (2016) conducted independent reviews of replication studies in special education, respectively identifying 0.4% and 0.5% of publications in the special education literature as explicit replications (identified by authors using the term “replicat*” to indicate that the study was a replication). Lemons and colleagues required that replication studies specify an individual study was replicated, whereas Makel and colleagues considered studies to be replications based on any indication of replicating previous research. Given increased attention to replication research in the field of education (e.g., Banerjee, 2022; Cook et al., 2016; Chhin et al., 2018), we are conducting a systematic review to examine whether and how rates of replication studies published in special education journals have changed since 2014, when the previous reviews were conducted; examine characteristics of replication studies in special education since 2014; and examine differences in replication rates when using conservative (i.e., criteria used by Lemons et al.) and liberal (i.e., criteria used by Makel et al.) criteria for identifying replication studies. Research Questions and Hypotheses 1. What is the proportion of publications explicitly identified as replication studies in special education journals from 2015 to 2022 using both conservative and liberal criteria for identifying replications? Have replication rates of replication changed since 2014? 2. What proportion of replication studies in special education from 2015 to 2022 (a) are reported as successful, failed, or mixed replications; (b) are direct and conceptual replications; (c) use different research designs; and (d) have author overlap with the study(ies) being replicated? Have these proportions changed since 2014? We will examine this research question separately for replication studies identified using conservative and liberal criteria for replication studies. 3. Does replication success vary as a function of type of replication, research design, and author overlap? We will examine this research question separately for replication studies identified using conservative and liberal criteria for replication studies. Method<br/> This review is preregistered at https://osf.io/2n7x3 and is currently being conducted. Electronic search. Replicating Makel et al. (2016), we searched the 44 journals categorized by Journal Citation Reports (2021) as special education journals for “replicat*” using “any field” in the Clarivate Web of Science database from 1/1/2015 to 9/1/2022, resulting in 341 articles identified. For each journal, we also searched Web of Science for total number of publications (without any search term) from 1/1/2015 to 9/1/2022. The search was conducted in September 2022. Screening. We screened full texts of the 341 articles identified in the electronic search, with two presenters independently screening each article. The 168 articles that used replicat* to indicate the article was a replication study were included. Inter-rater agreement was 96%. Coding. We are currently coding articles to extract information on which studies meet conservative and liberal criteria for replications, types of replications reported, success of replication, research design, and author overlap. Analysis. We will use descriptive statistics and chi-square analyses to examine research questions. ---------- 2. The Open Scholarship Movement – Fostering to Think Beyond Open Access by Riya Thomas, Uttkarsha Bhosale, Shrutika Sirisilla, & Kriti Shukla With an aim to aid transparency and awareness of the advances in research, the scholarly community has increasingly adopted open scholarship practices. If recent developments in research are not openly available, it obstructs learning and innovation and hampers scientific progress. Over the past few years, with countless open access (one of the pillars of open scholarship) mandates from universities and research funding bodies, the scholarly communication landscape has changed substantially. More research is now readily available online to be freely downloaded, read, and repurposed. However, the open scholarship approach goes beyond making publications open access and the underlying data available to encompass the entire research process and to make all aspects of the research cycle accessible. For instance, simple compliance with open access publication is not sufficient to foster reuse and reproducibility in research. Repositories are a crucial component of research infrastructure, underpinning not just open access to publications but open data resources and the broader concepts of open scholarship that need to be more inclusive. As the understanding and need for open access, open data, open educational resources, and other open scholarship practices grow exponentially, it is found that widespread adoption of such practices is yet to be achieved to its full potential. This is probably due to the lack of knowledge and awareness of open scholarships among academics. We conducted a comprehensive research survey to understand the perceived value of the open science approach amongst researchers across the globe. Our review and analyses are evidence of the ways researchers are exploring open science beyond open access, the methods they follow to make their research more accessible, and their awareness of the significant benefits associated with it compared to the traditional models in research and education. Predictably, open access was the most familiar and used approach among all open scholarship practices while a significant group of respondents were unaware of or had not ever used open data repositories, open peer review, open sources, and open educational resources. This demands the need to educate and spread awareness of this dynamic mechanism. As open scholarship practices are a forward trend in the academic world and their policies often change or update, publishers must move toward open processes in their workflow to ensure greater transparency. They should invest and use upcoming technologies to add value to the existing publishing workflow and improve efficiency. Considering that some concerns around the adoption of current open scholarship practices are from the user perspective, there appear to be new opportunities for education publishers to help address these challenges. To facilitate a smooth transition, researchers can gain awareness from publisher-run training programs, masterclasses, editorial workshops, and community support sites. ---------- 3. Open Accessible Summaries in Language Studies: A case study in systematically making research results more accessible by Inge Alferink & Emma Marsden There is a well-documented gap between research and practice in the field of language studies (e.g., British Academy/Royal Society, 2018; Sato & Loewen, 2022). For example, despite generally positive perceptions of research, language teachers report having limited direct contact with research and research findings (Borg, 2009; Marsden & Kasprowicz, 2017; Nassaji, 2012). Key reasons teachers give are 1) practical – a lack of time and access, and 2) conceptual – academic papers can be difficult to read (Plavén-Sigray et al., 2017). In turn, researchers worry their research is not (or does not need to be) relevant to pedagogy. This contribution presents one large-scale initiative to bridge some research-pedagogy divides: Open Accessible Summaries in Language Studies (OASIS; oasis-database.org). OASIS aims to facilitate interaction between research and pedagogy, by making research into language learning, language teaching, and multilingualism physically and conceptually accessible. The one-page OASIS summaries, written in non-technical language, provide information about what the study was about, its importance, and what the researcher(s) did and found. Four major international journals now require all authors to write these summaries and many others encourage their authors to write them. All OASIS summaries are freely available from our database and are published under a CC-BY-NC-SA 4.0 license. In this lightning presentation, we introduce the OASIS initiative and its author and journal driven set up. We discuss the initiative’s aims to affect systematic change to make open, accessible studies part of the fabric of academic publishing in language education and the wider area of language studies. We also highlight some challenges we have encountered along the way. ---------- 4. Open Science at the University of Toronto by Madelin Burt-D'Agnillo This lightning talk will focus on an ongoing, exploratory study of Open Science at the University of Toronto. The project is guided by Mindy Thuna (Associate Chief Librarian Science Research & Information), with support from Madelin Burt-D'Agnillo (Toronto Academic Library Intern and Master's of Information student). This talk will provide an overview of our goals, what we've accomplished, and some preliminary insights into our findings. Our project asks open-ended questions about Open Science at the University: who, what, where, when, and why? We want to understand the ways in which university-affiliated individuals learn about, think about, and interact with Open Science. Our objective is to map out the landscape of Open Science initiatives across the University's three campuses, numerous disciplines, and roles. We obtained ethics clearance for this study from the University. From March – October 2022, we conducted 44 semi-structured interviews with faculty members, student researchers, librarians, and administrators. Open Science principles have been applied to this study; we committed to maintain participant’s confidentiality while also striving to create an open data set of interview transcripts. We are actively coding and analyzing transcripts now, and we intend to publish our results in summer 2023. ---------- 5. Starting Research Right: RDM for Reproducible Research by Monique Grenier In this lightning talk I will advocate for and offer lessons learned from teaching an introductory RDM session to grad students and faculty researchers at the beginning of project terms. By first evaluating the reproducibility crisis and discussing foundational data management practices, I ensure participants leave the session with actionable tasks that help facilitate reproducible research which in turn ensures their research projects are well established to promote sharing and reuse later on in the research lifecycle. ---------- 6. Breaking Barriers to Open Educational Resources – Key to Ensure Equity and Quality by Riya Thomas, Shrutika Sirisilla, Uttkarsha Bhosale, & Kriti Shukla Over the past few years, most institutions, publishers, and educators have pivoted their knowledge transfer practices and made a gradual yet perpetual shift toward digital educational resources. This has led to the rise in the creation and adoption of Open Educational Resources (OER) especially because of the proven value of OERs in addressing issues of educational equity. As open access, open data, open educational resources, and other open scholarship practices grow exponentially, it is found that widespread adoption of such practices is yet to be achieved to its full potential. In a recently conducted global survey by the Enago Academy to understand the perceived value of the open scholarship practices amongst researchers across the globe, it was observed that some researchers were still unaware of or do not practice the open scholarship approach for various reasons. However, after open access, OER was considered a familiar open scholarship approach, while about 66% of the respondents were still either unaware of or had never used OER. The lack of awareness or adoption of OER could be due to several unresolved problems like difficulty in discovering the resources, issues related to the financial self-sustainability of OER programs, and perception of lower quality resources as they are freely available. Awareness regarding searching the internet by license and learning outcome would be a significant step forward for making OER easier to find and use. Though the idea of OER is relatively young, the advocates of OER are striving to support and ensure the sustainability of OER with robust research. While there are many obstacles to be tackled if this latest educational technology is to fulfill its potential, the first step to supporting it is to encourage and share awareness for a smooth transition and to support organizations and businesses that contribute to the OER world with state-of-the-art research and publishing-related educational services, including the blogs, interactive webinar sessions, on-demand workshops, and consultation and mentorship for academic scholars. This session will introduce various modes of learning and how Enago educational services support all aspects of open scholarship. ---------- 7. Developing and disseminating student projects with lay audiences (Display Project)by Sarah MacQuarrie In line with ethical regulations participants receive debriefing detail following participation. However, debriefing can only report intended, rather than actual outcomes. This project addressed this barrier and explored what and how engagement can occur within student research projects. This talk will focus on the open access website developed to share and disseminate summaries written by students about their own student research and explore open science principles embedded in this project. This project introduces a means for higher education institutions to interact and engage with communities through sharing student research projects as well embed dissemination and openness principles within projects completed by students during their education studies. Such activity offers multiple benefits. Students are supported in preparing for life after university by strengthening their communication skills. Students appear distinctive by virtue of their achievements combined with their research and knowledge translation skills documented in real-life research projects. Students and staff will acquire a greater understanding regarding their role, recognise their value is appreciated on many levels and realise their contribution is vital for education and institutional goals to be met. By providing an easy to use open access website, an enhanced dissemination route is provided to support translation of knowledge and make evident the value of participants and organisations that contribute to students’ success in higher education. ---------- 8. Open Scholarship Practices: A Conversation for Graduate Students and Their Advisors by Ashley Taconet Open scholarship practices can and should be implemented in every stage of someone’s career including as a graduate student. This lightning talk will discuss ways mentors or advisors can include graduate students in open scholarship practices as well as ways for graduate students to implement these practices into their own research. The talk will start by discussing materials that advisors can share with graduate students to build an understanding of the importance of open scholarship practices. Next, the presenter will share examples of ways their mentor included them in projects that used open scholarship practices including pre-registering studies, submitting registered reports, and a literature review of open scholarship practices for content specific articles. This will allow mentors to reflect on their own advising practices to see if they can increase involvement in open scholarship practices for their advisees. Finally, the presenter will share their experience pre-registering their Educational Psychology-Special Education dissertation study on the Open Science Framework, ways that it is like current dissertation practices, and ways that it is different from current dissertation practices. To close, other areas where graduate students can increase open scholarship practices in their own personal research will be discussed. ---------- 9. A 4D Framework to Translate Bioscience Research to the Teaching Space by Xyanthine Parillon The current field of open science fully supports the biosciences by focusing on transparency of process. How the field of bioscience can use open science tools to translate new discoveries among teams to learners is unclear. Here twelve open source digital tools will be provided that allow documentation for projects, curriculum, and results in a linear feedback mechanism. The feedback of learners as an accountability perspective will be linked to initial synthesis to drive research process. In this session the four dimensions of data, research, evaluation, and teaching will be highlighted in one figure depicting how to translate research into the teaching space. These four dimensions will be aligned with free digital platforms. Digital platforms as bioscience repositories, sharable workflows, in-class evaluation platforms, and dissemination platforms will be displayed in the 4D framework. Open source digital platforms will be accessible to participants with a QR code. Participants will leave with specific open source digital platforms that allow synthesis for research and/or research teams with data and evaluation to be translated to learners and will be able to devise their own open source plan. ---------- 10. Special Education Research Accelerator Pilot Results by William Therrien, Bryan Cook, Christina Taylor, & Vivian Wong Like research in other fields, educational research reflects bias and error that can compromise the validity of findings and may result in the application of ineffective policy and practice. Conducting crowdsourced research is one innovation related to open science that may help the special education field mitigate some sources of bias in our research practices. Crowdsourcing can involve multiple laboratories conducting the same study. In other fields, crowdsourcing has been used to great effect. For example, in one of the Many Labs projects, crowdsourcing was used to conduct systematic replication of 28 effects across > 15,000 participants (Klein et al., 2018) to advance understanding of whether and how findings replicate. Crowdsourced research also has the potential to be impactful in special education. Democratization of the research process is likely one of the greatest benefits of conducting crowdsourced research in special education. Crowdsourcing enables many researchers to participate collaboratively in rigorous, large-scale projects. Increased inclusion of researchers promotes greater diversity in the student participant samples and increases the generalizability of study findings. Other benefits of conducting crowdsourced research include ensuring that studies are sufficiently powered; this is especially beneficial in studies with low-incidence disability populations. Ultimately, the use of crowdsourcing in special education can increase our ability to identify and examine the boundaries of effectiveness for evidence-based practices for students with disabilities. Supported by an IES grant, we developed the Special Education Research Accelerator (SERA) as a platform designed to enable education researchers to engage in crowdsourced projects. Modeled on the Psychological Research Accelerator (Moshontz, et al., 2018), SERA is designed to facilitate the implementation of large-scale crowdsourced research projects utilizing researchers and participants from across the U.S. (and potentially the world). Over the past 2 years, we used SERA to conduct a pilot study. Along with testing the feasibility of using SERA, the purpose of the pilot study was to conceptually replicate the findings of the Scruggs, Mastropieri, and Sullivan (1994) study with students with high-incidence disabilities (i.e., learning disabilities, emotional and behavior disabilities, ADHD, mild intellectual disabilities, and autism spectrum disorder without comorbid intellectual disabilities). The study evaluated the effectiveness of elaborative interrogation as a means of promoting relational thinking. Student outcomes were compared across two conditions: student-generated explanation and a control. The goal of the replication study was to examine whether findings of Scruggs and colleagues replicated in a larger sample of students with high-incidence disabilities, which provides important information regarding how these populations acquire and apply information. In this lightning talk, we will briefly describe the SERA platform, detail results from the pilot study and discuss next steps for SERA. [1]: https://tspace.library.utoronto.ca/handle/1807/108154
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.