**Three journal projects** ---------- ---------- 1. Journal outreach for Badges and TOP Guidelines (POC: David Mellor david@cos.io) 2. Use of Stat-Check at article submission (POC: Stephen Lindsay slindsay@uvic.ca) 3. Limits of Generalizability (POC: Dan Simons dsimons@illinois.edu and Yuichi Shoda yshoda@uw.edu) ---------- ### Journal outreach strategy for Badges and TOP Guidelines ### Point of Contact: David Mellor, Center for Open Science, david@cos.io Dear SIPS participant: We want to use your professional contacts to encourage psychology journals to award open-science practices badges (i.e., pre-registration, open data, open materials). **To that end we invite you to complete a [Google form][2] listing individuals you know who are in positions of influence on psychology journals and/or societies** (e.g., journal editors, members of publications committees, etc.) and to whom you are willing to send an email highlighting the benefits of such badges or TOP Guidelines. We will integrate your responses to create a spreadsheet indicating who is going to contact whom; later, we’ll get back to you with assignments. At that time, we will provide you with template language for this outreach. Please take some time to think about the people you know who might be open to your influence. If helpful, [here is a spreadsheet][4] of psychology journals with fields that indicate TOP signatory and Badges statuses. Note that you can submit multiple names at once, and can also return to the form and update it in future. If you can get to this while SIPS is still going on, that would be great! Thanks. SIPS Breakout Group 2: What Journals and Societies Can Do Link to form: http://goo.gl/forms/3rHpCdnMFI2GdyNh2 Link to spreadsheet: https://goo.gl/BtNcjG ---------- - [Spreadsheet][5] of journals, editor contact information, TOP and Badges status - [Form][6] (for editing) to poll SIPS attendees on their connections. - [Spreadsheet with Journals by Impact and by Publisher][7] (this is being combined with above spreadsheet) - [Editable link to draft outreach paragraph][8] ---------- ### Stat-Check at article submission ### Point of contact: Stephen Lindsay slindsay@uvic.ca - Stat-check automation service (Steven Lindsay is contacting Michelle Nuijten re using stat-check for submission) ---------- ### "State Limits of Generalizability" ### Points of contact: Dan Simons (dsimons@illinois.edu) and Yuichi Shoda (yshoda@uw.edu) - A white paper that discusses how widely individual results should be expected to generalize to different populations or circumstances. The paper is currently in revision as part of a journal review process. Please contact authors for questions. ---------- ## Steps already taken by journals, Prior to June 2016 ## - [Psych Science Badges][10] - [Psych Science word count policy][11]: Does't include method and results - [TOPS guidelines][12] - [Perspectives RRRs][13] - [Registered Reports][14] - New journals focusing on registered reports for original and replication research - (e.g., [Comprehensive Results in Social Psychology][15]) - [PLoS Open Data Policy][16] - [PLoS Open Materials Policy][17] - [Science guidlines for open data, code, materials (pdf)][18] ## Steps already taken by Psychology and closely aligned orgnanizations ## - [Psychonomic Society statistics guidelines][19] - [Steve Lindsey's Psych Science editorial][20] - [APA principles on data sharing][21] - [German Psychological Society][22] Open Data By Default policy - [Society for Neuroscience][23] discussion, training, and practice resource - [NIH principles][24] that journals shold follow (some concrete recommendations, including checklists. Mostly focused on clinical trial publications. ## Steps taken by other disciplines (less closely aligned) ## - [American Society for Cell Biology][25] white paper - [Federation of American Societies for Experimental Biology (FASEB)][26] recommednations (mostly about cell biology and animal research). - [American Physiological Society][27] resources on reproducibility in research ## Other possibilities ## - Pottery Barn Rule - [Sanjay Srivastava Post][28] - [German Psychological Society][29] Open by default data policy - from Schönbrodt email: Providing data doesn't guarantee co-authorship, but an incremental contribution should. - from Schönbrodt email: data providers must be informed when reanalyses of their data set are published (blogs, presentations, or papers).They cannot prohibit the publication, but they have a fair chance to react accordingly - Science guidelines note: "Science encourages authors to follow relevant standards for their field for reporting key aspects of the research design and analysis. If such standards were followed, authors should report how they were used. (The editors encourage the submission to Science of good examples of such research standards, and will compile a list of those that have community acceptance at Science’s website.)" - [Proposal for creating journal and funder scorecards][30] for implementation of transparency policies - [SIPS Pitch to get 90% of psychology journals to adopt and implement TOP Guidelines][31] [1]: https://osf.io/tvyxz/wiki/home/ [2]: http://goo.gl/forms/B9jCjW8Z6DW5Io3u1 [3]: https://cos.io/top [4]: https://docs.google.com/spreadsheets/d/15I5Idc-Vgm3Cq3ot5lE5IIDG7JbUVx_YVT6r6CzKmLM/edit?ts=5745e1f5#gid=0 [5]: https://docs.google.com/spreadsheets/d/15I5Idc-Vgm3Cq3ot5lE5IIDG7JbUVx_YVT6r6CzKmLM/edit?ts=5745e1f5#gid=0 [6]: https://docs.google.com/a/cos.io/forms/d/1-MrlkyMVQvVeFdJkanG_qQZCQuOpCp1AoYqn2Utbtr8/edit [7]: https://docs.google.com/spreadsheets/d/1RWc328me8NMK_gv-GQox3RqKY22yv5FjTiUo_PaKaTg/edit?usp=sharing [8]: https://docs.google.com/document/d/1tB7gNIdz0IFQJ1ZMVUgrkO0rR9JGSr_0Nz0X_GG8khQ/edit?usp=sharing [9]: https://docs.google.com/document/d/17KS4zU7--Gi5HqiOn-tIoSFMFTGaBDgO6PraSXFSTAM/edit?usp=sharing [10]: http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002456 [11]: http://www.psychologicalscience.org/index.php/publications/journals/psychological_science/ps-submissions [12]: https://cos.io/top/ [13]: http://www.psychologicalscience.org/index.php/replication [14]: https://osf.io/8mpji/ [15]: http://explore.tandfonline.com/cfp/beh/comprehensive-results-in-social-psychology [16]: http://journals.plos.org/plosone/s/data-availability [17]: http://journals.plos.org/plosone/s/materials-and-software-sharing [18]: http://science.sciencemag.org/content/sci/suppl/2016/06/01/352.6290.1147.DC1/aag2359-editorial.McNutt.SM.pdf [19]: http://www.springer.com/psychology?SGWID=0-10126-6-1390050-0 [20]: http://pss.sagepub.com/content/26/12/1827 [21]: http://www.apa.org/science/leadership/bsa/data-sharing-report.pdf [22]: http://www.dgps.de [23]: https://www.sfn.org/~/media/SfN/Documents/Advocacy/Research%20Practices%20for%20Scientific%20Rigor.ashx [24]: https://www.nih.gov/research-training/rigor-reproducibility/principles-guidelines-reporting-preclinical-research [25]: http://actualizestudiodev.com/ascb/wp-content/uploads/2015/11/How-can-scientist-enhance-rigor.pdf [26]: http://www.faseb.org/Portals/2/PDFs/opa/2016/FASEB_Enhancing%20Research%20Reproducibility.pdf [27]: http://www.the-aps.org/mm/SciencePolicy/Agency-Policy/Reproducibility [28]: https://hardsci.wordpress.com/2012/09/27/a-pottery-barn-rule-for-scientific-journals/ [29]: http://www.dgps.de [30]: https://docs.google.com/document/d/1t5GIzF6QsPECQjRESpwvKiLgOIi5sA1lAzAUoKH1X2I/edit# [31]: https://osf.io/emfkw/