Review Process and Guidelines
**Preamble** The review shall be conducted in the spirit of preproducibility (https://www.nature.com/articles/d41586-018-05256-0). Authors shall put in their best efforts to make their work transparent and reproducible, and reviewers shall respond to such openness with respect and kindness, so that fascinating results are put forward for the advancement of science. Please see your review as a way to guide the researchers and citizens towards the best manuscript they can create, but also be fair and frank about shortcomings. **Process** - The organisers will create OSF projects for all submission (to be able to group the works within one “parent” project (https://osf.io/68uwn/). Authors submit their manuscript to the organisers, who briefly screen the submission, create a new component for it, and assign reviewers. - Reviewers may use the commenting functionality for short/overall remarks (see http://help.osf.io/m/collaborating/l/524111-comment-on-a-project), or provide a PDF with the review and more extensive comments/suitable formatting to the organisers, who will upload it to the OSF project. - If reviews are provided as documents and not as public comments, the organisers will upload them once both reviews are completed. *(added by DN, 2019-07-01)* - We welcome a discussion within the comments or in other form between the authors and reviewers, as long as the results of that communication are added to the OSF project (e.g. in form of a rebuttal letter (not a bad thing to happen) or a text file containing an exchange via e-mail or chat. - The programme committee chairs will make a decision (accept, accept with revision, reject) based on the reviews and comments. - Manuscripts accepted with revisions must be updated by the authors, who may also reply to reviewer comments in a separate document. The revision will be evaluated by the programme committee chairs; original reviewers are invited but not required to evaluate the second revision. *(update 2019-07-10)* - Accepted manuscripts and manuscripts accepted with revisions are invited for presentation at the conference, pending the final decision after submitting a revision until a given deadline. *(update 2019-07-10)* - Authors of rejected papers may reach out to reviewers for clarification or address the programme committee chairs with a rebuttal. *(update 2019-07-10)* - The component will be clearly marked as "accepted" or "preprint" in the component description. The accepted papers will be listed on a Wiki page on OSF and compiled to a proceedings document (PDF) to be published during the conference. *(update 2019-07-10)* - Authors of accepted papers are invited to provide additional data or material (e.g. presentation slides) within their respective projects in accord with the organisers. **Review** - Reviewers should not concern themselves with the format of the submission. - Please consider using the following criteria to structure the review and use keywords such as "high", "medium", "low", "not applicable". - Contribution to the call and relevance for the audience - Quality of the submission (language, organization of content, completeness, originality of the argument, relevance of the results) - Clarity (considering breadth of audience in CitSci) - Scientific rigour, transparency and reproducibility (data, code, references etc.) - Please include a clear recommendation in your review: "accept", "accept with revisions", "reject". A rejection should be clearly motivated, e.g. because a manuscripts needs major work and changes to the workflow, is out of scope, etc. - Please sign your review or reach out to the programme committee if and why you prefer to remain anonymous.