## Research Questions ## **Replicability / Descriptive Research Questions** 1. Can the methods of Kristin Antelman’s 2004 paper “Do Open-Access Articles Have a Greater Research Impact?” be reproduced from the methods provided in that paper? 2. Do the results from the replication match those of Antelman's original study? 3. For each of Antleman’s subject areas, what proportion of documents from Antelman’s sample journals and publication years are freely available? 4. In each of Antelman’s subject areas, how frequently are the freely available articles the published version, and how frequently are they other versions? 5. In each of Antelman’s subject areas, what types of sites are hosting the freely available articles? 6. How much does the choice of database (Web of Science Core Collection vs. the “traditional” subset of Science Citation Index Expanded, Social Science Citation Index, and Arts & Humanities Citation Index) affect the citation counts, overall and by subject area? 7. Assuming we observe a citation advantage for open access, does the citation advantage disappear under alternative routes for testing -- specifically logistic regression with multiple covariates? **Hypothesis-Driven Questions** 1. Is there a statistically significant citation advantage (if any) for freely available articles from the Antelman sample journals and publication years) relative to toll-access articles? 2. Does the citation advantage change if an alternative data analysis method (e.g. logistic regression for cited / not cited rather than Wilcoxon signed rank test of citation counts) is used? 3. Does the choice of database (Web of Science Core Collection vs. the “traditional” subset of Science Citation Index Expanded, Social Science Citation Index, and Arts & Humanities Citation Index) affect the size or direction of any citation advantage found in each of the four subjects? ## Hypotheses ## **Replicability / Descriptive Research Questions** 1. We’ve already attempted a pilot test to reproduce the methods. In the pilot test, we found gaps in the description of the methodology and used professional judgement to fill those gaps. The descriptive results will describe the gaps that had to be filled in, both during the pretest and in the actual study. 2. We will reproduce the descriptive statistics, graphs, and tables produced in the Antelman report and discuss the similarities and differences between those results. 3. We will report the proportion of articles in each of the subject areas that are freely available and compare those to Antelman’s results without performing a statistical test of equality. 4. For each of the subject areas, we will report how frequently are the freely available articles the published version, and how frequently are they other versions and compare those to Antelman’s results without performing a statistical test of equality. 5. We will tabulate and describe the types of sites hosting the open access documents and will discuss the similarities and differences from Antleman’s results without a formal comparison. 6. The effect of the database selection on citation counts will be reported, e.g. change in number of citations or percent increase when additional databases are added. We do not plan to perform a statistical comparison among the subjects. 7. The same data used in the initial analysis will be reanalyzed using a logistic regression, progressively adding multiple covariates and reporting whether or how much it diminishes the size of the effect and whether the effect drops below statistical significance when multiple covariates are added. **Hypothesis-Driven Questions** 1. We predict a statistically significantly higher number of citations for the open access articles than for the closed access articles in all four subject areas. 2. We predict that some or all of this advantage will NOT be observed under a logistic regression, , i.e. that it will only be observed in some but not all of the subject areas. 3. We predict that although the total citation counts will be larger when more databases are included, , there still will be greater citations to open access articles in each of the four subject areas.