OSF page for Experiment 2 of Mah et al. - ***A Direct Replication and Extension of Popp and Serra (2016, Experiment 1): Better Free Recall and Worse Cued Recall of Animal Names than Object Names, Accounting for Semantic Similarity*** See the "README" file in OSF Storage below for a description of the included files. Below this line is an informal "project record" (e.g., planning documents, preregistration drafts, pilot results, sequential testing milestones) --------------------------------------------------------------------------------------------------------------------------------------------------- Records Google doc (e.g., Word Selection process): https://docs.google.com/document/d/1IgxIRkxdAmbhMLFJWwPwnzHjfCqwkVJ9_WLGTunIPS8/edit?usp=sharing [ARCHIVED] Preregistration draft: https://docs.google.com/document/d/18oKBq7jxl-4Pz7zZ6iQBIVeflmaU83QU7oXk0UzGi1s/edit Post-programming-error preregistration draft: https://docs.google.com/document/d/1qrONfRBnHY6bCxGJGIn9A4jW9BFeI-Jxpa955iE2S60/edit?usp=sharing **Preliminary SONA pilot testing report: November 9th, 2020** - ***N* = 6** - No exclusions for: Distractions, words understood < 75%, 0 correct on any lists, fast CR skips - However, all 6 participants ended up in the "Animal advantage" condition. Eric tested the program and was able to get assigned to the "Equal" condition, so it could be a fluke with the preliminary pilot sample - Patterns promising thus far (FR animacy advantage, CR object advantage, performance away from floor/ceiling): @[osf](b64j3) ------------------------------------------------- ------------------------------------------------- **Preliminary Prolific pilot testing report: November 18th, 2020** - ***N* = 7** (10 participants signed up, 9 entered the correct completion code, but only 6 uploaded data. After prompting, one more participant uploaded their data. May need to remind Prolific participants to upload data at the end of the survey. Also noticed that many more people signed up but then opted out than is usual w/Prolific studies. Likely due to extra steps involved in completing the study). - No exclusions for: Distractions, words understood < 75%, fast CR skips. 1 exclusion for low performance (Participant scored 0 on one free recall list) - 4 participants in the "Animal advantage" condition, 3 in the "Equal similarity" condition. - Performance seems to be a bit lower than SONA sample, but still not at floor. Hints of expected effects in "Animal advantage" similarity condition. Only 2 non-excluded participants in the "Equal" similarity condition, so hard to draw any conclusions there: @[osf](xvpwm) (Small points = individual data points) - But, seems like data collection via Prolific would be viable ------------------------------------------------- ------------------------------------------------- **Data collection deadline report: January 25th, 2021** - As of our initial data collection deadline of January 25th 11:59 PM, we had complete data from 143 participants. - EM performed preliminary data preprocessing to determine exclusion rates: 6 participants who reported a major distraction, 16 who reported understanding less than 75% of the studied words, 2 who reported substantial technical difficulties, and 3 who had >= 5 "fast skips" on at least one CR list. - With overlapping exclusions, our dataset included 120 participants. - Because of our preregistered lower-bound sample size of 150, we decided to reopen data collection. **Extended data collection deadline report: February 6th, 2021** - After this deadline, we had post-exclusion data from *N* = 153 participants. - This was from a sample of 179 participants with complete data files. Of these, we made the following exclusions: 7 who reported a major distraction, 18 who reported understanding less than 75% of the words they studied, 5 who reported a substantial technical difficulty, and 6 who had >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - In the complete sample, participants provided 3,577 FR responses, 270 of which required manual checking for correctness (of those, two coders agreed on 87% with the remaining 13% resolved by a third coder) - In the complete sample, participants provided 4,200 CR responses, 699 of which required manual checking for correctness (of those, two coders agreed on 94% with the remaining 6% resolved by a third coder) - However, during preprocessing we came across a critical programming error that we had previously failed to detect. That is, participants got equal #s of animal/object words across the two CR/FR lists, but *within each list* the relative #s of animal/object words varied (e.g., a participant might get 6 animals/10 objects on FR list 1, then 10 animals/6 objects on FR list 2, 9 animals/7 objects on CR list 1, 7 animals/9 objects on CR list 2). - In light of this programming error, we decided to halt the preregistration process, modify our planned analyses, and to treat this experiment as more exploratory. More details can be found in the "SemanticAnimacy_2020_ArchivedRegistration" document. - However, we analyzed results in the full sample (and in subsets restricting the max animal/object ratio to 10:6 and 9:7). Results for these analyses can be found here: @[osf](r4hse) ------------------------------------------------- ------------------------------------------------- **Prolific pilot II: February 28th, 2021** - Prior to preregistering the corrected program experiment, we recruited *N* = 50 participants on Prolific with the new program to test the feasibility of online testing on a larger scale than the previous pilot. - We recruited 50 participants, received complete data from 40, but ended up excluding a substantial proportion of this: 1 who reported understanding less than 75% of the words, 1 participant who reported a major distraction, 3 who reported cheating, 10 who didn't get at least 1 correct on all lists, and 2 participants with >= 5 "fast skips" on any CR list, leaving us with a final *N* = 26 - We decided to treat these data as pilot data, and made several small revisions to the program/instructions with the aim of increasing the inclusion rate. However, we did modify our preregistered sampling strategy (lower total *N* of 150 rather than 200 and sequential tests at every *n* = 10 rather than *n* = 20) in light of the high anticipated exclusion rates. - We also conducted analyses of the pilot data to check for any anomalous findings (e.g., extreme performance on the tests, lack of variability). These results are shown below: @[osf](6xmbz) - Because results were generally within the realm of what we might expect, we opted to continue data collection via Prolific. **Prolific Main Sample: 1st sequential testing threshold** - We collected a post-exclusion sample of *N* = 101 Prolific participants (one more than our 1st sequential testing threshold). - This was from a sample of 173 participants recruited, of which 134 uploaded complete data files. Of these 134, we made the following exclusions: 2 who reported a major distraction, 6 who reported cheating, 11 who didn't get at least one correct on all lists, 16 who reported understanding < 75% of the words, and 10 who have >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. Results for automatic and manual coding of errors were similar (i.e., we met the BF thresholds under both coding methods), so we report only the manual coding results here. Results are shown in the figure below: @[osf](qn67g) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Main effect of test, F(1, 54) = 15.05, p < .001 - Interaction b/w test & animacy, F(1, 54) = 12.02, p = .001 - Non-sig. FR animacy effect: t(54) = 2.01, p = .05 (Wilcox p = .05), **but** BF10 = 10.24 - Sig. CR reverse animacy effect: t(54) = 2.91, p = .01, BF10 = 9.87 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 99) = 4.31, p = .04 - Non-sig. interaction b/w condition & animacy, F(1, 99) = 3.34, p = .07 - Sig. CR reverse animacy effect in “Animals-more-similar”: t(54) = 2.91, p = .01, BF10 = 9.87 - Non-sig. CR reverse animacy effect in “Equal”: t(45) = .15, p = .88, BF01 = 61.73 - Because we met the BF thresholds, we stopped data collection. These results are striking, and generally support H2, but differ from the results we observed in the student sample with the programming error (i.e., lack of support for H2). ------------------------------------------------- ------------------------------------------------- **Prolific 2nd sample (EFL only): 1st sequential testing threshold (July 2nd, 2021)** - We collected a post-exclusion sample of *N* = 101 Prolific participants (one more than our 1st sequential testing threshold). - This was from a sample of 140 participants recruited, of which 118 uploaded complete data files. Of these 134, we made the following exclusions: 6 who reported understanding less than 75% of words, 1 who reported cheating, 9 who didn't get at least one correct on all lists, and 5 who have >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](esrdk) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 52) = 11.63, p = .001 - Non-sig. FR animacy effect: t(52) = 1.42, p = .16 (Wilcox p = .16), BF01 = 1.25 - Sig. CR reverse animacy effect: t(52) = 3.14, p = .002 (Wilcox p = .003), BF10 = 21.6 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 99) = 11.90, p < .001 - Non-sig. interaction b/w condition & animacy, F(1, 99) = 1.09, p = .30 - Sig. CR reverse animacy effect in "Animals-more-similar": t(52) = 3.14, p = .002 (Wilcox p = .003), BF10 = 21.6 - Non-sig. CR reverse animacy effect in “Equal”: t(47) = 1.74, p = .09 (Wilcox p = .13), BF01 = 5.9 - Because we did not meet the BF threshold for 1/3 of the critical comparisons, we continued data collection. **Prolific 2nd sample (EFL only): 2nd sequential testing threshold (July 8th, 2021)** - We collected a post-exclusion sample of *N* = 110 Prolific participants (Our 2nd sequential testing threshold). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](y3jcq) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 54) = 11.83, p = .001 - Non-sig. FR animacy effect: t(54) = 1.46, p = .15 (Wilcox p = .14), BF01 = 1.17 - Sig. CR reverse animacy effect: t(54) = 3.04, p = .004 (Wilcox p = .005), BF10 = 16.46 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 108) = 14.62, p < .001 - Non-sig. interaction b/w condition & animacy, F(1, 108) = .61, p = .44 - Sig. CR reverse animacy effect in "Animals-more-similar": t(54) = 3.04, p = .004 (Wilcox p = .005), BF10 = 16.46 - Sig. CR reverse animacy effect in “Equal”: t(54) = 2.33, p = .02 (Wilcox p = .03), BF10 = 1.17 - Because we did not meet the BF threshold for 2/3 of the critical comparisons, we continued data collection. **Prolific 2nd sample (EFL only): 3rd sequential testing threshold (July 12th, 2021)** - We collected a post-exclusion sample of *N* = 120 Prolific participants (Our 3rd sequential testing threshold). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](zb9em) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 61) = 8.51, p = .005 - Non-sig. FR animacy effect: t(61) = 1.47, p = .15 (Wilcox p = .14), BF01 = 1.57 - Sig. CR reverse animacy effect: t(61) = 2.41, p = .02 (Wilcox p = .03), BF10 = 2.83 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 118) = 10.47, p = .002. - Non-sig. interaction b/w condition & animacy, F(1, 118) = .13, p = .72 - Sig. CR reverse animacy effect in "Animals-more-similar": t(61) = 2.41, p = .02 (Wilcox p = .05), BF10 = 2.83 - Sig. CR reverse animacy effect in “Equal”: t(57) = 2.19, p = .02 (Wilcox p = .03), BF01 = 1.72 - Because we did not meet the BF threshold for any of the critical comparisons, we continued data collection. **Prolific 2nd sample (EFL only): 4th sequential testing threshold (July 15th, 2021)** - We collected a post-exclusion sample of *N* = 130 Prolific participants (Our 4th sequential testing threshold). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](8xpm2) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 65) = 10.10, p = .002 - Non-sig. FR animacy effect: t(65) = 1.73, p = .09 (Wilcox p = .08), BF10 = 1.08 - Sig. CR reverse animacy effect: t(65) = 2.53, p = .01 (Wilcox p = .02), BF10 = 3.31 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 128) = 11.69, p = .001. - Non-sig. interaction b/w condition & animacy, F(1, 128) = .09, p = .76 - Sig. CR reverse animacy effect in "Animals-more-similar": t(65) = 2.53, p = .01 (Wilcox p = .02), BF10 = 3.31 - Sig. CR reverse animacy effect in “Equal”: t(63) = 2.31, p = .02 (Wilcox p = .03), BF01 = 1.07 - Because we did not meet the BF threshold for 2/3 of the critical comparisons, we continued data collection. **Prolific 2nd sample (EFL only): 5th sequential testing threshold (July 15th, 2021)** - We collected a post-exclusion sample of *N* = 140 Prolific participants (Our 5th sequential testing threshold). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](cz5bg) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 70) = 11.79, p = .001 - Sig. FR animacy effect: t(70) = 2.12, p = .04 (Wilcox p = .04), BF10 = 4.75 - Sig. CR reverse animacy effect: t(70) = 2.37, p = .02 (Wilcox p = .02), BF10 = 2.55 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 138) = 13.92, p < .001. - Non-sig. interaction b/w condition & animacy, F(1, 138) = .02, p = .88 - Sig. CR reverse animacy effect in "Animals-more-similar": t(70) = 2.37, p = .02 (Wilcox p = .02), BF10 = 2.55 - Sig. CR reverse animacy effect in “Equal”: t(68) = 2.98, p = .004 (Wilcox p = .007), BF10 = 3.03 - Because we did not meet the BF threshold for 1/3 of the critical comparisons, we continued data collection. **Prolific 2nd sample (EFL only): Final sequential testing threshold (July 19th, 2021)** - We collected a post-exclusion sample of *N* = 150 Prolific participants (Our 5th sequential testing threshold). - This was from a sample of 213 participants recruited, of which 180 uploaded complete data files (missing files likely due to idiosyncratic computer setups, e.g., antivirus software preventing the creation of data files). Of these 180, we made the following exclusions: 7 who reported understanding less than 75% of words, 4 who did not report English as a first language, 1 who reported cheating, 17 who didn't get at least one correct on all lists, 2 who reported a technical difficulty that we deemed substantial, and 9 who had >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](rg8fn) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 78) = 15.01, p < .001 - Sig. FR animacy effect: t(78) = 2.29, p = .02 (Wilcox p = .02), BF10 = 3.76 - Sig. CR reverse animacy effect: t(78) = 2.70, p = .008 (Wilcox p = .009), BF10 = 5.90 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 148) = 16.40, p < .001. - Non-sig. interaction b/w condition & animacy, F(1, 148) = .004, p = .95 - Sig. CR reverse animacy effect in "Animals-more-similar": t(78) = 2.70, p = .008 (Wilcox p = .009), BF10 = 5.90 - Sig. CR reverse animacy effect in “Equal”: t(70) = 3.11, p = .003 (Wilcox p = .005), BF10 = 3.21 **Prolific 2nd sample (EFL only): Manual Error coding (September 3rd, 2021)** - 205 instances of FR commission errors (out of 3377 FR responses) that were manually checked to see if they were "close enough". Of these, two coders agreed on 191, with the remaining 14 adjudicated by a 3rd coder. Ultimately, 105 of the 205 commission errors were counted as correct after manual coding. - 825 instances of CR commission errors (out of 5728 CR responses) that were manually checked to see if they were "close enough". Of these, two coders agreed on 796, with the remaining 29 adjudicated by a 3rd coder. Ultimately, 109 of the 825 commission errors were counted as correct after manual coding. - These changes did not affect the # of exclusions for low list performance (*n* = 17). New results are shown in the figure below: @[osf](n7kwy) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 78) = 14.54, p < .001 - Sig. FR animacy effect: t(78) = 2.35, p = .02 (Wilcox p = .03), BF10 = 2.83 - Sig. CR reverse animacy effect: t(78) = 2.47, p = .02 (Wilcox p = .02), BF10 = 4.96 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 148) = 14.02, p < .001. - Non-sig. interaction b/w condition & animacy, F(1, 148) = .007, p = .93 - Sig. CR reverse animacy effect in "Animals-more-similar": t(78) = 2.47, p = .02 (Wilcox p = .02), BF10 = 4.96 - Sig. CR reverse animacy effect in “Equal”: t(70) = 2.90, p = .005 (Wilcox p = .008), BF10 = 4.60 ------------------------------------------------- ------------------------------------------------- **SONA 2nd sample: 1st sequential testing threshold (February 1st, 2022)** - We collected a post-exclusion sample of *N* = 100 SONA participants (our 1st sequential testing threshold). - This was from a sample of 124 participants. Of these, we made the following exclusions: 3 who reported a major distraction, 4 who reported a substantial technical difficulty, 6 who reported understanding less than 75% of words, 2 who reported cheating, 7 who didn't get at least one correct on all lists, and 8 who had >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](t4f5w) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 51) = 25.65, p < .001 - Sig. FR animacy effect: t(51) = 2.27, p = .03 (Wilcox p = .03), BF01 = 1.12 - Sig. CR reverse animacy effect: t(51) = 4.20, p < .001 (Wilcox p < .001), BF10 = 577.65 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 97) = 32.34, p < .001 - Non-sig. interaction b/w condition & animacy, F(1, 97) = .01, p = .92 - Sig. CR reverse animacy effect in "Animals-more-similar": t(51) = 4.20, p < .001 (Wilcox p < .001), BF10 = 577.65 - Sig. CR reverse animacy effect in “Equal”: t(46) = 3.85, p < .001 (Wilcox p = .001), BF10 = 1.69 * 10^4 - Because we did not meet the BF threshold for 1/3 of the critical comparisons, we continued data collection. **SONA 2nd sample: 2nd sequential testing threshold (March 3rd, 2022)** - We collected a post-exclusion sample of *N* = 110 SONA participants (our 2nd sequential testing threshold). - This was from a sample of 139 participants. Of these, we made the following exclusions: 5 who reported a major distraction, 5 who reported a substantial technical difficulty, 6 who reported understanding less than 75% of words, 2 who reported cheating, 8 who didn't get at least one correct on all lists, and 9 who had >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](yrbxf) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 59) = 31.56, p < .001 - Sig. FR animacy effect: t(59) = 2.22, p = .03 (Wilcox p = .04), BF10 = 1.59 - Sig. CR reverse animacy effect: t(59) = 5.01, p < .001 (Wilcox p < .001), BF10 = 4,555.15 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 108) = 35.76, p < .001 - Non-sig. interaction b/w condition & animacy, F(1, 108) = .29, p = .59 - Sig. CR reverse animacy effect in "Animals-more-similar": t(59) = 5.01, p < .001 (Wilcox p < .001), BF10 = 577.65 - Sig. CR reverse animacy effect in "Equal": t(49) = 3.54, p < .001 (Wilcox p = .002), BF10 = 6,831.68 - Because we did not meet the BF threshold for 1/3 of the critical comparisons, we continued data collection. **SONA 2nd sample: 3rd sequential testing threshold (May 30th, 2022)** - We collected a post-exclusion sample of *N* = 120 SONA participants (our 3rd sequential testing threshold). - This was from a sample of 152 participants. Of these, we made the following exclusions: 5 who reported a major distraction, 5 who reported a substantial technical difficulty, 10 who reported understanding less than 75% of words, 2 who reported cheating, 10 who didn't get at least one correct on all lists, and 9 who had >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](n78y2) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 62) = 33.45, p < .001 - Sig. FR animacy effect: t(62) = 2.37, p = .02 (Wilcox p = .03), BF10 = 2.79 - Sig. CR reverse animacy effect: t(62) = 5.17, p < .001 (Wilcox p < .001), BF10 = 7,392.39 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 118) = 35.91, p < .001 - Non-sig. interaction b/w condition & animacy, F(1, 118) = .78, p = .38 - Sig. CR reverse animacy effect in "Animals-more-similar": t(62) = 5.17, p < .001 (Wilcox p < .001), BF10 = 7,392.39 - Sig. CR reverse animacy effect in "Equal": t(56) = 3.40, p = .001 (Wilcox p = .002), BF10 = 2,338.73 - Because we did not meet the BF threshold for 1/3 of the critical comparisons, we continued data collection. **SONA 2nd sample: 4th sequential testing threshold (September 7th, 2022)** - We collected a post-exclusion sample of *N* = 134 SONA participants (our 4th sequential testing threshold). - This was from a sample of 168 participants. Of these, we made the following exclusions: 5 who reported a major distraction, 5 who reported a substantial technical difficulty, 10 who reported understanding less than 75% of words, 2 who reported cheating, 11 who didn't get at least one correct on all lists, and 11 who had >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). To get our *N* to 130, we also excluded from analysis the last 4 participants. - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](3q6jd) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 67) = 36.75, p < .001 - Sig. FR animacy effect: t(67) = 2.34, p = .02 (Wilcox p = .03), BF10 = 1.46 - Sig. CR reverse animacy effect: t(67) = 5.52, p < .001 (Wilcox p < .001), BF10 = 3.93*10^4 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 128) = 38.25, p < .001 - Non-sig. interaction b/w condition & animacy, F(1, 128) = 1.14, p = .29 - Sig. CR reverse animacy effect in "Animals-more-similar": t(67) = 5.52, p < .001 (Wilcox p < .001), BF10 = 3.93*10^4 - Sig. CR reverse animacy effect in "Equal": t(61) = 3.37, p = .001 (Wilcox p = .003), BF10 = 1,480.77 - Because we did not meet the BF threshold for 1/3 of the critical comparisons, we continued data collection. **SONA 2nd sample: 5th sequential testing threshold (September 12th, 2022)** - We collected a post-exclusion sample of *N* = 140 SONA participants (our 5th sequential testing threshold). - This was from a sample of 176 participants. Of these, we made the following exclusions: 5 who reported a major distraction, 5 who reported a substantial technical difficulty, 10 who reported understanding less than 75% of words, 2 who reported cheating, 11 who didn't get at least one correct on all lists, and 11 who had >= 5 "fast skips" on any CR list (note that some participants met multiple exclusion criteria). - We then conducted our key hypothesis tests against our pre-registered Bayes Factor (BF) criteria of BF > 3. We did not manually code errors. Results are shown in the figure below: @[osf](v8zh5) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 70) = 40, p < .001 - Sig. FR animacy effect: t(70) = 2.60, p = .01 (Wilcox p = .02), BF10 = 4.23 - Sig. CR reverse animacy effect: t(70) = 5.64, p < .001 (Wilcox p < .001), BF10 = 5.73 * 10^4 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 138) = 44.57, p < .001 - Non-sig. interaction b/w condition & animacy, F(1, 138) = .74, p = .39 - Sig. CR reverse animacy effect in "Animals-more-similar": t(70) = 5.64, p < .001 (Wilcox p < .001), BF10 = 5.73 * 10^4 - Sig. CR reverse animacy effect in "Equal": t(68) = 3.90, p < .001 (Wilcox p < .001), BF10 = 9.32 * 10^3 - Because we met the BF threshold for all critical comparisons, we stopped data collection. **SONA 2nd sample: Manual Error coding (September 20th, 2022)** - 304 instances of FR commission errors (out of 3439 FR responses) that were manually checked to see if they were "close enough". Of these, two coders agreed on 257, with the remaining 47 adjudicated by a 3rd coder. Ultimately, 172 of the 304 commission errors were counted as correct after manual coding. - 708 instances of CR commission errors (out of 3771 CR responses) that were manually checked to see if they were "close enough". Of these, two coders agreed on 667, with the remaining 41 adjudicated by a 3rd coder. Ultimately, 150 of the 708 commission errors were counted as correct after manual coding. - These changes resulted in 3 fewer participants being excluded for low list performance (*n* = 8). New results are shown in the figure below: @[osf](fs75d) - **H1 (Animacy and reverse animacy effects in the "Animals-more-similar" condition)**: - Interaction b/w test & animacy, F(1, 78) = 14.54, p < .001 - Sig. FR animacy effect: t(78) = 2.35, p = .02 (Wilcox p = .03), BF10 = 2.83 - Sig. CR reverse animacy effect: t(78) = 2.47, p = .02 (Wilcox p = .02), BF10 = 4.96 - **H2 (Reverse animacy effects in the "Animals-more-similar" and "Equal" conditions)**: - Main effect of animacy, F(1, 148) = 14.02, p < .001. - Non-sig. interaction b/w condition & animacy, F(1, 148) = .007, p = .93 - Sig. CR reverse animacy effect in "Animals-more-similar": t(78) = 2.47, p = .02 (Wilcox p = .02), BF10 = 4.96 - Sig. CR reverse animacy effect in “Equal”: t(70) = 2.90, p = .005 (Wilcox p = .008), BF10 = 4.60 ------------------------------------------------- -------------------------------------------------