**Title:** Are jurors swayed by a high DNA likelihood ratio even in the presence of a strong alibi? **Collaborators:** Gianni Ribeiro, Jason Tangen, Blake McKimmie, Jason Chin. **Contact Information:** For more information, please contact the lead researcher: Gianni Ribeiro (g.ribeiro@uq.edu.au) Background and Rationale ------------------------ This experiment is part of a broader project where we are developing a general framework for how to communicate forensic evidence to non-experts, such as jurors. For over a century, forensic techniques such as fingerprint comparison have been used as evidence in court despite their validity and reliability being largely untested. Until recently, forensic examiners were permitted to testify that two prints, samples, marks, etc. “match” to the exclusion of all other possible sources. However, to date, there have been numerous exonerations on the basis of faulty or misleading forensic science (Saks & Koehler, 2005). One of the most highly publicised cases of forensic misidentification is that of Brandon Mayfield, an American lawyer who was wrongly accused of orchestrating the 2004 Madrid train bombings on the basis of a fingerprint that the FBI determined to have originated from Mayfield (Thompson & Cole, 2005). Interestingly, Mayfield had a strong alibi. He had no record of ever travelling to Spain, he had not left his state of Oregon in two years, and had not been abroad since 1993 (Gumbel, 2004). In light of these misidentifications, the forensic science community has come under considerable scrutiny (NAS, 2009; PCAST, 2016). Numerous authoritative reports have called for research to establish evidence-based standards for reporting forensic analyses to ensure that examiners’ testimony is scientifically sound, including that examiners “should always state that errors can and do occur” (PCAST, 2016, p. 19). Despite these calls for evidence-based testimony, there is uncertainty around what this new model should look like and more research is needed to determine the best way for forensic experts to testify about their decisions. One potential method that is gaining momentum is the likelihood ratio. A likelihood ratio aims to communicate the probability of observations given the two alternative hypotheses: that the two samples share the same source (H1), or that the two samples originate from different sources (H2). For example, a fingerprint likelihood ratio would communicate the probability of observing the similarities between a print of known origin and the crime scene fingerprint under the hypothesis that the two prints come from the same person (H1) versus the hypothesis that the two prints come from two different people (H2) (Martire, Kemp, Sayle, & Newell, 2014). Many people in the forensic science community have welcomed the use of likelihood ratios, declaring them to be “the most appropriate foundation for assisting the court in establishing the weight that should be assigned...” to evidence in a position statement signed by 31 agencies and stakeholders (De Kinder & Olsson, 2011). However, little research has investigated the use — and jurors’ comprehension and understanding — of likelihood ratios. Experiments by Martire and colleagues (2013; 2014) suggest that jurors struggle to understand and interpret likelihood ratios. For example, a weak (but still inculpatory) likelihood ratio led participants towards a decision of innocence (referred to as the weak evidence effect). Current experiment ------------------ We propose that not only are likelihood ratios difficult for jurors to understand and interpret, they do not take into account the potential for human error, which is problematic. The PCAST report notes that the risk of human error occurring (e.g., through mislabelling samples, contamination, etc) is far more likely than the possibility of two DNA samples from two different sources sharing the same profile (2016, p. 7). As the likelihood ratio only seeks to quantify the strength of the hypothesis that two samples share the same source against the hypothesis that two samples come from different sources, it fails to take into account the potential for human error. Thus, regardless of whether or not a human error has occurred, the likelihood ratio would remain the same and, particularly in the case of DNA, could very strongly support the prosecution’s case. In this experiment, we have created a framework to evaluate the extent to which jurors are swayed by a strong DNA likelihood ratio in the face of varying levels of strength of the suspect’s case. To do so, we have chosen to use alibi as the other factor participants must evaluate. We suspect that the presence of a strong DNA likelihood ratio would render participants insensitive to alibi evidence compared to when only the alibi evidence is presented. Thus, we are particularly interested in seeing whether or not participants are convinced by a strong likelihood ratio (that’s commonly encountered with DNA evidence) when the defendant’s alibi is strong — i.e., where one could be very convinced that the suspect could not have committed the crime (such as the alibi factors in Brandon Mayfield’s case) and thus the strong likelihood ratio must be a result of some kind of error. If we do find that participants are convinced by the DNA likelihood ratio even in the presence of a strong alibi, this would demonstrate that likelihood ratios may overwhelm the quality of other evidence presented at trial. **Participants:** Participants will be recruited online via Prolific and will participate in exchange for payment (£1.00). We will be able to collect data from 600 participants. To arrive at this sample size, we consulted both *GPower* and *PANGEA*. The *GPower* analysis suggested that 501 participants would be sufficient to detect an effect size of 0.2 (Cohen’s f) with a power of 0.95. With *PANGEA*, we estimated that a sample size of 600 would be sufficient to detect an effect size of 0.4 (Cohen’s d) with a power of approximately 0.941. Thus, we decided to proceed with the upper estimate of 600 participants. **Procedure:** Participants will complete the experiment on their own devices. All of our experimental materials will be uploaded to the OSF. The experiment has been programmed in Qualtrics online survey software. Prior to the experiment, participants will be provided with some basic information about the experiment, how we plan to use the data, and contact details of the researchers. Once participants click ‘OKAY’, they have consented to participate in the experiment. Participants will then begin the experiment, which is estimated to take 5-10 minutes. They will read a short case vignette about a murder that has occurred. Participants will be told that a DNA sample was retrieved, and that police have interviewed various suspects including a man with the initials B.M. Participants will then be randomly assigned to 1 of the 12 groups (described below) as per a 2 (Likelihood Ratio: absent, present) x 6 (Alibi Strength: 6 scenarios) between-groups design. Participants in the likelihood ratio absent conditions will just be told that the DNA sample is currently being analysed, whereas participants in the likelihood ratio present conditions will be told that a forensic examiner compared the DNA retrieved during the investigation with the suspect’s DNA and provided a likelihood ratio. All participants will then be randomly allocated to read one of six alibis. ---------- **Here is an example of the DNA likelihood ratio present condition with the strongest alibi:** In the present case, investigators allege that the victim was getting ready to leave home at 8:00pm for an 8:40pm movie with her friends. Shortly before she was due to leave, the victim heard a noise coming from the back door near the kitchen. The victim went to investigate. When she reached the back door, the offender grabbed the victim from behind, covered her mouth and dragged her back towards the living room. The offender pulled out a knife, holding it to the victim’s throat demanding that she hand over all her money and jewellery. The victim put up a fight, trying to free herself from the offender’s grasp. The victim and offender struggled over the knife, but the offender kicked the victim causing her head to hit the floor. The offender stabbed the victim several times in the neck and chest before fleeing the scene. The victim died as a result of these stab wounds. Investigators were able to retrieve a DNA sample from underneath the victim’s fingernails.  Police interviewed various suspects including a man with the initials B.M. A forensic DNA examiner compared the DNA sample retrieved from underneath the victim’s fingernails with a DNA sample obtained from B.M. When assessing the significance of any similarity or differences between two DNA samples, the likelihood of obtaining that similarity or difference is considered against two alternative propositions: (Hypothesis 1) the two DNA samples originated from the same person; (Hypothesis 2) the two DNA samples did not originate from the same person. In the examiner's opinion, the correspondence between the DNA sample retrieved from underneath the victim’s fingernails and the DNA sample obtained from B.M. is 5,500 times more likely if the two DNA samples originated from the same person (Hypothesis 1) than if the two DNA samples originated from different people (Hypothesis 2). When B.M. was interviewed by police, B.M. claimed that he was overseas for one week, including the evening in question, for a friend’s destination wedding in which he was a groomsman. Border security records confirmed that the suspect was out of the country during the evening in question. The newlyweds were also contacted; they provided time-stamped images of B.M. in the bridal party. ---------- After reading the scenario, participants are asked to make a rating regarding the likelihood that the evidence belongs to the suspect on a scale from 0 to 100, a rating regarding the likelihood that the suspect committed the crime on a scale from 0 to 100, and a rating regarding the likelihood that an error could have occurred during DNA analysis on a scale from 0 to 100. After completing the dependent variables, participants are asked a series of manipulation check questions. Firstly, they are asked to identify the likelihood ratio with the options: a) the above information was not presented during the case (correct if in Likelihood Ratio absent condition), b) 32, c) 5,500 (correct if in Likelihood Ratio present condition), or d) 90,372. Secondly, they are asked to select which statement best describes B.M.’s alibi and are provided with a one line summary of each of the 6 alibis. Lastly, participants are asked to rate the extent to which they agree with the statement that “B.M.’s alibi is believable” on a scale from 1 (strongly disagree) to 7 (strongly agree). Participants will then provide their demographic information (age, gender, country of residence, highest level of education, and whether they’ve ever served as a juror in a criminal trial). The Qualtrics survey software will generate a single comma-separated file at the end of the experiment containing data from all participants, which we will upload to the OSF. **Materials:** Four of the alibis in this experiment were adapted from Olson and Wells (2004). The other two alibis were created by the primary researcher. All six alibis were selected on the basis of a pilot study (consisting of a total of seven alibis) which assessed perceived believability and likelihood of a suspect committing the crime if they had each particular alibi. **Data collection to date:** No experimental data have been collected to date. **Timeline:** We aim to start collecting data as soon as the experiment is pre-registered on the OSF. We hope to wrap up data collection by December 2018. **Exclusion Criteria:** Prior to reading the scenario, we have included an attention check question to ensure that participants have read the instructions carefully. Participants are asked to respond to a question “In your opinion, what is the most common type of crime?” by selecting the ‘other’ option and typing the number ‘4’ into the field. Participants who do not follow these instructions will not be able to participate in the study and will be directed back to the Prolific recruitment platform and another participant will be recruited to take their place. We have also included an attention check question in the demographics section at the end of the study. Participants will be asked “In the case you just read, what type of crime was being investigated?”. The correct answer is ‘murder’. Participants who respond incorrectly will be excluded from the analyses. Partial survey responses (i.e., participants who do not get redirected back to the Prolific platform after competing the study) will also be excluded from analyses and another participant will be recruited to take their place. With regards to the manipulation check questions, we will report the percentage of participants who did and did not answer correctly and examine whether there is any systematic variation between conditions. We will conduct the analyses with and without excluding the participants who fail the manipulation checks. If there is no difference, the participants will be retained. **Missing data:** Participants who have missing data on any of the three dependent variables will be excluded from analyses. **Hypotheses:** 1. Main effect of Likelihood Ratio: we expect that participants will be more likely to rate the source of the evidence as belonging to the suspect and more likely to rate that the suspect committed the crime when a likelihood ratio is presented compared to when it is not presented. However, participants will be less likely to conclude that an error could occur during the DNA analysis when a likelihood ratio is presented compared to when it is not presented. 2. Main effect of Alibi Strength: we expect that participants will be more likely to rate the source of the evidence as belonging to the suspect and and more likely to rate that the suspect committed the crime as the strength of the suspect’s alibi increases. We also expect that participants will be more likely to conclude that an error could occur during the DNA analysis as the strength of the suspect’s alibi increases. 3. Likelihood Ratio x Alibi Strength interaction: we expect Alibi Strength to have a greater effect on the dependent variables when a likelihood ratio is not presented (compared to when it is presented). **(11-01-18) Preregistration Amendment to Hypothesis 2:** This is an amendment to preregistration: https://osf.io/pwyx9/ In reading over the preregistration, the lead researcher noticed a mistake in the hypothesised direction of the main effect of Alibi Strength. Thus, Hypothesis 2 should read as follows (changes in bold): Main effect of Alibi Strength: we expect that participants will be **less** likely to rate the source of the evidence as belonging to the suspect and and **less** likely to rate that the suspect committed the crime as the strength of the suspect’s alibi increases. **However, we** expect that participants will be more likely to conclude that an error could occur during the DNA analysis as the strength of the suspect’s alibi increases. **Planned Analyses:** We will conduct a 2x6 between-groups ANOVA on each of our three continuous dependent variables (likelihood that the suspect is the source of the DNA, likelihood of suspect of committing the crime, and likelihood that an error could occur during DNA analysis). For follow-up analyses (i.e., to follow-up a significant interaction), we will analyse the simple effects where we look at the differences in alibi at each level of DNA evidence present (absent and present). **Ethics:** This study was granted Institutional Human Research Ethics Approval from The University of Queensland on the 5th of June 2018. The information sheet to be given to participants at the start of the study can be seen in the project files folder. Following the completion of the study, participants will be given a debrief sheet (see project files). Participants will all be given the opportunity to contact the researcher for further information. Participation is entirely voluntary, and participants are free to withdraw at any time. References --------------- - De Kinder, J., & Olsson, T. (2011). Expressing evaluative opinions: a position statement. Science & Justice, 51(1), 1-2. doi:10.1016/j.scijus.2011.01.002 - Gumbel, A. (2004, May 13). Madrid suspect ‘never went to Spain’. Retrieved from: https://www.independent.co.uk/news/world/americas/madrid-suspect-never-went-to-spain-563188.html - Martire, K. A., Kemp, R. I., Sayle, M. A., & Newell, B. R. (2014). On the interpretation of likelihood ratios in forensic science evidence: Presentation formats and the weak evidence effect. Forensic Science International, 240(1), 61-68. doi: 10.1016/j.forsciint.2014.04.005 - Martire, K. A., Kemp, R. I., Watkins, I., Sayle, M. A., Newell, B. R. (2013). The expression and interpretation of uncertain forensic science evidence: Verbal equivalence, evidence strength, and the weak evidence effect. Law and Human Behavior, 37(3), 197-207. doi: 10.1037/ lhb0000027 - National Academy of Sciences [NAS]. (2009). Strengthening forensic science in the United States: A path forward. Washington, DC: National Academies Press. Retrieved from: https://www.ncjrs.gov/pdffiles1/nij/grants/228091.pdf - Olson, E. A., & Wells, G. L. (2004). What makes a good alibi? A proposed taxonomy. Law & Human Behavior, 28(2), 157-176. doi: 10.1023/B:LAHU.0000022320.47112.d3 - President’s Council of Advisors on Science and Technology [PCAST]. (2016). Forensic science in criminal courts: Ensuring scientific validity of feature-comparison methods. Retrieved from: https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensic_science_report_final.pdf - Saks, M., & Koehler, J. (2005). The coming paradigm shift in forensic identification science. Science, 309, 892-895. doi: 10.1126/science.1111565 - Thompson, W. C., & Cole, S. A. (2005). Lessons from the Brandon Mayfield case. The Champion, 29(3), 42-44.