# Expertise in spotting differences between fingerprints *The contributors to this project are listed alphabetically until the resulting manuscript has been submitted for publication.* ## Rationale Visual skills like identifying, classifying, and matching objects enable humans to navigate a world in which they continually encounter new stimuli. With practice, however, these general skills allow people to acquire more specific expertise in a variety of domains concerning complex visual structure like air-traffic control, radiology and fingerprint identification. Identifying the locations of features that are relevant to the task and determining the most appropriate action to take with this information are two of the most important skills one must have to attain genuine expertise in domains like these (Roads, Mozer & Busey, 2016). The question that remains is whether domain experts are better than novices at locating information within stimuli they have vast amounts of experience with. We plan to investigate this question and identify possible mechanisms and constraints for such an advantage by studying an under-researched field: fingerprint examination. We indeed already have evidence that experts in several domains have superior non-analytic abilities relative to novices. Take the field of diagnostic medicine; radiologists can detect abnormalities better than chance given only a very short amount of time (Drew, Evans, Võ, Jacobson, & Wolfe, 2013; Evans et al., 2013). Mammographers are even able to detect whether someone's left breast has a tumour after a quick glance at the scan of the same individual's right breast (Evans et al., 2016). Similarly, fingerprint examiners can tell whether two fingerprints match after only a fraction of a second (Thompson & Tangen, 2014) and can detect prints left by different fingers of the same person despite no overlapping features (Searston & Tangen, 2017). Radiologists, however, also possess better analytic skills than novices. Those with experience tend to detect lesions earlier in their search and cover less area than those without (Krupinski, 1996). Expert radiologists are also more accurate, confident and quicker to diagnose skeletal fractures, especially when the anomalies are more ambiguous (Wood et al., 2013). We know far less about the analytic skills of fingerprint examiners, but they too may outperform novices if given a visual search task relevant to their domain of expertise. Human examiners are relied upon to discern whether two fingerprints belong to the same finger or from two different fingers. This process not only involves locating confirmatory evidence in the form of points of correspondence, but also necessitates spotting disconfirmatory evidence or points of difference (Dror et al., 2011). Both of these skills play a critical role in the judgements that examiners make. Expert fingerprint examiners should be well-equipped to efficiently spot differences between two prints. In fact, working for an alternative solutions and finding disconfirmatory evidence can help people solve problems as it prompts them to avoid being distracted by surface elements in the case at hand - a method known as differential processing (Catrambone & Holyoak, 1989; Gentner, Loewenstein, & Thompson, 2003). Differential processing has been shown to minimise medical errors arising from premature closure (i.e. deciding on the first diagnosis that comes to mind without considering all the evidence), and has been documented as an effective learning strategy in medical training (Ark, Brooks, & Eva, 2007). Experts, because of their training, should sensitive to the differences between stimuli with which they are familiar. As visual expertise develops, viewers pay more attention to task-relevant information and filter out noise more readily. Rehder and Hoffman (2005) demonstrated that participants often fixate on all dimensions of a stimulus early in training, but their eye movements then become restricted to more relevant dimensions later on. In medicine, there are clear diagnostic features that experts look for while attending very little to the surrounding noise. Radiologists who restrict their eye-movements to specific diagnostic areas make more accurate decisions (Drew et al., 2013). And those with greater experience in their domain show increased sensitivity to the critical areas of an image (Krupinski, Graham, & Weinstein, 2013). Experts it seems should be specifically well-equipped to spot differences if the differences are task-relevant. The capabilities of many experts often rely on their implicit long-term memory and sensitivity to the structural regularities of the class of stimuli with which they are familiar. Disrupt the structure and their advantage disappears. For instance, relative to novices, chess experts can remember the positions of far more pieces on a chess board if the pieces are presented in a game-like configuration (Chase & Simon, 1973). However, their memory advantage fades away when the positions of the pieces are randomised. The same should occur for fingerprint examiners because they too rely on the structure of the stimuli they have been exposed to for hours of practice. If the features and configural information of a print are obliterated, but the low-level properties remain intact, we would expect the visual search performance of experts to return to that of novices. So, are experts able to rapidly spot differences located in diagnostic areas of two prints and does the structure of these fingerprints facilitate this skill? ## The Current Study In this experiment, we will test to see whether expert fingerprint examiners have better visual search abilities compared to novices with a class of stimuli they are familiar with. We will also test whether their superior performance disappears when the structural regularities of the stimuli are removed, or if asked to spot non-diagnostic information. We have conceptualised a fingerprint-like visual search task much like the spot-the-difference tasks you might see in a newspaper to assess how well people can find points of disagreement between prints. Participants will be presented with two fingerprint images side-by-side that are identical but for one fragment that has been replaced with alternative ridge information. They will be asked to spot this difference. Here is a demonstration of the task: @[youtube](https://youtu.be/0yK2D9dPkAI) ## Participants ### Experts Data will be collected from as many fingerprint examiners as possible, subject to their availability. Our goal is to reach 30 expert participants. We define an ‘expert’ as one who is appropriately qualified in their jurisdiction, and is court-practicing. We will also collect data from professional fingerprint examiners who are not yet fully qualified. These examiners are often referred to as ‘trainees’. These data will not be analysed in this experiment, but may be used in a future project. ### Novices Data will be collected from an equal number of novices as experts. So, for example, if we collect data from 34 experts, then we will collect data from 34 novices. Novice participants who have no formal experience in fingerprint examination will be recruited from The University of Adelaide, The University of Queensland, and/or Murdoch University communities, and the general Australian public. Novice participants may be offered a cash or gift card incentive for their participation, or participate for course credit, with the chance to win an additional amount on reaching a threshold level of performance, and on completing a larger series of fingerprint tasks, including this one. Our aim is to recruit novices who are motivated to perform well. ### Sensitivity Analysis Based on previous expert-novice studies, we anticipate a large difference in performance between the groups. (d > .80). Twelve observations from 30 participants for each condition provides sufficient sensitivity (power > 0.9) to detect an effect size of d = 0.45 for all planned analyses. ### Exclusion criteria - Participants who respond faster than 1000ms on more than 20% of trials will be excluded. - If a participant fails to complete the experiment because of fatigue, illness, or excessive response delays (i.e., longer than the experimental session allows), then their responses will not be recorded. - If a participant is excluded based on these criteria, then another participant will be recruited to take their place. Participants are also required to bring and use any necessary aids, including glasses, contact lenses, and hearing aids. ### Data Collection No data have been collected yet. We will be visiting police departments during February 2019 and plan to have novice data by the middle of the year. ## Design This experiment employs 2 (Expertise: expert, novice; between subjects) x 2 (Structure: intact, scrambled; within subjects) x 2 (Diagnosticity: diagnostic, non-diagnostic; within subjects) mixed design 'yoked' to expertise. A total of 64 unique sequences have been pre-generated where each sequence contains 48 trials. Half of the trials will require participants to spot differences located in diagnostic areas and half will require them to locate differences in non-diagnostic areas. Half of the trials will also require participants to spot differences between intact, unaltered fingerprint images and half will require them to spot differences between images that have been scrambled. The order of these trials has been randomised and we kept the appearance and locations of the original fragments equivalent across the intact and scrambled conditions to control for possible confounds. Our DVs of concern are proportion correct and Rate Correct Score (RCS), which is an integrated speed-accuracy measure (Woltz & Was, 2006) that expresses the proportion of correct responses produced per second. The 64 pre-generated participant sequences can be downloaded in the [Sequences][2] component of this project. ## Materials Participants in this task will be presented with two fingerprint images side-by-side that are identical but for one fragment on the image to the right that has been replaced with alternative ridge information. ### Intact images The 72 intact array stimuli (750 x 750 pixels) in this task were originally obtained from the FIBR database (see Tear, Thompson, & Tangen, 2010). We chose and cropped 50 latents and 50 tenprints such that the entire image was filled with ridge detail. We then removed the 28 prints we believed to be the noisiest to bring us to a total of 72 images (46 tenprints and 26 latents). ### Scrambled images We generated eight different scrambled versions of the the 72 intact fingerprints. The scrambling process we used removed the fingerprint-like structure of the images but kept the low-level visual properties intact as well as the the location and appearance of the fragments located within. To accomplish this, we scrambled the images using Photoshop’s Content Aware Tool.* We deselected the location of the relevant fragment and then selected one quadrant of the print, deleted it and replaced it with alternative information generated by Content Aware. We continued this process for the other three quadrants moving around clockwise (each of the four quadrants was used as the starting quadrant in equal amounts) until the entire image was scrambled apart from the fragment that we deselected. We applied this scrambling procedure to each of the intact images in the context of eight different fragments (four diagnostic and four non-diagnostic). In total, we created a pool of 576 scrambled array images. ### Alternative Images We generated an alternative image for each of the scrambled and intact images chosen for our event sequences. We used data from a previous experiment to generate hundreds of fragments (see here for the pre-registration: https://osf.io/rxe25). In this earlier experiment, we presented the same 100 prints mentioned above to 30 novices and 30 experts asking them to draw a point in the middle of the area they consider most informative (diagnostic) and least informative (non-diagnostic) on each of the prints. We used all of these judgements as centre-points for generating circular fragments of various sizes. We then gave another 48 novices the spot-the-difference task in a subsequent experiment where we gradually increased the size of the fragments over time to determine at what point novices could perform at 50% (https://osf.io/3g4e7). We found the 'optimal' fragment size to be 3.71% the size of the larger image. To generate the alternative images, we simply selected these fragments within the images and used Content Aware to replace it with alternative information. **Content Aware Fill simplifies the process of removing objects from an image and runs on an algorithm called PatchMatch (Barnes, Shectman, Finkelstein & Goldman, 2009). PatchMatch uses a randomised algorithm to approximate nearest neighbour matches between image patches (small, square regions). It quickly finds correspondences between patches of an image. Initially, the nearest-neighbour is filled with either random offsets or some prior information, Next, an iterative update process is applied to the nearest-neighbour field, in which good patch offsets are propagated to adjacent pixels, followed by random search in the neighbourhood of the best offset found so far.* ## Procedure Participants first read an information sheet about the project, and watch an instructional video about the task with examples. They are then presented with 48 randomly sequenced spot-the-difference trials where two images are presented side-by-side, which are identical but for one fragment on the right image that has been altered. Participants are asked to click where they think the difference is on the image to the right. Upon clicking within the image, immediate feedback is provided. If participants make an incorrect click (that is, they do not click on the change) they will see a red cross, a coloured highlight of the correct location, and hear a dull tone. However, if correct, participants will see a green tick, a highlight of the correct location, and hear a light tone. They will then move onto the next trial. There is a 500 millisecond window for this feedback and then 500 milliseconds of blank screen before the next set of images appears. All prints remain on the screen until the participant indicates their response. A text prompt appears during the inter-trial interval if participants take longer than 15 seconds to respond, with the message: “try to respond in less than 15 seconds.” A progress bar in the top right hand corner of the screen indicates to Participants how many trials they have completed and how many remain. ## Software The video instructions are presented to participants on a 13-inch MacBook Pro or MacBook Air laptop screen, with over-ear headphones. The software used to generate the trial sequences, present stimuli to participants, and record their responses, was developed in LiveCode (version 9.0.2; the open source ‘community edition’). The LiveCode source files and experiment code to run the experiment locally can be downloaded in the [Software][3] component of this project. The data analytic scripts and plots for this project are produced in RStudio with R Markdown and a list of the packages needed to reproduce our plots and analyses are listed in the data visualisation and analysis html file in the [Analysis][4] component of this project. ## Hypotheses Prior work has demonstrated that expert fingerprint examiners can detect prints from different fingers of the same person (Searston & Tangen, 2017) and can tell whether two fingerprints match or not after only a quick glance (Thompson & Tangen, 2014) Given these previous demonstrations of expert-novice differences and the nature of fingerprint comparison, we hypothesise that examiners will also be able to rapidly spot differences between prints if they are located in diagnostic areas of the print. Novices, however, have no prior experience of fingerprints to draw on, and therefore no appreciation for any of the structural regularities that facilitate the spotting of differences. We also have evidence from fields like diagnostic medicine that experts tend to ignore task-irrelevant information after significant training with a particular class of stimuli (see, for example Drew, Võ, & Wolfe, 2013) and so their expertise may be constrained to only diagnostic information. Lastly, we have evidence that the advantage of experts can disappear if the structure of the class of stimuli they are familiar with is obliterated. For example, chess experts are far better than novices at remembering the location of chess pieces if presented in a game-like formation, but not so if presented in random locations (Chase & Simon, 1973). Given this prior research we predict the following: 1. When comparing novices to experts, we expect experts to outperform novices only for diagnostic fragments within intact images, resulting in a large effect size (*d* > .5) in the difference between their proportion correct for this comparison. 2. When comparing novices to experts, we expect experts to outperform novices only for diagnostic fragments within intact images, resulting in a large effect size (*d* > .5) in the difference between their rate correct score for this comparison. Here is a ideal graphical representation of what we expect to find: ![enter image description here][1] ## Planned Analyses To test whether and to what extent expert fingerprint examiners outperform novices at the task, for diagnostic versus nondiagnostic fragments, with intact versus scrambled prints, a between-groups ANOVA (or nonparametric equivalent if distributional assumptions are not met) will be conducted comparing them on proportion correct and RCS. ### Simulated Data and Analyses To pilot our experiment and data analysis script, we ran 12 'sim' experts and novices through the experiment. These simulated participants are programmed to respond pseudorandomly on each trial in the experiment with the assumption that they would be correct on half of the trials. We used the .txt files produced by these simulations to generate and test a LiveCode data extraction tool, and an R Markdown analysis script for plotting and analysing the data in this project. They provide a useful model for how participants would perform if their proportion correct were 50%. They are also helpful for debugging our experiment code and analysis script. The simulated data, LiveCode data extraction tool, R Markdown source files, resulting plots, and analysis script can be downloaded in the [Analysis][5] component of this project. A few plots from these simulated data in the current project are presented below, where each participant either randomly locates the correct spot on each trial or not. Note that 50% is far beyond the 3.17% chance level of performance on this task, but randomly selecting the correct location on each trial was far easier to simulate than a random mouse click for illustration purposes (colour filled = intact, white filled = scrambled). ![Simulated Results][6] ## Ethics We have ethics clearance from human research ethics committees at The University of Adelaide: (Project codes: 13-39-18 and 18-93; Real World Perceptual Learning) and The University of Queensland (Project code: 2014001677; Expertise & Non-Analytic Cognition, and 16-PSYCH-PHD-25-AH; Turning Novices into Experts). ## References Ark, T. K., Brooks, L. R., & Eva, K. W. (2007). The benefits of flexibility: the pedagogical value of instructions to adopt multifaceted diagnostic reasoning strategies. Medical education, 41(3), 281-287. Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009). PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (ToG), 28(3), 24. doi:10.1145/1531326.1531330 Catrambone, R., & Holyoak, K. J. (1989). Overcoming contextual limitations on problem-solving transfer. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(6), 1147. Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive psychology, 4(1), 55-81. doi:10.1016/0010-0285(73)90004-2 Drew, T., Evans, K., Võ, M. L.-H., Jacobson, F. L., & Wolfe, J. M. (2013). Informatics in Radiology: What Can You See in a Single Glance and How Might This Guide Visual Search in Medical Images? RadioGraphics, 33(1), 263–274. doi: 10.1148/rg.331125023 Drew, T., Vo, M. L. H., Olwal, A., Jacobson, F., Seltzer, S. E., & Wolfe, J. M. (2013). Scanners and drillers: Characterizing expert visual search through volumetric images. Journal of vision, 13(10), 3-3. doi10.1167/13.10.3 Drew, T., Võ, M. L. H., & Wolfe, J. M. (2013). The invisible gorilla strikes again: Sustained inattentional blindness in expert observers. Psychological science, 24(9), 1848-1853. doi: 10.1177/0956797613479386 Dror, I. E., Champod, C., Langenburg, G., Charlton, D., Hunt, H., & Rosenthal, R. (2011). Cognitive issues in fingerprint analysis: Inter-and intra-expert consistency and the effect of a ‘target’comparison. Forensic Science International, 208(1-3), 10-17. doi: 10.1016/j.forsciint.2010.10.013 Evans, K. K., Georgian-Smith, D., Tambouret, R., Birdwell, R. L., & Wolfe, J. M. (2013). The gist of the abnormal: Above-chance medical decision making in the blink of an eye. Psychonomic Bulletin and Review, 20(6), 1170–1175. doi:10.3758/s13423-013-0459-3 Evans, K. K., Haygood, T. M., Cooper, J., Culpan, A. M., & Wolfe, J. M. (2016). A half-second glimpse often lets radiologists identify breast cancer cases even when viewing the mammogram of the opposite breast. Proceedings of the National Academy of Sciences, 113(37), 10292-10297. doi:10.1073/pnas.1606187113 Loewenstein, J., Thompson, L., & Gentner, D. (2003). Analogical learning in negotiation teams: Comparing cases promotes learning and transfer. Academy of Management Learning & Education, 2(2), 119-127. Krupinski, E. A., Graham, A. R., & Weinstein, R. S. (2013). Characterizing the development of visual search expertise in pathology residents viewing whole slide images. Human pathology, 44(3), 357-364. doi: 10.1016/j.humpath.2012.05.024 Krupinski, E. A. (1996). Visual scanning patterns of radiologists searching mammograms. Academic radiology, 3(2), 137-144. doi: 10.1016/S1076-6332(05)80381-2 Rehder, B., & Hoffman, A. B. (2005). Eyetracking and selective attention in category learning. Cognitive psychology, 51(1), 1-41. doi:10.1016/j.cogpsych.2004.11.001 Roads B, Mozer MC, Busey TA (2016) Using Highlighting to Train Attentional Expertise. PLoS ONE 11(1): e0146266. doi: 10.1371/journal.pone.0146266 Searston, R. A., & Tangen, J. M. (2017). The style of a stranger: Identification expertise generalizes to coarser level categories. Psychonomic bulletin & review, 24(4), 1324-1329. https://doi.org/10.3758/s13423-016-1211-6. Tear, M. J., Thompson, M. B., & Tangen, J. M. (2010, September). The importance of ground truth: An open-source biometric repository. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 54, No. 19, pp. 1464-1467). Sage CA: Los Angeles, CA: SAGE Publications. doi: 10.1177/154193121005401923 Thompson, M. B., Tangen, J. M., & McCarthy, D. J. (2014). Human matching performance of genuine crime scene latent fingerprints. Law and human behavior, 38(1), 84. doi: 10.1037/lhb0000051 Woltz, D. J. and Was, C. A. (2006). Availability of related long-term memory during and after attention focus in working memory. Memory & Cognition 34(3): 668–684. doi: 10.3758/BF03193587 Wood, G., Knapp, K. M., Rock, B., Cousens, C., Roobottom, C., & Wilson, M. R. (2013). Visual expertise in detecting and diagnosing skeletal fractures. Skeletal Radiology, 42(2), 165–172. doi: 10.1007/s00256-012-1503-5 [1]: https://files.osf.io/v1/resources/aphxg/providers/osfstorage/5c57f632e16f55001a884b36?mode=render [2]: https://osf.io/7jybd [3]: https://osf.io/8xcv6 [4]: https://osf.io/my28c [5]: https://osf.io/my28c [6]: https://files.osf.io/v1/resources/aphxg/providers/osfstorage/5c54e8ade16f5500188724e7?mode=render