# Expertise in locating information within fingerprints # *The contributors to this project are listed alphabetically until the resulting manuscript has been submitted for publication.* ## Rationale Visual skills like identifying, classifying, and matching objects enable humans to navigate a world in which they continually encounter new stimuli. With practice, however, these general skills allow people to acquire more specific expertise in a variety of domains concerning complex visual structure like air-traffic control, radiology and fingerprint identification. Identifying the locations of features that are relevant to the task and determining the most appropriate action to take with this information are two of the most important skills one must have to attain genuine expertise in domains like these (Roads, Mozer & Busey, 2016). The question that remains is whether domain experts are better than novices at locating information within stimuli they have had vast amounts of experience with. We plan to investigate this question and identify possible mechanisms and constraints for such an advantage by studying an under-researched field: fingerprint identification. We indeed already have evidence that experts in several domains have superior non-analytic abilities relative to novices. Take the field of diagnostic medicine; radiologists can detect abnormalities better than chance given only a very short amount of time (Drew, Evans, Võ, Jacobson, & Wolfe, 2013; Evans et al., 2013). Mammographers are even able to detect whether someone's left breast has a tumour after a quick glance at the scan of the same individual's right breast (Evans et al., 2016). Similarly, fingerprint examiners can tell whether two fingerprints match after only a fraction of a second (Thompson & Tangen, 2014) and can detect prints left by different fingers of the same person despite no overlapping features (Searston & Tangen, 2017). Radiologists, however, also possess better analytic skills than novices. Those with experience tend to detect lesions earlier in their search and cover less area than those without (Krupinski, 1996). Expert radiologists are also more accurate, confident and quicker to diagnose skeletal fractures, especially when the anomalies are more ambiguous (Wood et al., 2013). We know far less about the analytic skills of fingerprint examiners, but they too may outperform novices if given a visual search task relevant to their domain of expertise. Human examiners are relied upon to discern whether two fingerprints belong to the same finger or two different fingers. This process involves locating features on two different prints to see if these features match up. Experience with making such judgements means that experts should be well-equipped to efficiently search for particular features within fingerprints. As visual expertise develops, viewers pay more attention to task-relevant information and filter out noise more readily. Rehder and Hoffman (2005) demonstrated that participants often fixate on all dimensions of a stimulus early in training, but their eye movements then become restricted to more relevant dimensions later on. In medicine, there are clear diagnostic features that experts look for while attending very little to surrounding noise. Radiologists who restrict their eye-movements to specific diagnostic areas make more accurate decisions (Drew et al., 2013). And those with greater experience in their domain show increased sensitivity to the critical areas of an image (Krupinski, Graham, & Weinstein, 2013). Experts its seems are specifically better at searching for task-relevant information as opposed to just any information. An experiment on the inattentional blindness of medical experts highlights this point further. Participants in this study were asked to spot lung nodules in a scan, but on the very last frame, experimenters inserted an image of a gorilla 48 times the size of the nodule itself (Drew, Võ, & Wolfe, 2013). In 83% of cases the gorilla went undetected. medical experts appear to focus intensely on diagnostic information and ignore much of the non-diagnostic information. Fingerprint examiners, just like these diagnositicians, should be able to spot diagnostic information far faster and more accurately than novices, but may even be hindered if asked to spot information irrelevant to them in their day-to-day job. The capabilities of many experts often rely on their implicit long-term memory and sensitivity to the structural regularities of the class of stimuli with which they are familiar. Disrupt the structure and their advantage disappears. For instance, relative to novices, chess experts can remember the positions of far more pieces on a chess board if the pieces are presented in a game-like configuration (Chase & Simon, 1973). However, their memory advantage fades away when the positions of the pieces are randomised. The same should occur for fingerprint examiners because they too rely on the structure of the stimuli they have been exposed to for hours of practice. If the features and configural information of a print are obliterated, but the low-level properties remain intact, we would expect the visual search performance of experts to return to that of novices. Are experts able to rapidly locate diagnostic information within a print and does the structure of these fingerprints facilitate this skill? ## The Current Study In this experiment, we will test to see whether expert fingerprint examiners have better visual search abilities compared to novices with a class of stimuli they are familiar with. We will also test whether their superior performance disappears when the structural regularities of the stimuli are removed, or if asked to spot information that is non-diagnostic. We have conceptualised a fingerprint-like visual search task much like a Where's Wally puzzle that assesses how well participants can find points of correspondence (the find-the-fragment task). Participants will be asked to find a small fragment of print information (presented on the left) within a larger fingerprint image (presented on the right) as quickly as they can. Here is a demonstration of the task: @[youtube](https://youtu.be/3dBDBKxZO7c) ## Participants ### Experts Data will be collected from as many fingerprint examiners as possible, subject to their availability. Our goal is to reach 30 expert participants. We define an ‘expert’ as one who is appropriately qualified in their jurisdiction, and is court-practicing. We will also collect data from professional fingerprint examiners who are not yet fully qualified. These examiners are often referred to as ‘trainees’. These data will not be analysed in this experiment, but may be used in a future project. ### Novices Data will be collected from an equal number of novices as experts. So, for example, if we collect data from 34 experts, then we will collect data from 34 novices. Novice participants who have no formal experience in fingerprint examination will be recruited from The University of Adelaide, The University of Queensland, and/or Murdoch University communities, and the general Australian public. Novice participants may be offered a cash or gift card incentive for their participation, or participate for course credit, with the chance to win an additional amount on reaching a threshold level of performance, and on completing a larger series of fingerprint tasks, including this one. Our aim is to recruit novices who are motivated to perform well. ### Sensitivity Analysis Based on previous expert-novice studies, we anticipate a large difference in performance between the groups. Twelve observations from 30 participants for each condition provides sufficient sensitivity (power > 0.9) to detect an effect size of d = 0.45 for all planned analyses. ### Exclusion criteria - Participants who respond faster than 1000ms on more than 20% of trials will be excluded. - If a participant fails to complete the experiment because of fatigue, illness, or excessive response delays (i.e., longer than the experimental session allows), then their responses will not be recorded. - If a participant is excluded based on these criteria, then another participant will be recruited to take their place. Participants are also required to bring and use any necessary aids, including glasses, contact lenses, and hearing aids. ### Data Collection No data have been collected yet. We will be visiting police departments during February 2019 and plan to have novice data by the middle of the year. ## Design This experiment employs a 2 (Expertise: expert, novice; between subjects) x 2 (Structure: intact, scrambled; within subjects) x 2 (Diagnosticity: diagnostic, non-diagnostic; within subjects) mixed design 'yoked' to expertise. A total of 64 unique sequences have been pre-generated where each sequence contains 48 trials. Half of the trials will require participants to locate diagnostic fragments and half will require them to locate non-diagnostic fragments. Half of the trials will also require participants to find fragments within an intact, unaltered fingerprint images and half will require them to locate fragments within images that have been scrambled. The order of these trials have been randomised and we kept the appearance and locations of the fragments equivalent across the intact and scrambled conditions to control for possible confounds. Our DVs of concern are proportion correct and Rate Correct Score (RCS), which is an integrated speed-accuracy measure (Woltz & Was, 2006) that expresses the proportion of correct responses produced per second. The 64 pre-generated participant sequences can be downloaded in the [Sequences][2] component of this project. ## Materials Participants in this task will be presented with a small fragment of fingerprint ridge detail on the left of the screen and a larger fingerprint image (the array) to the right. ### Intact Array images The 72 intact array stimuli (750 x 750 pixels) in this task were originally obtained from the FIBR database (see Tear, Thompson, & Tangen, 2010). We chose and cropped 50 latents and 50 tenprints such that the entire image was filled with ridge detail. We then removed the 28 prints we believed to be the noisiest to make a total of 72 intact images (46 tenprints and 26 latents). ### Fragment images We generated fragments using data from a previous experiment (see here for the pre-registration: https://osf.io/rxe25). We presented the same 100 prints mentioned above to 30 novices and 30 experts asking them to draw a point in the middle of the area they consider most informative (diagnostic) and least informative (non-diagnostic) on each of the prints. We used all of these judgements as centre-points to generate thousands of circular fragments with various sizes. We then gave 48 novices the find-the-fragment task in a subsequent experiment and gradually increased the size of the fragments over time to determine at what point novices could perform at 50% (https://osf.io/3g4e7). We found fragments 1.45% the size of the array to be the 'optimal' size. ### Scrambled Array Images We generated eight different scrambled versions of the the 72 intact fingerprints. The scrambling process we used removed the fingerprint-like structure of the images but kept the low-level visual properties intact as well as the the location and appearance of the fragments located within. To accomplish this, we scrambled the images using Photoshop’s Content Aware Tool.* We deselected the location of the relevant fragment and then selected one quadrant of the print, deleted it and replaced it with alternative information generated by Content Aware. We continued this process for the other three quadrants moving around clockwise (we started the process equally in each of the four quadrants) until the entire image was scrambled apart from the fragment that was deselected. We applied this scrambling procedure to each of the intact images in the context of eight different fragments (four diagnostic and four non-diagnostic). In total, we created a pool of 576 scrambled array images. **Content Aware Fill simplifies the process of removing objects from an image and runs on an algorithm called PatchMatch (Barnes, Shectman, Finkelstein & Goldman, 2009). PatchMatch uses a randomised algorithm to approximate nearest neighbour matches between image patches (small, square regions). It quickly finds correspondences between patches of an image. Initially, the nearest-neighbour is filled with either random offsets or some prior information, Next, an iterative update process is applied to the nearest-neighbour field, in which good patch offsets are propagated to adjacent pixels, followed by random search in the neighbourhood of the best offset found so far.* ## Procedure Participants first read an information sheet about the project, and watch an instructional video about the task with examples. They are then presented with 48 randomly sequenced find-the-fragment trials where a small fragment of fingerprint detail is presented on the left of the screen and the larger fingerprint image (the array) is presented to the right. Participants are asked to click where they think the smaller fragment is within the larger array. Upon clicking within the array, immediate feedback is provided. If participants make an incorrect click (that is, they do not click on where the fragment is located) they will see a red cross and a coloured highlight of the correct location, and hear a dull tone. However, if correct participants will see a green tick, a highlight of the correct location, and hear a light tone. They will then move onto the next trial. There is a 500 millisecond window for this feedback and then 500 milliseconds of blank screen before the next set of images appears. All prints remain on the screen until the participant indicates their response. A text prompt appears during the inter-trial interval if participants take longer than 15 seconds to respond, with the message: “try to respond in less than 15 seconds.” A progress bar in the top right hand corner of the screen indicates to Participants how many trials they have completed and how many remain. ## Software The video instructions are presented to participants on a 13-inch MacBook Pro or MacBook Air laptop screen, with over-ear headphones. The software used to generate the trial sequences, present stimuli to participants, and record their responses, was developed in LiveCode (version 9.0.2; the open source ‘community edition’). The LiveCode source files and experiment code to run the experiment locally can be downloaded in the [Software][3] component of this project. The data analytic scripts and plots for this project are produced in RStudio with R Markdown and a list of the packages needed to reproduce our plots and analyses are listed in the data visualisation and analysis html file in the [Analysis][4] component of this project. ## Hypotheses and Predictions Prior work has demonstrated that experienced fingerprint examiners can detect prints from different fingers of the same person (Searston & Tangen, 2017) and can tell whether two fingerprints match or not after only a quick glance (Thompson & Tangen, 2014) Given these previous demonstrations of expert-novice differences, we hypothesise that examiners will also be able to spot diagnostic features fairly rapidly given they spend their days comparing important features across prints. Novices, however, have no prior experience of fingerprints to draw on, and therefore no appreciation for any of the structural regularities that facilitate locating features within a print. Nonetheless, we also have evidence from fields like diagnostic medicine that experts tend to ignore task-irrelevant information after significant training with a particular class of stimuli (see, for example Drew, Võ, & Wolfe, 2013) and therefore their advantage may be constrained only to diagnostic information. Lastly, we have evidence that the advantage of experts can disappear if the structure of the class of stimuli they are familiar with is obliterated. For example, chess experts are far better than novices at remembering the location of chess pieces if presented in a game-like formation, but not so if presented in random locations (Chase & Simon, 1974). Given this prior research we predict the following: 1. When comparing novices to experts, we expect experts to outperform novices only for diagnostic fragments within intact images, resulting in a large effect size (*d* > .5) in the difference between their proportion correct for this comparison. 2. When comparing novices to experts, we expect experts to outperform novices only for diagnostic fragments within intact images, resulting in a large effect size (*d* > .5) in the difference between their rate correct scores for this comparison. Here is a ideal graphical representation of what we expect to find: ![enter image description here][1] ## Planned Analyses To test whether and to what extent expert fingerprint examiners outperform novices at the task, for diagnostic versus nondiagnostic fragments, with intact versus scrambled prints, a between-groups ANOVA (or nonparametric equivalent if distributional assumptions are not met) will be conducted comparing them on proportion correct and RCS. ### Simulated Data and Analyses To pilot our experiment and data analysis script, we ran 12 'sim' experts and novices through the experiment. These simulated participants are programmed to respond pseudorandomly on each trial in the experiment with the assumption that they would be correct on half. We used the .txt files produced by these simulations to generate and test a LiveCode data extraction tool, and an R Markdown analysis script for plotting and analysing the data in this project. They provide a useful model for how participants would perform if their proportion correct were 50%. They are also helpful for debugging our experiment code and analysis script. The simulated data, LiveCode data extraction tool, R Markdown source files, resulting plots, and analysis script can be downloaded in the [Analysis][5] component of this project. A few plots from these simulated data in the current project are presented below, where each participant either randomly locates the correct spot on each trial or not. Note that 50% is far beyond the 1.45% chance level of performance on this task, but randomly selecting the correct location on each trial was far easier to simulate than a random mouse click for illustration purposes (colour filled = intact, white filled = scrambled). ![Simulated results][6] ## Ethics We have ethics clearance from human research ethics committees at The University of Adelaide: (Project codes: 13-39-18 and 18-93; Real World Perceptual Learning) and The University of Queensland (Project codes: 2014001677; Expertise & Non-Analytic Cognition, and 16-PSYCH-PHD-25-AH; Turning Novices into Experts). ## References Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009). PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (ToG), 28(3), 24. doi:10.1145/1531326.1531330 Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive psychology, 4(1), 55-81. doi:10.1016/0010-0285(73)90004-2 Drew, T., Evans, K., Võ, M. L.-H., Jacobson, F. L., & Wolfe, J. M. (2013). Informatics in Radiology: What Can You See in a Single Glance and How Might This Guide Visual Search in Medical Images? RadioGraphics, 33(1), 263–274. doi: 10.1148/rg.331125023 Drew, T., Vo, M. L. H., Olwal, A., Jacobson, F., Seltzer, S. E., & Wolfe, J. M. (2013). Scanners and drillers: Characterizing expert visual search through volumetric images. Journal of vision, 13(10), 3-3. doi10.1167/13.10.3 Drew, T., Võ, M. L. H., & Wolfe, J. M. (2013). The invisible gorilla strikes again: Sustained inattentional blindness in expert observers. Psychological science, 24(9), 1848-1853. doi: 10.1177/0956797613479386 Evans, K. K., Georgian-Smith, D., Tambouret, R., Birdwell, R. L., & Wolfe, J. M. (2013). The gist of the abnormal: Above-chance medical decision making in the blink of an eye. Psychonomic Bulletin and Review, 20(6), 1170–1175. doi:10.3758/s13423-013-0459-3 Evans, K. K., Haygood, T. M., Cooper, J., Culpan, A. M., & Wolfe, J. M. (2016). A half-second glimpse often lets radiologists identify breast cancer cases even when viewing the mammogram of the opposite breast. Proceedings of the National Academy of Sciences, 113(37), 10292-10297. doi:10.1073/pnas.1606187113 Krupinski, E. A., Graham, A. R., & Weinstein, R. S. (2013). Characterizing the development of visual search expertise in pathology residents viewing whole slide images. Human pathology, 44(3), 357-364. doi: 10.1016/j.humpath.2012.05.024 Krupinski, E. A. (1996). Visual scanning patterns of radiologists searching mammograms. Academic radiology, 3(2), 137-144. doi: 10.1016/S1076-6332(05)80381-2 Rehder, B., & Hoffman, A. B. (2005). Eyetracking and selective attention in category learning. Cognitive psychology, 51(1), 1-41. doi:10.1016/j.cogpsych.2004.11.001 Roads B, Mozer MC, Busey TA (2016) Using Highlighting to Train Attentional Expertise. PLoS ONE 11(1): e0146266. doi: 10.1371/journal.pone.0146266 Searston, R. A., & Tangen, J. M. (2017). The style of a stranger: Identification expertise generalizes to coarser level categories. Psychonomic bulletin & review, 24(4), 1324-1329. https://doi.org/10.3758/s13423-016-1211-6. Tear, M. J., Thompson, M. B., & Tangen, J. M. (2010, September). The importance of ground truth: An open-source biometric repository. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 54, No. 19, pp. 1464-1467). Sage CA: Los Angeles, CA: SAGE Publications. doi: 10.1177/154193121005401923 Thompson, M. B., Tangen, J. M., & McCarthy, D. J. (2014). Human matching performance of genuine crime scene latent fingerprints. Law and human behavior, 38(1), 84. doi: 10.1037/lhb0000051 Woltz, D. J. and Was, C. A. (2006). Availability of related long-term memory during and after attention focus in working memory. Memory & Cognition 34(3): 668–684. doi: 10.3758/BF03193587 Wood, G., Knapp, K. M., Rock, B., Cousens, C., Roobottom, C., & Wilson, M. R. (2013). Visual expertise in detecting and diagnosing skeletal fractures. Skeletal Radiology, 42(2), 165–172. doi: 10.1007/s00256-012-1503-5 [1]: https://files.osf.io/v1/resources/azesx/providers/osfstorage/5c57f69ee16f55001888a74b?mode=render [2]: https://osf.io/b2r9h [3]: https://osf.io/mwqfk [4]: https://osf.io/94zf6 [5]: https://osf.io/94zf6 [6]: https://files.osf.io/v1/resources/azesx/providers/osfstorage/5c54eefd76653c001b22263f?mode=render