This project was created by Tarryn Balsdon. The project is a collaboration with Dr Pascal Mamassian and Dr Valentin Wyart, who are the senior authors and sourced the funding for the project. **Aims** This experiment is part of a wider project examining the neural computations of decision confidence. As an initial step, this behavioural experiment tests an experimental design to be implemented in the context of EEG. The main goal is therefore to be able to measure, on a behavioural level, variability in decision processes and confidence, such that they can be further examined on a neural level. However, the experimental design is rich enough to allow the exploration of some further interesting hypotheses. The main hypothesis (Q1) to test is whether the sub-optimality in decision-making and the sub-optimality in decision confidence are correlated. A secondary hypothesis (Q2) is whether the ability of observers to collect evidence for a particular level of accuracy in linked to their metacognitive ability. A further question (Q3), linked to the first hypothesis, is how additional or insufficient evidence affects decision-making and decision confidence (and their relation). Given the expected richness of the data, we outline below the planned analysis with the expectation that further exploratory analysis will be required. Having recorded the planned analyses, sufficient correction can be made to any further inferential statistics not outlined here. **Methods** **Participants** 20 Observers will be recruited via mailing lists and word of mouth. They will be required to have normal or corrected to normal vision. Written consent will be requested before beginning the experiment, after a full explanation of the task has been given. In the case where a participant is unable to perform the task (details below), they will be excluded from further analysis and replaced, such that the data of 20 participants are included in the full analysis, and any instance of this will be explicitly reported in any manuscript. **Material** Stimuli will be presented on a 24” BenQ LCD monitor running at 60 Hz with a resolution of 1920x1080 pixels and mean luminance 45cd/m2. Stimulus generation and presentation is controlled by MATLAB (Mathworks) and the Psychophysics toolbox (Brainard, 1997; Kleiner et al., 2007; Pelli, 1997), run on a Mini Mac (Apple Inc). An EyeLink 1000 infrared monocular eye-tracker (SR Research Ltd. Ontaro, Canada), running at 500 Hz on a dedicated PC, will be used to monitor blinks and pupil dilation in the observer’s dominant eye. Observers will view the monitor from a distance of 60 cm, with their head supported by a chin rest. **Stimuli** Stimuli will be oriented gabors subtending 4 degrees of visual angle, with a michaelson contrast of 0.15 and spatial frequency of 2 cycles/degree. Gabors will be embedded in spatially filtered noise with an amplitude spectrum of 1/f1.25. On each trial a set of orientations will be drawn from one of two circular Gaussian distributions, estimated by a Von Mises distribution with concentration kappa = 0.5, and means of -45 and 45 (ang) degrees from vertical (0 degrees). A circular colour guide will be drawn along with the gabor stimuli to aide participants in the visualisation of these distributions, where the red and blue RGB channels reflect the probability density of each angle in the two distributions respectively. An example stimulus, complete with central fixation dot, is shown in Figure 1B, along with a description of the probabilities of the orientation under each distribution. In each trial a number of stimuli will be presented sequentially, at a rate of 4 Hz: gabor stimuli are displayed for 200 ms (150 ms at full contrast, temporally bordered by a 25 ms cosine ramp) with a 50 ms inter-stimulus interval in which only the colour guide and fixation point are displayed, as shown in Figure 1A. For each observer, 100 trials will be generated for the experiment, each with 40 unique gabor stimuli. These pre-defined trials will be used repeatedly throughout the experiment. ![Figure 1. Stimulus presentation. A) Stimuli are presented at a rate of 4 Hz: 50ms inter-stimulus interval, followed by 200 ms of stimulus (including a 50 ms cosine ramp at onset and offset). The colour guide and fixation point are always present, even in the inter-stimulus interval. B) Example stimulus with colour guide. The distance of the orange and blue lines from the fixation point indicate the probability density of the orientation under each distribution (as also indicated by the colour wheel). This example stimulus (45 degrees) is far more likely under the orange distribution than the blue, as indicated by then length of the respective arrows.][1] ***Figure 1. Stimulus presentation.** A) Stimuli are presented at a rate of 4 Hz: 50ms inter-stimulus interval, followed by 200 ms of stimulus (including a 50 ms cosine ramp at onset and offset). The colour guide and fixation point are always present, even in the inter-stimulus interval. B) Example stimulus with colour guide. The distance of the orange and blue lines from the fixation point indicate the probability density of the orientation under each distribution (as also indicated by the colour wheel). This example stimulus (45 degrees) is far more likely under the orange distribution than the blue, as indicated by then length of the respective arrows.* **Procedure** The task is a modified version of the weather prediction task (Knowlton et al., 1996; Poldrack et al., 2001; Gluck et al., 2002; Yang and Shadlen, 2007). Throughout the experiment, the observer’s task is to estimate which distribution the stimuli were drawn from. They will be instructed to press the left arrow (of a standard querty keyboard) for the blue distribution and the right arrow for the orange distribution. The exact description of the task will be varied over three conditions: referred to henceforth as ‘stopping condition’, ‘free response condition’, and ‘replay condition’, completed over two sessions, each of less than one hour. The first session will begin with a brief training session of 20 trials. In each trial observers will be shown either 4, 8, 12, or 16 stimuli and will be asked to enter their response only after the sequence of stimuli is complete. After their response, they will receive feedback as to which was the actual distribution the stimuli were drawn from, by the colour change of the central fixation point (blue for the blue distribution and orange for the orange distribution). They will then have the opportunity to ask any questions before receiving the instructions for the first part of the experiment, the stopping condition. In the stopping condition, observers will be asked to enter their response as soon as they feel they have seen enough stimuli to reach a certain accuracy level. There will be three accuracy levels to aim for: 70% correct, 85% correct, and 90% correct. The stimuli will continue until a response is entered, up to 40 stimuli, and observers will be explicitly informed that the more stimuli they see, the better their chance of being correct. Before beginning the experiment, after the initial training, observers will have the opportunity to practice this condition over 10 trials for each accuracy level. There will be no single trial feedback in the main experiment, however, observers will be told their performance over the last 20 trials, every 20 trials (and over the 10 practice trials) in the ‘stopping condition’. Participants will be encouraged to perform close to the accuracy level with the symbolic award of points (10 points for reaching exactly the accuracy level aimed for over the last 20 trials, or 5 points for coming within 5% of the aim). Each observer will be asked to complete two blocks of each accuracy level (with 100 unique trials per block, in pseudo-random order) forming six blocks to be performed in pseudo-random order. In the second session, observers will complete the free response condition, followed by the replay condition. The free response condition will be exactly the same as the stopping condition, except that this time observers will be instructed to respond when they “feel ready”. Observers will be asked to complete three blocks of 100 trials, over which they will receive no feedback as to their performance. The replay condition will consist of another three blocks of 100 trials, also with no feedback. In this condition observers will be shown a specific number of stimuli on each trial and will be unable to respond until the sequence of stimuli is complete. After their response, observers will be prompted to give a confidence rating, with the word ‘Confidence?’ written on the screen. They will be instructed that the confidence rating should describe how certain they were that they were correct on that trial, ranging from 1 – 4, where 1 represents low confidence and 4 represents high confidence. The experiment halts until both responses have been given (in order) and there is no limit on the time allowed to respond. However, observers will be instructed to respond as quickly but as accurately as they can. Unbeknownst to the observers, the trials in the replay condition are defined based on their responses in the free response condition. The trials will be made up of three conditions defined by the number of stimuli the observer is shown in each trial. In one condition they will be shown the median number of stimuli that they chose to respond to, on those same trials when they were free to respond (each of the same 100 trials were repeated in the three blocks), in a second condition they will be shown 2 fewer stimuli than the minimum number of stimuli they used to respond on those trials in the free response condition, and in a third condition they will be shown 4 additional stimuli than the maximum number of stimuli they used to respond on those trials in the free response condition. These trials will all be pseudo-randomly interleaved throughout the three blocks of the replay condition. The number of stimuli used in the free response condition will be defined as the stimulus at which observers responded, after subtracting 150 ms from their response time, and rounding down to the nearest full stimulus shown. Because observers will need to respond only after a variable number of stimuli have been presented, a queue will be used to indicate when the stimuli have finished; the central fixation point will be changed to red for 200ms once all the stimuli have been presented. **Analysis** 1. Behavioural performance. Proportion correct will be examined over seven conditions: the three accuracy aims in the stopping condition, overall performance in the free response condition, and the three stimuli manipulations in the replay condition (-2 stimuli, same median number of stimuli, and +4 stimuli). Observers will be removed from further analysis if their performance does not rise significantly above chance (50% correct). Another key measure is the number of stimuli observers choose to respond based on. This will be examined in the stopping condition and in the free response condition, by the distribution of the number of stimuli seen over the 100 individual trials. Reliability will be assessed based on the difference in the number of stimuli across repeated trials. This can also be broken down into the actual decision evidence provided by each stimulus seen, based on the probability of the orientation under each distribution. These two measures in the stopping condition will then be combined to give an indication of how well an observer is able to monitor the probability that they are correct as they collect evidence for making a decision. This measure will then be compared to observers’ metacognitive ability in the replay condition, measured through their ability to distinguish correct and incorrect responses using their confidence ratings, given their underlying sensitivity. This analysis will address whether observers’ ability to collect evidence to a certain accuracy level is linked to their metacognitive ability (Q2). A final behavioural measurement concerns the influence of additional evidence on the decision and confidence level, in the replay condition (Q3). It is hypothesised that, when the observer is shown 4 additional stimuli in the replay condition, they could have already made their decision before all the stimuli are presented, hence, their performance in this condition will be similar to their performance when they are shown the same number of stimuli as previously, in the free response condition. The additional evidence, however, could still influence their confidence, such that their metacognitive accuracy increases whilst their performance remains the same. A different prediction is made for the case where observers are shown 2 fewer stimuli than in the free response condition. This should impair their performance, and result in them being less confident. It is therefore expected that performance will decrease, but metacognitive accuracy may stay relatively the same as when observers are shown the same number of stimuli as previously. 2. Pupil dilation Raw pupil dilation data will first be cleaned by eliminating trials with large changes in pupil dilation measures, such as those measured with blinks. The data will then be downsampled to 10Hz by averaging over successive bins of 100 ms and the DC component of the signal removed by subtracting the mean. An autoregressive model with exogenous inputs (ARX) will then be used to assess the impulse response function of the pupil to experimental variables (Zénon, 2017). The standardised luminance of the stimuli (from the average pixel values), and the decision update of the stimuli will be used as exogenous inputs to the model, along with the behavioural outcome of each trial - whether the observer was correct. In the replay condition, the observer’s confidence report will also be incorporated. Previous research suggests that pupil response may vary with confidence, in which case, the impulse response of the pupil will vary significantly with confidence rating. If this is the case, the same analysis can be conducted on those trials which were the same as presented in the replay condition, but in which the observer did not make a confidence report. The impulse response to luminance, decision update, and performance will be used as sanity checks on the data and any differences correlated with confidence. 3. Computational modelling Computational modelling, based on Drugowitsch et al., 2016, will be used to assess the contribution of several parameters to behavioural decisions. The models assume that observers accumulate evidence for a decision, with each stimulus presentation (the decision update, based on the probably of the gabor orientation, given the underlying distributions of each decision option). This evidence accumulation is corrupted by additive noise, which is normally distributed. The standard deviation of the additive noise is parameterised in the model. The accumulation of evidence is also moderated by a weight function, where observers may display a recency bias (weighting more recent decision updates more strongly than earlier ones) or a primacy bias (weighting earlier decision updates more strongly than later ones). This weight is parameterised in the model as an exponential function affecting the decision update and associated noise. Observers may also adopt some bias, for example, by preferring one response to another. This is parameterised as a constant change in the starting point of the evidence accumulation, where 0 would correspond to unbiased accumulation. Finally, it is assumed that the observer adopts a threshold at which they have enough evidence to make a response. It is this threshold that is altered to control performance; where lowering the threshold will decrease the observer’s probability of being correct (less evidence is accumulated before making a decision), and raising the threshold will increase the observer’s probability of being correct (more evidence is accumulated before making a decision). The suboptimality in decision making can be assessed based on the noise and weight parameters fitted to individual observers’ responses. If the suboptimality in confidence judgements is related to the suboptimality in decision making, then observers’ metacognitive ability will be at least correlated to the magnitude of these parameters (Q1). Furthermore, if observers’ ability to collect evidence to a particular level of accuracy is based on metacognitive evidence, then, in answer to Q2, their ability to set and maintain evidence accumulation thresholds will also be linked to their metacognitive ability (despite the underlying suboptimalities in decision making). Finally, in address of Q3, modelling can further be used to examine how evidence is treated when it surpasses an implicit threshold (when observers are shown additional stimuli in the replay condition), or when there is insufficient evidence to reach that implicit threshold (when observers are shown fewer stimuli in the replay condition). Here, three hypotheses can be distinguished: 1) evidence stops being accumulated once the threshold is reached (absorbing boundary, Zhang, Bogacz, and Holmes, 2009), 2) evidence will not be accumulated beyond the threshold, but contrary evidence and noise still affect accumulation (reflecting boundary, Zhang, Bogacz, and Holmes, 2009), or 3) the observer ignores their implicit threshold and accumulates all evidence as an ideal observer. If the threshold remains important for the decision (cases 1 and 2), despite the observer being unable to control accumulation in the replay condition, it can be questioned how these changes to evidence accumulation might affect decision confidence. **References** Brainard, D. H. (1997). The psychophysics toolbox. *Spatial vision,* 10, 433-436.
 Drugowitsch, J., Wyart, V., Devauchelle, A. D., & Koechlin, E. (2016). Computational precision of mental inference as critical source of human choice suboptimality. *Neuron,* 92(6), 1398-1411. Gluck, M.A., Shohamy, D., and Myers, C. (2002). How do people solve the ‘‘weather prediction’’ task?: individual variability in strategies for probabilistic category learning. *Learn. Mem.* 9, 408–418. Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What’s new in Psychtoolbox-3. *Perception,* 36(14), 1. Knowlton, B.J., Mangels, J.A., and Squire, L.R. (1996). A neostriatal habit learning system in humans. *Science* 273, 1399–1402. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. *Spatial vision,* 10(4), 437-442. Poldrack, R.A., Clark, J., Pare ́ -Blagoev, E.J., Shohamy, D., Creso Moyano, J., Myers, C., and Yang, T., and Shadlen, M.N. (2007). Probabilistic reasoning by neurons. *Nature* 447, 1075–1080. Zénon, A. (2017). Time-domain analysis for extracting fast-paced pupil responses. *Scientific Reports*, 7, 41484. Zhang, J., Bogacz, R., & Holmes, P. (2009). A comparison of bounded diffusion models for choice in time controlled tasks. *Journal of mathematical psychology*, 53(4), 231-241. [1]: https://mfr.osf.io/export?url=https://osf.io/4hz3s/?action=download&mode=render&direct&public_file=False&initialWidth=739&childId=mfrIframe&parentTitle=OSF%20%7C%20Figure1.jpg&parentUrl=https://osf.io/4hz3s/&format=2400x2400.jpeg