Tutorial Wiki --------- Last updated: August 2, 2021. [Wilma Bainbridge][1] ---------- This is the wiki associated with the tutorial: Bainbridge, W.A. (2021). A tutorial on capturing mental representations through drawing and crowd-sourced scoring. Behavior Research Methods. You can access the paper [here][2]. I also now have a Youtube video series going through these steps and code that you can [watch here][3]. This may be especially helpful for walking through parts of the code, browser, server, and experimental interface. This tutorial is designed to teach the reader how to program an online experiment to **record drawings** as well as **score drawings** objectively. Wrapped into the tutorial are general lessons about image processing and web architecture. The affiliated code is heavily commented and designed in an iterative fashion to suit different skill levels. Please cite the above reference if you use any of the information learned from this tutorial! Getting Started --------------- First read through the [paper here][4] to learn the bigger framework behind running these drawing studies. There are two main parts to our methodology: 1) collecting drawing data, and 2) crowd-sourced scoring of drawing data. Those make up the two key parts of this tutorial, this wiki, and the code base. Before diving in, you may want to refer to [Useful Devices and Software][5] that you may need to prepare to begin experimenting. If you are encountering any problems, please also refer to [Currently Known Issues][6]. 1 - Collecting Drawing Data ----------------------- Drawing is an incredibly flexible method, and in many cases, drawing data can be collected in-person with just a pen and paper. However, you can also use the power of online experiments to collect drawings from hundreds or thousands of participants. Online drawing experiments give lots of flexibility and analytical advantages -- you can collect pen stroke data over time, you can look at erasing behavior, and these experiments are easy to deploy widely. 1. To get started with collecting drawing data, check out [Task Design Considerations][7]. You can also check out the [Youtube tutorial video][8] on this topic. 2. To code your experiment, you can use my code base for [Coding an Online Drawing Experiment][9]. In the video series, you can watch this [video tutorial on how I code drawing experiments][10]. 3. Decide how you will [Host Your Experiment][11]. You can watch this [video tutorial with some guidance on how to host your experiment][12]. Then, when it's done, here is a [tutorial on how to interpret your incoming data][13]. Also, if you want other options for implementing drawing experiments, check out other [Javascript Libraries for Drawing][14]. 2 - Crowd-Sourced Scoring of Drawings ------------------------------------- Once you have a set of drawings, you want to have them scored rapidly and objectively. In order to do this, you can create online experiments to crowd-source the scoring of the drawings. This generally requires a three step process that we follow with our code examples here: 1) generate the trials based on the drawings you have, 2) create and run the online scoring experiment, 3) analyze the scores from the scoring experiment. In order to score the drawings, these are good steps to take: 1. Determine [What To Score][15] from the drawings 2. Decide how you will [Host Your Experiment][16]. I recommend Amazon Mechanical Turk here because of its large subject population and rapid response times. 3. Then, you are ready to dive into the code for creating your [Online Scoring Experiments][17]! Here is a [video tutorial overall on how to design, host, and create these crowd-sourced scoring experiments][18]. 4. You can also score your drawings for measures like saliency or deep learning classification using [Toolboxes for Computationally Quantifying Drawings][19]. > Happy drawing! [1]: http://wilmabainbridge.com [2]: https://link.springer.com/article/10.3758/s13428-021-01672-9 [3]: https://www.youtube.com/playlist?list=PLGCpbhCUuouPqkugaICdhCNyHR0We1Li0 [4]: https://link.springer.com/article/10.3758/s13428-021-01672-9 [5]: https://osf.io/tgavx/wiki/Useful%20devices%20and%20software%20for%20drawing%20studies/ [6]: https://osf.io/tgavx/wiki/Currently%20Known%20Issues/ [7]: https://osf.io/tgavx/wiki/Drawing%20Task%20Design%20Considerations/ [8]: https://youtu.be/mR9RbBdvI04 [9]: https://osf.io/tgavx/wiki/Coding%20an%20Online%20Drawing%20Experiment/ [10]: https://youtu.be/BS14kcqEVZU [11]: https://osf.io/tgavx/wiki/Hosting%20Online%20Experiments/ [12]: https://youtu.be/3Q0egTL4xDw [13]: https://youtu.be/0QerZspmre0 [14]: https://osf.io/tgavx/wiki/JavaScript%20Libraries%20for%20Drawing/ [15]: https://osf.io/tgavx/wiki/What%20to%20Score%20in%20Drawings/ [16]: https://osf.io/tgavx/wiki/Hosting%20Online%20Experiments/ [17]: https://osf.io/tgavx/wiki/Scoring%20Drawings%20Online/ [18]: https://youtu.be/sreHb1B66XM [19]: https://osf.io/tgavx/wiki/Toolboxes%20for%20Computationally%20Quantifying%20Drawings/