This repository contains human and network data as presented and used in the paper: "SATBench: Benchmarking the speed-accuracy tradeoff in object recognition by humans and dynamic neural networks".
Our human dataset is collected using a reaction time paradigm proposed by McElree & Carrasco [2] where observers are forced to respond at a beep which sounds at a specific time after target presentation. Varying the beep interval across several blocks helps us collect object recognition data across different reaction times (500ms, 900ms, 1100ms, 1300ms, 1500ms). We evaluate dynamic neural networks using the same paradigm with computational FLOPs used as an analog for reaction time. Resulting speed-accuracy tradeoff (SAT) curves for humans and networks are shown below.
![SAT curves for humans and dynamic neural networks across all conditions][1]
[1]: https://github.com/ajaysub110/satbench/blob/main/assets/human-network-sat.jpeg
