Paul J. Ferraro and J. Dustin Tracy **A reassessment of the potential for loss-framed incentive contracts to increase productivity: a meta-analysis and a real-effort experiment** forthcoming *Experimental Economics* Abstract: Substantial productivity increases have been reported when incentives are framed as losses rather than gains. Loss-framed contracts have also been reported to be preferred by workers. The results from our meta-analysis and real-effort experiment challenge these claims. The meta-analysis' summary effect size of loss framing is a 0.16 SD increase in productivity. Whereas the summary effect size in laboratory experiments is a 0.33 SD, the summary effect size from field experiments is 0.02 SD. We detect evidence of publication biases among laboratory experiments. In a new laboratory experiment that addresses prior design weaknesses, we estimate an effect size of 0.12 SD. This result, in combination with the meta-analysis, suggests that the difference between the effect size estimates in laboratory and field experiments does not stem from the limited external validity of laboratory experiments, but may instead stem from a mix of underpowered laboratory designs and publication biases. Moreover, in our experiment, most workers preferred the gain-framed contract and the increase in average productivity is only detectable in the subgroup of workers ($\sim$20\%) who preferred the loss-framed contracts. Based on the results from our experiment and meta-analysis, we believe that behavioral scientists should better assess preferences for loss-framed contracts and the magnitude of their effects on productivity before advocating for greater use of such contracts among private and public sector actors. "**ReadMe**": Supporting files for: A reassessment of the potential for loss-framed incentive contracts to increase productivity The files in the "code" and "data" subfolder reproduce the figure and tables in the paper. "code" subfolder: - "Meta.R" can be run in R, change line 1 to download directory, reads "data/MetaData.csv" makes a "figures" subdirectory, creates pgn's for Figures 1 through 4, and S.1 through S.8 within the subdirectory. - "ImportClean.do" [optional] can be run with Stata 14 or newer. It imports all the zTree files from the rawdata subfolder, saves them as Stata files, appends them drops extraneous variables and constructs variables required for analysis. Ultimately producing "AllSessions.dta" and "AllQuest.dta". - "PowerCalcs.do" can be run with Stata 14 or newer. It reads "data/AllSessions.dta" performs power calculations as described in the paper. Either run after ResultsTables.do or unmute line 6. - "ResultsTables.do" can be run with Stata 14 or newer. See line 1 regarding the cgmreg installation. It reads "data/AllSesions.dta" makes a "tables" subdirectory writes all tables (except 1, 2 & S1) in the paper to the subdirectory. "data" subfolder: - "AllSessions.dta" is Stata dataset for versions 14 or newer and contains all the data from the reported experiment. Variables: Period: period number Subject: subject number within sesion breaks: number of breaks a subject took that period Errors: number of incorrect attempts LossFrame: =1 if payment was lossframed that period. correct: number of correctly counted grids that period BreakPay: pay from breaks Session: session number ID: subject ID in experiment 100*Session + Subject Pref_LF_Tri_egen: =1 if subject indicated they preferred LF, 0 in they indicated no preference and -1 if they indicated a preferred gain frame. Pref_LF_egen: =1 if subject indicated they preferred LF, 0 otherwise Indif_egen: =1 if subjected indicated no fram preference, 0 otherwise Pref_LF_egen_LossFrame: Pref_LF_egen * LossFrame Indif_egen_LossFrame: Indif_egen * LossFrame StartLF: =1 if subject's period 1 pay was loss framed Period2: =1 if period==2 BiBreaks: =1 if breaks>0 - "AllQuest.dta" is Stata dataset for versions 14 or newer and contains demographic data for the subjects in the experiment. Variables: Period: period number Age: subject's reported age gender: subject's reported gender Session: session number Female: =1 is subject reported gender as female Citizen =1 if subject reported being a US citizen Race subject's reported race ID: subject ID in experiment 100*Session + Subject - "MetaData.csv" are summary statistics for the studies in our meta-analysis. Variables: Author: author(s) of the study, First et al. if >2 Note: descriptions if there are multiple experiments in one paper Year: year of publication Setting: categorization for Figure S7 SetNote: additional details about setting Setting3: categorization for Figure S8 Setting4: categories with distinguish induce cost from real effort Setting5: categorization for the main analysis. Reward: was reward "piece-rate" or "threshold" BenchmarkKnown: =1 if cut off for reward was disclosed to subjects, 0 otherwise OutsideOption: =1 if subjects had an alternative activity to the real effort task, 0 otherwise Within: =1 if the experiment was within subject design, 0 otherwise Subjects: description of subjects used Place: description of place experiment was conducted Location: country+ experiment was conducted Task: name of effort task used Contract Frame Duration: length of time of contract Prize: description of any non-monetary prizes used Stated: =1 if stated effort rather than real effort,0 otherwise Flag: =1 if the paper is not typical associate with loss-framed contracts RewardAdvance: =1 if subjects were actually given the reward (payment) before effort task. M.GF: mean effort in gain frame M.LF: mean effort in loss frame SD.GF: standard deviation of effort in gain frame SD.LF: standard deviation of effort in loss frame N.GF: number of subjects in gain frame N.LF: number of subjects in loss frame Source: where were the means SDs and Ns found, in most instances this is within the paper. exceptions are noted and url's are provided if available. The files in the "materials" subfolder allow replication of our experiment. Files require ztree 3.6.7 or newer. In background, the number of subjects should be set to 2 or more. Subject 1 is controlled by the experimenters and ensured stages did not advance until payments were complete "materials" subfolder: - "GainLoss_Grid2_4.ztt" for sessions in which Gain Frame Contracts are used in Round 1. - "LossGain_Grid2_4.ztt" for sessions in which Loss Frame Contracts are used in Round 1. - "instructions.pdf" A hard copy of page 1 was given to subjects. All pages were projected and read aloud, before relevant round. - "Questionnaire_Counting4.ztq" ztree questionnaire in which we elicit demographic controls. "rawdata" subfolder: - raw ztree files [Link to data and code for previous drafts][1] [1]: https://osf.io/3nqgd/