Within-subject designs tend to have higher statistical power and require fewer participants than between-subject designs, but this method presents its own detriments in terms of measurement. Carryover effects, such as practice and fatigue effects, may confound the observed effects in within-subjects designs. The present study aims to remedy these confounds by presenting participants with split-halves of the same psychometric scale, with participants never encountering the same item more than once. We compared two studies which were exactly the same except a scale was randomly split for 1 study. Study 1 had a traditional implementation, whereas Study 2 used split-halves. The effects of split-half implementation were evaluated using measures of scale and item reliability—Cronbach’s Alpha, McDonald’s Omega, individual item discrimination on a latent trait, and predictive ability. We observed that when items were randomized with this method, carryover effects were reduced.