When asked to find a target dyad amongst non-interacting individuals, participants respond faster when the individuals in the target dyad are shown face-to-face, than when they are presented back-to-back. A growing body of literature interprets this in terms of specialized social interaction processing. However, human faces and bodies are salient directional cues that exert a strong influence on how observers distribute their attention. Here we report that a similar search advantage exists for ‘point-to-point’ and ‘point-to-face’ target arrangements constructed using arrows – a non-social directional cue. These findings indicate that the search advantage found for face-to-face dyads is a product of the directional cues present within arrangements, not the fact that they are processed as social interactions, per se. We then extend these findings to a variety of other objects, showing that all objects that elicit directional cueing also show effects in visual search similar to those attributed to social interactions. While the study of social interaction perception is an exciting new field, paradigms are still being refined. A multitude of published studies have already employed the face-to-face vs. back-to-back manipulation to isolate the neural, perceptual, and mnemonic processes recruited by interacting individuals. Our results highlight the influence of the directional cues present within these arrangements. It may prove significant (and problematic) that this manipulation not only alters the perception of social interaction, but also the way that participants attend to these displays.