If the surrounding of a visual target unexpectedly starts to move during a fast goal-directed hand movement, the hand reflexively moves along with it. This is known as the ‘manual following response’. One explanation for this response is that it is a compensation for inferred self-motion in space. Previous studies have shown that background motion gives rise to both postural responses and deviations in goal-directed hand movements. To evaluate whether compensation for inferred self-motion is responsible for the manual responses we examined whether galvanic stimulation of the vestibular system would give rise to similar deviations in hand movements. Standing participants tried to quickly tap on targets that were presented on a horizontal screen. Participants could infer self-motion on some of the trials, either from galvanic vestibular stimulation or from background motion. Both perturbations took place during the hand movement. It took both the head and hand about 45 ms longer to respond to background motion than to respond to galvanic stimulation. The head responded in a similar manner to both types of perturbations. The hand responded about as expected to galvanic stimulation, but much more vigorously to background motion. Thus, the manual response to background motion is probably not a direct consequence of trying to compensate for inferred self-motion. Perhaps the manual following response is a consequence of an error in binding motion information to objects.