Visuospatial neglect, a common lateralised disorder of perception following acute stroke, is widely considered a disconnection syndrome, though the patterns of dysconnectivity associated with neglect remain unclear. Here we had 530 acute stroke survivors undertake standardised neglect testing and routine clinical imaging. The data were used to conduct voxel-wise, tract-level, and network-level lesion mapping analyses aimed at localising the neural correlates of left and right egocentric (body-centred) and allocentric (object-centred) neglect. Only minimal anatomical homogeneity was present between the correlates of right and left egocentric neglect across all analysis types. This finding challenges previous work suggesting that right and left neglect are anatomically homologous, and instead suggests that egocentric neglect may involve damage to a shared, but hemispherically asymmetric attention network. By contrast, egocentric and allocentric neglect were associated with dysconnectivity in a distinct but overlapping set of network edges, with both deficits related to damage across the dorsal and ventral attention networks. Critically, this finding suggests that the distinction between egocentric and allocentric neglect is unlikely to reflect a simple dichotomy between dorsal versus ventral networks dysfunction, as is commonly asserted. Taken together, the current findings provide a fresh perspective on the neural circuitry involved in regulating visuospatial attention, and provide important clues to understanding the cognitive and perceptual processes involved in this common and debilitating neuropsychological syndrome.