Several previous studies reported reduced leftward lateralization in blind participants’ samples compared to the sighted population. The origins of this difference remain unknown. Here, we tested whether functional lateralization is connected with the structural characteristics of white matter tracts [corpus callosum (CC), uncinate fasciculus (UF), and superior longitudinal fasciculus (SLF)], as suggested by previous studies conducted in the typical sighted population. Twenty-three blind and 21 sighted adult participants were tested during fMRI with a semantic decision paradigm presented both auditorily and in the modality appropriate for reading (actually for the blind and visually for the sighted). Lateralization indices (LI) were calculated based on the activations. The fractional anisotropy (FA) measure was extracted from the white matter tracts of interest. Correlation analyses testing the relationship between FA and LI were conducted. The reduced leftward lateralization of both speech processing and reading-related activations was replicated. Nevertheless, the relationship between the structural integrity of the CC and LI and between the asymmetry of the intrahemispheric tracts and LI was not confirmed, possibly due to the lack of power. The sources of the reduced lateralization of the language network in the sensory-deprived population remain unknown. Further studies should account for environmental variables (e.g., the frequency of contact with written language) and the complexity of the factors that may influence the functional lateralization of the human brain.