Across Annelida, accessing the water column drives morphological and lifestyle modifications—yet in the primarily “benthic” scale worms, the ecological significance of swimming has largely been ignored. We investigated genetic, morphological and behavioural adaptations associated with swimming across Polynoidae, using mitogenomics and comparative methods. Mitochondrial genomes from cave and pelagic polynoids were highly similar, with non-significant rearrangements only present in cave Gesiella. Gene orders of the new mitogenomes were highly similar to shallow water species, suggestive of an underlying polynoid ground pattern. Being the first phylogenetic analyses to include the holopelagic Drieschia, we recovered this species nested among shallow water terminals, suggesting a shallow water ancestry. Based on these results, our phylogenetic reconstructions showed that swimming evolved independently three times in Polynoidae, involving convergent adaptations in morphology and motility patterns across the deep sea (Branchipolynoe), midwater (Drieschia) and anchialine caves (Pelagomacellicephala and Gesiella). Phylogenetic generalized least-squares (PGLS) analyses showed that holopelagic and anchialine caves species exhibit hypertrophy of the dorsal cirri, yet, these morphological modifications are achieved along different evolutionary pathways, i.e., elongation of the cirrophore versus style. Together, these findings suggest that a water column lifestyle elicits similar morphological adaptations, favouring bodies designed for drifting and sensing.