Governments across the world have implemented restrictive policies to slow the spread of COVID-19. Mandatory face mask use has been a controversially discussed policy, among others, due to potential adverse effects on physical distancing. Using a randomized field experiment (N=300), we show that individuals keep a significantly larger distance from someone wearing a face mask than from an unmasked person. According to an additional survey experiment (N=456), masked individuals are not perceived as being more infectious than unmasked ones, but they are believed to prefer more distancing. This result suggests that, in times where mask use is voluntary, wearing a mask serves as a social signal for a preferred greater distance that is respected by others. Our findings provide strong evidence against a potential negative effect of masking on physical distancing, suggesting that mandatory masking would indeed be effective. However, as the social signal from masks may become diluted under a universal masking policy, the observed positive effect of masks on distancing may weaken under mandatory masking.