Both temporal and frontoparietal brain areas are associated with the representation of knowledge about the world, in particular about actions. However, what these brain regions represent and precisely how they differ remains unknown. Here, we reveal fundamentally distinct functional profiles of lateral temporal and frontoparietal cortex: Using fMRI-based MVPA we found that frontoparietal areas encode representations of observed actions and corresponding written sentences in an overlapping way, but these representations did not generalize across stimulus type. By contrast, only left lateral posterior temporal cortex (LPTC) encoded action representations that generalize across observed action scenes and sentences. The representational organization of stimulus-general action information in LPTC could be predicted from models that describe basic agent-patient relations (object- and person-directedness) and the general semantic similarity between actions. The match between action videos and sentences in LPTC and its representational profile indicate that this region encodes general, conceptual aspects of actions whereas frontoparietal representations appear to be tied to specific stimulus types.