Toward a universal decoder of linguistic meaning from brain activation

  1. Bin Lou
  2. Sam Ritter
  3. Nancy Kanwisher
  4. Matthew Botvinick

Date created: | Last Updated:

Identifiers: DOI | ARK

Creating DOI and ARK. Please wait...

Create DOI / ARK

Category: Project

Description: Prior work decoding linguistic meaning from imaging data has been largely limited to concrete nouns, using similar stimuli for training and testing, from a relatively small number of semantic categories. Here we present a new approach for building a brain decoding system in which words and sentences are represented as vectors in a semantic space constructed from massive text corpora. By efficiently sampling this space to select training stimuli shown to subjects, we maximize the ability to generalize to new meanings from limited imaging data. To validate this approach, we train the system on imaging data of individual concepts, and show it can decode semantic vector representations from imaging data of sentences about a wide variety of both concrete and abstract topics, from two separate datasets. These decoded representations are sufficiently detailed to distinguish even semantically similar sentences, and to capture the similarity structure of meaning relationships between sentences.


Datasets Table of contents Presentation Scripts Experiment 1 Experiments 2 and 3 Materials Experiment 1: Original 200 concepts Experiment 1: Images Experiment 1: Sentences Experiment 2: Passages Experiment 3: Passages Brain Maps Probabilistic activation overlap map for the language network Voxel informativeness overlap map for decoding in Expts. 1-3


Loading files...



Recent Activity

Loading logs...


Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message