Toward a universal decoder of linguistic meaning from brain activation

Contributors:
  1. Sam Ritter
  2. Nancy Kanwisher
  3. Matthew Botvinick

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: Prior work decoding linguistic meaning from imaging data has been largely limited to concrete nouns, using similar stimuli for training and testing, from a relatively small number of semantic categories. Here we present a new approach for building a brain decoding system in which words and sentences are represented as vectors in a semantic space constructed from massive text corpora. By efficiently sampling this space to select training stimuli shown to subjects, we maximize the ability to generalize to new meanings from limited imaging data. To validate this approach, we train the system on imaging data of individual concepts, and show it can decode semantic vector representations from imaging data of sentences about a wide variety of both concrete and abstract topics, from two separate datasets. These decoded representations are sufficiently detailed to distinguish even semantically similar sentences, and to capture the similarity structure of meaning relationships between sentences.

Wiki

Article http://rdcu.be/IqEP (link to Nature Communications web site) Datasets Table of contents Code leanDecoder.m trainVoxelwiseTargetPredictionModels.m Presentation Scripts Experiment 1 Experiments 2 and 3 Materials Experiment 1: Original 200 concepts Experiment 1: Images Experiment 1: Sentences Experiment 2: Passages Experiment 3: Passages Brain Maps Probabilistic activation overlap map for the...

Files

Loading files...

Citation

Tags

Recent Activity

Loading logs...

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message