Meaningfulness and Familiarity Expand Visual Working Memory Capacity

Visual working memory is traditionally studied using abstract, meaningless stimuli. Although studies using such simplified stimuli have been insightful in understanding the mechanisms of visual working memory, they also potentially limit our ability to understand how people encode and store conceptually rich and meaningful stimuli in the real world. Recent studies have demonstrated that meaningful and familiar visual stimuli that connect to existing knowledge are better remembered than abstract colors or shapes, indicating that meaning can unlock additional working memory capacity. These findings challenge current models of visual working memory and suggest that its capacity is not fixed but depends on the type of information that is being remembered and, in particular, how that information connects to preexisting knowledge.

lab scale up to behavior in the real world.Building on classic findings (Ballard et al., 1995), recent working memory studies have incorporated more realistic settings, for example, by immersing participants in virtual reality and allowing them to move more freely while performing memory tasks (Draschkow et al., 2021).Furthermore, recent work has used visually more complex and semantically rich stimuli, such as images of real-world objects, and has found that participants remember more about these meaningful stimuli relative to meaningless and abstract stimuli (e.g., Brady et al., 2016;Brady & Störmer, 2022;Thibeault et al., 2024;Torres et al., 2023).A recent study that used ambiguous two-tone images that could either be recognized as a face or not demonstrated that recognizing the stimulus as meaningful-and not physical differences between stimuli-is the critical determinant for driving these memory benefits (Asp et al., 2021).Similarly, Brown and Wesley (2013) showed that visuospatial working memory for arbitrary patterns is improved when those patterns are more semantically meaningful to observers.Thus, how a stimulus is interpreted by the visual system and in particular how it connects to existing knowledge influences working memory capacity even for physically identical stimuli.This is consistent with earlier models of conceptual short-term memory that postulate that conceptual and semantic information of current perceptual input can be rapidly retrieved from longterm memory, allowing perceptual information to be associated with conceptual knowledge at least over brief periods of time (Potter, 1993; see also Endress & Potter, 2014).The more recent studies on the role of meaningfulness in visual working memory show that these associations can be maintained over longer intervals and thus play a more persistent role extending throughout the encoding, maintenance, and retrieval phases.Overall, this suggests that because they used simple stimuli, many past working memory studies systematically underestimated our capacity to process and remember information (e.g., in more naturalistic settings information is not simple and abstract but conceptually rich and meaningful).
These increases in task performance for meaningful stimuli are modulated by additional aspects, such as familiarity and expertise.For example, more common and familiar real-world objects are better remembered than less familiar ones (Li et al., 2020;Starr et al., 2020), and memory performance for letters and characters from a familiar relative to a nonfamiliar language is increased (Alvarez & Cavanagh, 2004;Ngiam et al., 2018;Zimmer & Fischer, 2020).Furthermore, individuals with expertise in specific object categories such as national flags (Conci et al., 2021), cars (Curby et al., 2009), or Pokémon (Xie & Zhang, 2017) can also exhibit improved working memory for their objects of expertise.
One important consideration is whether these improvements in task performance are indeed a reflection of increased working memory capacity-that is, whether people are able to maintain more visual information in an active state or, alternatively, if they are either storing familiar and recognizable information more efficiently or relying on other more passive forms of memory (see more on this below), both of which could also result in similar behavioral benefits.Efficient coding accounts (e.g., Bates et al., 2019;Brady et al., 2009) would suggest that if participants have knowledge about familiar objects they might be able to store information about these objects in a compressed format, which would reduce the overall amount of information to be maintained and thus free up additional resources.According to this view, rather than meaningful objects increasing working memory capacity, familiar objects would instead each take less capacity to store.To test this possibility, several investigations have examined neural delay activity during the working memory maintenance period, a strong marker of persistent active maintenance of information.Specifically, these studies used the contralateral delay activity (CDA) measured using electroencephalography (EEG) that has been strongly associated with visual working memory capacity (Vogel & Machizawa, 2004).Comparing the amplitudes of the CDA during the working memory delay period between meaningful stimuli versus more abstract stimuli, this work found reliable increases in neural activity for meaningful and familiar stimuli relative to nonmeaningful stimuli (Asp et al., 2021;Brady et al., 2016;Thibeault et al., 2024; but also see Quirk et al., 2020). 2 Thus, the increase in task performance is associated with more neural delay activity, not less, strongly suggesting that more information is being actively maintained during the retention period.This interpretation aligns with how increases in neural delay actively have been interpreted in the past, in which lower versus higher neural delay activity has been associated with lower versus higher behavioral indexes of capacity for simple stimuli (for a review, see Luria et al., 2016).In addition to these neural measures, another piece of evidence supporting our account that familiar and recognizable objects can increase working memory capacity (and are not compressed or reduced to highly abstracted representations) comes from an experiment in which participants were tested on details of the to-be-remembered objects (e.g., within-category change at test): A similar performance benefit was found for these objects-with-detail conditions (Brady et al., 2016).Collectively, these results suggest that observers are able to maintain more information during the delay period of working memory tasks with meaningful stimuli, effectively increasing the capacity of visual working memory.

Relation to Episodic and Semantic Long-Term Memory
To understand the role of meaningfulness in visual working memory it is important to consider how longterm memory, a passive memory storage with essentially unlimited capacity (Brady et al., 2008), may be involved.Different kinds of long-term memories need to be considered separately when addressing this question.Episodic long-term memory refers to memory representations that are specific to a particular item seen at a particular time.Semantic long-term memory refers to previously learned broad conceptual knowledge that is mostly context-free (e.g., knowing what a car looks like and what it is used for).
Previous work has often focused on the role of episodic long-term memory in working memory tasks.For example, to try to create a process-pure measure of working memory, prior work often avoided using more realistic stimuli to eliminate possible contributions from episodic long-term memory or chunking strategies based on familiarity or knowledge (Cowan, 2001;Lin & Luck, 2012).Thus, one possibility for how meaningfulness interacts with working memory is that meaningful stimuli allow the engagement of episodic long-term memory, which allows participants to replace or supplement working memory with long-term memory representations (making these stimuli non-process-pure measures of working memory).This can definitely occur in some situations.For instance, studies have shown that searching for objects in natural scenes can involve both short-term memory and long-term memory, which can effectively increase the amount of information one can remember (Hollingworth, 2004).Several studies have also shown that when participants are encouraged to form episodic long-term memories of the to-be-tested stimuli either by implementing a separate learning block (Bartsch & Shepherdson, 2022), or by repeating the same stimulus array throughout the experiment (Carlisle et al., 2011), task performance is improved.
However, most studies that examine the role of meaningfulness in visual working memory do not encourage-or even allow-participants to form episodic long-term memories of task-relevant features (e.g., Asp et al., 2021;Brady & Störmer, 2022).The specific stimuli used are not learned prior to the task, are presented in random order and at random locations during the experiment, and (at least in most studies) are not repeated (or repeated minimally).Thus, no specific episodic long-term memory can be formed and used to perform these tasks.Instead, the visual stimuli activate preexisting broad conceptual or semantic knowledge about them, and this semantic memoryrather than the engagement of episodic long-term memoryimproves working memory performance.Consistent with this interpretation, experiments that have recorded neural activity using EEG have shown that neural delay activity during the retention interval is higher for meaningful and recognizable stimuli relative to unrecognizable stimuli as well as simple abstract features (Asp et al., 2021;Brady et al., 2016;Thibeault et al., 2024; discussed above in more detail).In contrast, studies that allowed or even encouraged participants to use episodic long-term memories to replace working memory have found decreases in this delay activity (Carlisle et al., 2011).Thus, meaningfulness can unlock additional working memory capacity by connecting to semantic long-term memories and sometimes improve performance because specific episodic long-term memories supplement working memory representations.

Structure of Meaningful Visual Working Memory Representation
How can long-term knowledge support visual working memory?A model of visual working memory that assumes hierarchically structured memory representations, in which lower level visual features are connected to higher level semantic representations, naturally predicts that meaningful objects should be more robust because they can be represented in a higher dimensionality (Brady & Alvarez, 2011;Brady et al., 2011;Markov et al., 2021).For instance, consider the case of an ambiguous, difficult-to-recognize stimulus: When not recognized as meaningful, this image would be stored only in terms of its low-level visual features; however, once an observer can recognize it as meaningful (e.g., a car), this image instantly connects to knowledge and can thus be remembered in terms of these meaningful features (e.g., shapes of headlights, function of the car, sizes of the wheels, etc.) and not just in terms of its low-level visual dimensions (Asp et al., 2021; see Fig. 1).Thus, activating additional high-level features can increase distinctiveness between stimuli (Cohen et al., 2014;Veldsman et al., 2017) and reduce interference (Wyble et al., 2016).
The framework of a hierarchically structured working memory system, in which lower level representations are directly linked with higher level representations, predicts that even simple, low-level features should be influenced by the meaningfulness of a stimulus.Indeed, recent studies have found that this is the case: Working memory performance for simple low-level features that are not themselves meaningful is increased if they are encoded in a meaningful context (Allen et al., 2021;Chung et al., 2023aChung et al., , 2023b)).Specifically, when participants were asked to remember just the colors of a set of objects, they had stronger memories for these colors when they were part of real-world objects (or recognizable silhouettes) compared with unrecognizable or scrambled shapes even when object identities remained irrelevant throughout the task (Chung et al., 2023a).Importantly, the colors were randomly assigned to these color-neutral objects (e.g., people saw a red or blue sofa or a green or yellow balloon), ensuring that this effect could not be due to episodic long-term memories for specific colored objects.Thus, these recent results may suggest that meaningful stimuli can serve as an effective scaffold to encode and maintain associated low-level features.This contrasts with previous theories of visual working memory that assume only the higher level, abstracted object is represented in working memory (Huang & Awh, 2018; see Fig. 2).

Encoding Strategies Matter
For the meaningfulness benefit to arise, it is critical that participants enough time to process and recognize the stimuli at encoding.Thus, visual working memory studies using meaningful stimuli have consistently used relatively long encoding times-approximately 1,000 ms for set sizes between four and six (e.g., Asp et al., 2021;Brady et al., 2016;Thibeault et al., 2024) versus approximately 100 ms for studies using simple stimuli (e.g., Alvarez & Cavanagh, 2004;Luck & Vogel, 1997) 3 -with the idea that this would allow deep processing of each item on the display, ensuring that the stimuli can be recognized and connected to semantic knowledge.A more direct test of the role of encoding and in particular in-depth processing of each item was provided by a recent study that presented stimuli at encoding either sequentially to encourage participants to focus on each item one at a time or simultaneously all at once (Brady & Störmer, 2022).The results showed that the meaningfulness benefit in visual working memory was consistently present across both encoding conditions, but its magnitude was modulated by how stimuli were presented at encoding such that sequential presentation resulted in a larger real-world object benefit relative to simultaneous presentation (see also Chung et al., 2023a, Fig. 1.Example displays of visual working memory tasks and illustration of representational maps in working memory.Researchers have conventionally used (a) abstract, simple features such as colored circles as working memory stimuli.Such stimuli can be represented only in low-level visual features (e.g., location and color), resulting in increased interference among memoranda and fewer distinct features to store in memory.More meaningful, familiar memory stimuli (b) such as real-world objects activate additional semantic features, taking advantage of a much larger representational space (e.g., functions of objects, conceptual knowledge).This can increase distinctiveness between stimuli, effectively reducing interference and aiding visual working memory performance (Wyble et al., 2016).Stimuli that are complex but are not meaningful (i.e., scrambled objects) cannot be mapped onto a semantic space, resulting in lower working memory performance (e.g., Alvarez & Cavanagh, 2004).
Fig. 2. Two potential structures of object memory representation.A hierarchical working memory model suggests that the memory content is represented as (a) bundles of visual features along the visual hierarchy.In this case, a car can be represented as a bundle of lowlevel features (i.e., edges, lines, colors) and higher level parts (i.e., headlights, wheels), all the way up to a whole object representation (i.e., the car).Such a structure allows higher level features to act as an effective memory scaffold for associated lower level features (see the dashed line; e.g., Chung et al., 2023a).A content-free pointer model suggests that what is stored in working memory is simply (b) the broad and abstracted concept of "car," which serves only as a pointer to a long-term memory representation, allowing only retrieval of common car features, not the specific detailed features of this car.In this case, the visual details of the object would be degraded in working memory.
Experiment 3).These findings suggest that the encoding format and strategy matter for visual working memory and may differentially affect performance depending on the stimulus type.Specifically, although sequential encoding might be particularly beneficial for recognizable and meaningful objects, this may not be the case for simple visual features (Brady & Störmer, 2022;Chung et al., 2023a) for which other encoding strategies, such as ensemble processing or chunking, a larger role (Chunharas & Brady, 2023).Thus, how information is encoded can modify its representational structure.

Labeling in Visual Working Memory
Because stimuli that are semantically meaningful (such as real-world objects) can easily be coded verbally, not just visually, another aspect to consider is whether and to what extent verbal codes may be used in working memory tasks with semantically rich stimuli.The classic dual-code theory of memory posits that there is a memory advantage for information that can be coded in two formats, for example, visually and verbally, relative to a single format (Paivio, 1971).In the case of working memory studies using recognizable versus unrecognizable stimuli, the meaningfulness benefit could be supported by such dual coding.To test whether verbal encoding is solely responsible for such benefits, several studies have implemented verbal interference tasks in which participants perform a concurrent articulatory suppression task (e.g., verbally repeating several digits each trial) as they complete the visual working memory task and found the same pattern of data as in conditions without such an additional verbal task (e.g., Brady et al., 2016;Brady & Störmer, 2022;Chung et al., 2023b;Starr et al., 2020).These results provide strong evidence that verbal labeling is not the main source of such benefits.In addition, as discussed above, the observed increases in neural delay activity over parietal-occipital cortices are consistent with enhanced engagement of visual areas, not language areas, for recognizable versus unrecognizable objects (Asp et al., 2021;Brady et al., 2016;Thibeault et al., 2024).Nevertheless, many other studies have demonstrated an important role of verbal labeling in visual working memory when participants are instructed to use labels (e.g., Overkott & Souza, 2023;Souza & Skóra, 2017).Thus, it is possible that participants can use verbal strategies to support visual working memory but tend not to do so unless explicitly instructed.Because other studies have shown that labels can help the recognition of objects (e.g., Boutonnet & Lupyan, 2015;Enge et al., 2023), it appears plausible that labeling could modulate the meaningful benefit in working memory in some cases (e.g., when recognizing the stimuli without labels is difficult).

Conclusion
We reviewed recent studies on visual working memory that have found an advantage for remembering familiar and meaningful stimuli relative to simple features and abstract shapes.These findings challenge a dominant view in visual working memory research of a general "fixed" capacity limit and suggest that capacity is more flexible than previously assumed and strongly depends on what type of information is being maintained.The role of meaningfulness in working memory parallels its crucial role in long-term memory in which semantic understanding also shapes memory representations (Bartlett, 1932).Thus, we propose that future work on visual working memory should consider how incoming perceptual inputs connect to preexisting knowledge that structure working memory representations in fundamental ways, thereby having the ability to unlock additional working memory capacity.Looks at working memory and attention in more realworld settings, with a focus not purely on their capacity limits but on how they are actually used.

Transparency
Action Editor: Robert L. Goldstone Editor: Robert L. Goldstone Author Contributions T. F. Brady and V. S. Störmer contributed equally to this manuscript.All of the authors approved the final manuscript for submission.
Bates, C. J.,Lerch, R. A., Sims, C. R., & Jacobs, R. A. (2019).Adaptive allocation of human visual working memory capacity during statistical and categorical learning.Journal of Vision, 19(2), Article 11.Provides evidence for and examines a view of working memory in which we adaptively allocate limited resources on the basis of an awareness of natural image statistics.Bays, P., Schneegans, S., Ma, W., & Brady, T. F. (2024).(See References).Reviews computational models of visual working memory and the role of knowledge in visual working memory.Brady, T. F., Robinson, M. M., & Williams, J. R. (2024).Noisy and hierarchical visual memory across time scales.Nature Reviews Psychology, 3, 147-163.Reviews visual working and long-term memory and their interaction.Brady, T. F., Störmer, V. S., Shafer-Skelton, A., Williams, J. R., Chapman, A. F., & Schill, H. (2019).Scaling up visual attention and visual working memory to the real world.Psychology of Learning and Motivation, 70, 29-69.Reviews attention and working memory studies that make small steps toward being more like the real world while still being well controlled and computationally tractable.Kristjánsson, Á., & Draschkow, D. (2021).Keeping it real: Looking beyond capacity limits in visual cognition.Attention, Perception, & Psychophysics, 83, 1375-1390.