This is the companion repository for data and scripts used in the following paper: Chen, Z., A. Mädebach, E. Gualdoni, G. Boleda. 2024. On the Use of Language and Vision Models for Cognitive Science: The Case of Naming Norms. Proceedings of CogSci 2024. Please cite the paper if you make use of it. ### Folder: Analysis Contains the scripts to reproduce the summary tables and results as presented in the paper as well as the more comprehensive set of summary tables and results in the file ** exploration-spaces-and-analyses.html** * **check_spaces.r** creates the plot and figure summarizing the spaces and their overlap * **prepare_data.r** creates the file **evaluations_prepped.Rds** which is the input of the 3 analysis scripts. * **clip_analysis1.r, clip_analysis2.r, clip_analysis3.r** create the respective summary tables and plots * **polysemy.r** gather ranks for images with polysemous names (in THINGS spaces) ### Folder: Evaluation Contains the scripts to calculate cosine similarities between image features and prototype features. * **evaluation_botup.r, evaluation_linguistic.r, evaluation_visual.r** serve to calculate the similarities for, respectively, the (visual) Bottom-up space, the linguistic Clip-spaces and the visual Clip-spaces. * **fnc_evaluation.r** contains the custom wrapper functions as called in the evalution scripts. * **gather_evaluations.py** combines the individual files created with the evalution scripts into the single file called **evaluations.json**. This file is provided in the data folder and provides the input to the analysis scripts. ### Folder: Data Contains all data necessary to run the scrips in the *evaluation* and the *analysis* folders. * **prototype_spaces.json** contains all prototype spaces involved in the analysis. * **botup_framed_image_features.pkl, clip_framed_image_features.pkl** contain feature vectors for images from picture naming databases gathered via Bottom-up Attention and Clip. * **botup_MN_object_features.pkl, clip_MN_object_features.pkl** contain feature vectors for target objects from ManyNames pictures gathered via Bottom-up Attention and Clip. * **mn_data.csv** Naming data (topname etc.) for the ManyNames images * **naming_norms_data.csv** Naming data (topname etc.) as collected from the picture naming databases for the analyses. * **evaluations.json** Contains all evaluation results, i.e., the similarity scores for image and their topname prototype. This file is generated by the scripts in the "evaluation" folder based on the feature files, the prototype spaces, and the naming data files. * **THINGS_polysemy.xlsx, polysemy_ranks.csv** Files used in the polysemy analysis ### Folder: docs Contains the article and an appendix to the article: Chen, Z., A. Mädebach, E. Gualdoni, G. Boleda. 2024. On the Use of Language and Vision Models for Cognitive Science: The Case of Naming Norms. Proceedings of CogSci 2024.