RESET: RElational Similarity Evaluation dataseT for V3C1 Video dataset ------------------------------------------------------------------------ This repository host RESET - a RElational Similarity Evaluation dataseT, extending the publicly available V3C1 video collection. The dataset contains over 17K annotated triples of query-candidate-candidate video keyframes. RESET covers both close and distant triplets of the general V3C1 keyframes as well as two of its compact sub-domains: wedding and diving. RESET features fine-grained similarity judgements, multiple per-triplet annotations, context of the judgements as well as similarity estimations of 30 pre-trained models. As such, it can be utilized for two main purposes: to evaluate the level of concordance between human-perceived and model-induced similarities of videos and for fine-tuning visual embeddings to better comply with human-perceived similarity. **Repository content:** - RESET_data: dataset of human similarity judgements on video keyframes (consult the [data description][1] page) - RESET_preliminary: preliminary dataset of human similarity judgements utilized in [MMM23]. Note that this dataset contained only binary feedback and several contextual features as well as distances w.r.t. 8 of 30 pre-trained models were not observed. - DataAnalysis: IPython notebooks with basic analysis of collected data - DataUsage: A basic pipeline for extractors fine-tuning. Details are available form a separate [readme][2] file **V3C1 dataset:** V3C1 dataset is a first part of Vimeo Creative Commons Collection publicly available video dataset [V3C1] containing 7475 videos and over 1M stored keyframes corresponding to the individual scenes detected within the videos. In order to download the dataset (which is provided by NIST), you need to fill-in a data agreement form (available from http://www-itec.aau.at/~klschoef/VBS2019/V3C_Org.Form.txt) and send a scan to angela.ellis@nist.gov with CC to gawad@nist.gov and ks@itec.aau.at. You will be provided with a link for downloading the data. [MMM23] Patrik Vesely and Ladislav Peska. 2023. Less is More: Similarity Models for Content-based Video Retrieval. To appear in Proceedings of 29th International Conference on MultiMedia Modeling. Springer [V3C1] Fabian Berns, Luca Rossetto, Klaus Schoeffmann, Christian Beecks, and George Awad. 2019. V3C1 Dataset: An Evaluation of Content Characteristics. In ICMR’19. ACM, 334–338. [1]: ../DatasetDescription [2]: ../DatasetUsage