## Overview
This repository contains the resources associated with the paper *A Multimodal Symphony: Integrating Taste and Sound through Generative AI*. The study explores how generative AI can synthesize music based on taste descriptions, leveraging neuroscientific and psychological research on crossmodal correspondences. The dataset, code, and fine-tuned model are provided for reproducibility and further research.
## Repository Structure
The repository is organized into the following sections:
### 1. Model Fine-Tuning
- **Fine-Tuning Code:** The source code used to fine-tune the MusicGen model is available on [GitHub](https://github.com/matteospanio/tasty-musicgen-small).
- **Fine-Tuned Model Weights:** The trained model is hosted on [Hugging Face](https://huggingface.co/csc-unipd/tasty-musicgen-small).
#### Training Dataset
- **Dataset Generation Code & Training Data:** The scripts for creating the dataset used in the fine-tuning process, along with the dataset itself, can be found on [GitHub](https://github.com/matteospanio/taste-music-dataset).
### 2. Survey and User Study
- **Survey Implementation:** The PsyToolkit file used to conduct the online survey.
- **Survey Results:** Collected responses from participants evaluating the generated music.
- **Survey Analysis Code:** The scripts used to analyze the survey results are available on [GitHub](https://github.com/matteospanio/multimodal-symphony-survey-analysis).
## Citation
If you use this repository or any of its resources in your work, please cite:
```
@misc{spanio2025multimodalsymphonyintegratingtaste,
title={A Multimodal Symphony: Integrating Taste and Sound through Generative AI},
author={Matteo Spanio and Massimiliano Zampini and Antonio Rodà and Franco Pierucci},
year={2025},
eprint={2503.02823},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2503.02823},
}
```
For more details, refer to the paper [preprint on ArXiV](https://arxiv.org/abs/2503.02823).
