# CoDraw-iCR
This is the data repository accompanying the following publications:
> CoDraw-iCR (v1): [Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Dataset](https://aclanthology.org/2023.eacl-main.169) (Madureira & Schlangen, EACL 2023)
> CoDraw-iCR (v2): [“Are you telling me to put glasses on the dog?” Content-Grounded Annotation of Instruction Clarification Requests in the CoDraw Dataset](https://arxiv.org/abs/2306.02377) (Madureira & Schlangen, arXiv preprint)
In visual instruction-following dialogue games, players can engage in repair mechanisms in face of an ambiguous or underspecified instruction that cannot be fully mapped to actions in the world. In this work, we annotate **Instruction Clarification Requests (iCRs)** in CoDraw, an existing dataset of interactions in a multi-modal collaborative dialogue game. We show that it contains lexically and semantically diverse iCRs being produced self-motivatedly by players deciding to clarify in order to solve the task successfully. With >8k iCRs found in 9.9k dialogues, CoDraw-iCR is a large spontaneous iCR corpus, making it a valuable resource for data-driven research on clarification in dialogue.
The definition of Instruction Clarification Requests used to classify utterances in CoDraw is:
> “This utterance indicates that the drawer is requesting further information about one or more instruction(s) previously given by the teller in order to perform an action accordingly, likely because part of the instruction was underspecified, ambiguous or not clear.”
## Directories
- ```codraw-icr-v1``` contains the first part of the annotation, as described in the EACL 2023 publication
- ```codraw-icr-v2``` contains the second part of the annotation, as described in TBA
## Attribution
This repository contains annotation on the CoDraw dataset, available at [https://github.com/facebookresearch/CoDraw](https://github.com/facebookresearch/CoDraw), which was published as:
> Jin-Hwa Kim, Nikita Kitaev, Xinlei Chen, Marcus Rohrbach, Byoung-Tak Zhang, Yuandong Tian, Dhruv Batra, and Devi Parikh. 2019. [CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication](https://aclanthology.org/P19-1651/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6495–6513, Florence, Italy. Association for Computational Linguistics.
If you use this resource, please cite our work and the original work.
```
@inproceedings{madureira-schlangen-2023-instruction,
title = "Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the {C}o{D}raw Dataset",
author = "Madureira, Brielen and
Schlangen, David",
booktitle = "Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics",
month = may,
year = "2023",
address = "Dubrovnik, Croatia",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.eacl-main.169",
pages = "2303--2319",
}
```
```
@misc{madureira2023icrv2,
title={"Are you telling me to put glasses on the dog?'' Content-Grounded Annotation of Instruction Clarification Requests in the CoDraw Dataset},
author={Brielen Madureira and David Schlangen},
year={2023},
eprint={2306.02377},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
```
@inproceedings{kim-etal-2019-codraw,
title = "{C}o{D}raw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication",
author = "Kim, Jin-Hwa and
Kitaev, Nikita and
Chen, Xinlei and
Rohrbach, Marcus and
Zhang, Byoung-Tak and
Tian, Yuandong and
Batra, Dhruv and
Parikh, Devi",
booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",
month = jul,
year = "2019",
address = "Florence, Italy",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/P19-1651",
doi = "10.18653/v1/P19-1651",
pages = "6495--6513",
}
```
## License
The annotation is licensed under the same license of the original dataset, i.e. the Creative Commons Attribution-NonCommercial 4.0 International Public License. Please check the complete terms in the```license.txt``` file.
