# Eye Gaze Drone Racing Dataset
This is the dataset accompanying the paper
```
C. Pfeiffer and D. Scaramuzza, "Human-Piloted Drone Racing: Visual Processing and Control," in IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3467-3474, April 2021, doi: 10.1109/LRA.2021.3064282.
```
## Coordinate Frames
___World___ frame coordinates are in x=Forward, y=Left, z=Up. World origin is in the center of the race track, floor is at z=0.
___Body___ frame coordinates are in x=Forward, y=Left, z=Up.
___Image___ frame coordinates are in x=right, y=down (, z=forward), origin
is the upper left image corner.
## FPV Camera
The first-person view (FPV) camera had a -30 degrees rotation about the
quadrotor y-axis (i.e., corresponding to a 30 degrees uptilt angle) and was
located at (x=0.2, y=0, z=-0.1) meters offset from drone body frame origin.
```
camera_uptilt = 30. # degrees
camera_transform = [0.2, 0., -0.1] # meters
```
FPV camera matrix, and distortion coefficients for normalized image
coordinates (0-1 range) are:
```
camera_matrix = np.array([[0.41, 0., 0.5],
[0., 0.56, 0.5],
[0., 0., 1.0]]),
dist_coefs = np.array([[0., 0., 0., 0.]])
```
## Data
Data is stored in ```/data``` in subfolders for each subject in csv files.
Data files are named as follows:
```<SUBJECT-ID>_<TRACK-NAME><RUN-NUMBER>_<DATA-TYPE>.csv```
where:
* __SUBJECT-ID__ is a 3-letter string, e.g. ```RUB```
* __TRACK-NAME__ is either ```FLAT``` or ```WAVE```
* __RUN-NUMBER__ is a digit between ```1-6```
* __DATA-TYPE__ is either ```DRONE```, ```CAMERA```, ```EVENTS```,
```GAZE```, ```AOI```, ```VECTORS```.
Data Types:
### ```DRONE```: Drone pose and control commands
* ```t```: Time in seconds.
* ```p```: Drone position in meters (world frame).
* ```q```: Drone rotation quaternion (world frame).
* ```v```: Drone velocity in m/s (world frame).
* ```a```: Drone acceleration in m/s/s (world frame).
* ```w```: Drone body rates in rad/s (body frame).
* ```throttle```: Control command for collective thrust (0-1 range).
* ```roll```, ```pitch```, ```yaw```: Control commands for body rates (-1 to
1 range).
### ```CAMERA```: Camera pose
* ```t```: Time in seconds.
* ```p```: Camera position in meters (world frame).
* ```q```: Camera rotation quaternion (world frame).
### ```GAZE```: Gaze position in screen coordinates
* ```t```: Time in seconds.
* ```gx```: Normalized gaze position along x axis (image frame, 0-1 range).
* ```gy```: Normalized gaze position along y axis (image frame, 0-1 range).
### ```VECTORS```: Velocity, Camera, Gaze, and Thrust vectors
* ```t```: Time in seconds.
* ```camera_vector```: Unit vector indicating camera focal point orientation
in 3D (world frame).
* ```gaze_vector```: Unit vector indicating gaze orientation in 3D (world
frame).
* ```thrust_vector```: Unit vector indicating thrust orientation in 3D
(world frame).
* ```velocity_vector```: Unit vector indicating velocity orientation in 3D
(world frame).
### ```AOI```: Area of interest hits and labels
* ```t```: Time in seconds.
* ```gaze_origin```: Gaze vector origin (=camera position) (world frame).
* ```gaze_vector```: Unit vector indicating gaze orientation in 3D (world
frame).
* ```aoi_label```: Label of the area of interest object currently hit by
the
gaze vector:
* ```NONE```: No object currently fixated / hit by the gaze vector.
* ```FLOOR```: Ground floor fixated / hit by the gaze vector.
* ```GATE```: Gate object fixated / hit by the gaze vector.
* ```aoi_hit```: Location of AOI hit in object coordinate frame (where
x=right, y=down)
* ```distance```: Distance in meters between gaze_origin and AOI-gaze vector
interaction point.
### ```EVENTS```: Gate passing and collision events
* ```t```: Time in seconds.
* ```label```: event labels
* ```START```: Liftoff of drone from start podium
* ```PASS```: Gate passing event with gates 0-9.
* ```COLL```: Collision event.
## Track
Track information is saved in ```/tracks```. Each file lists the gate positions, rotations, and dimensions in sequential order.
## Videos
Videos are stored in ```/videos``` in subfolders for each subject.
Note that ```FPVCAM``` videos for subjects ```RUB``` and ```REW``` are missing, due to technical problems during data acquisition.
Data files are named as follows:
```<SUBJECT-ID>_<TRACK-NAME><RUN-NUMBER>_<DATA-TYPE>.<FILE-TYPE>```
Video File types:
* ```.csv```: contains the timestamp ```t``` and frame number ```frame``` allowing to relate the video frames to data in the data folder
* ```.mp4```: the video
Video Data types:
* ```FPVCAM``` Screen capture videos of the FPV camera recorded with 60 FPS and
1080 x 960 resolution
## Citation
Please cite our work as follows:
Plain text:
```
C. Pfeiffer and D. Scaramuzza, "Human-Piloted Drone Racing: Visual Processing and Control," in IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3467-3474, April 2021, doi: 10.1109/LRA.2021.3064282.
```
BibTex:
```
@ARTICLE{9372809,
author={C. {Pfeiffer} and D. {Scaramuzza}},
journal={IEEE Robotics and Automation Letters},
title={Human-Piloted Drone Racing: Visual Processing and Control},
year={2021},
volume={6},
number={2},
pages={3467-3474},
doi={10.1109/LRA.2021.3064282}}
```
## Maintainer
```christian<dot>pfeiffer<at>uzh<dot>ch```
