This project contains data and scripts used to test MediaPipe eyebrow estimation in the context of head movement.
The project has several datasets, scripts, and graphs. It should be used the following way to recreate the results.
### Video recordings
For head rotation tests 36 videos were recorded, the names of the videos reflect the parameters:
- 3 condition for distance to camera (close, far, middle)
- 3 head movements (pitch, roll, yaw)
- 2 direction for each head rotation (up/down or right/left)
- 3 eyebrow movement (for roll and yaw only no eyebrow movement)
- no eyebrow movement (no postfix or *noER*)
- continuous eyebrow raise (*ER*)
- periodic eyebrow raise (*ERnoER*)
### Raw output of MediaPipe and OpenFace
For KRSL dataset:
`media_pipe_results_krsl.csv`, `open_face_results_krsl.csv`
Number of landmarks is very high for MediaPipe, so we only include landmarks that were used (you can see them in `canonical_face_model_uv_visualization_filtered_points.png`).
For head rotation dataset:
`media_pipe_results_head_rotation.csv`, `open_face_results_krsl_head_rotation.csv`
### Calculate distances for head rotation dataset
Use `calculate_distances.py` and `distances.py` to calculate eyebrow distances. Requires also a config file, `calculate_distances.py` has an example of the config.
`distances_head_rotation.csv` includes distances for all videos calculated from MediaPipe landmarks, OpenFace landmarks (
*inner_20_23_perp_dist39_42_3d*,
*outer_18_25_perp_dist39_42_3d*,
*inner_20_23_perp_dist36_45_3d*,
*outer_18_25_perp_dist36_45_3d*
) and OpenFace head rotation information (*pose* columns).
`head_rotation_test_distances.csv` includes only videos without eyebrow movement, `head_rotation_test_brows_distances.csv` includes videos with eyebrow movement.
Names of the distances include points they were calculated from, i.e.:
- 442perp133\_362\_3d - distance from point 442 to the line through points 133 and 362
- inner\_222\_442\_perp\_dist133\_362\_3d - mean of the inner eyebrow distances from points 222 and 442 to line through points 133 and 362
### Distances normalization
`normalize_distances.py` normalizes eyebrow distances between 0 and 1 for each distance to the camera (see arguments for usage)
`calculate_std_with_first_frame.py` calculates standard deviation of eyebrow distances from the first frame of each video (see arguments for usage)
### Eyebrow distance deviation calculation
Run the first part of `testingMediapipeClean.R`, requires `head_rotation_first_frame_std.csv` and `head_rotation_test_brows_frame_wise_0_frame_std.csv`.
### Visualization of eyebrow distances with head pitch
Run the second part of `testingMediapipeClean.R`, requires `head_rotation_test_brows_distances_normalized.csv`. The graphs can be found in the `Figures from testingMediapipeClean.R` folder, or re-created using the script.
### Visualization of face landmarks
`face_landmarks_visualization.py` visualizes face landmarks from 3 positions in the video, rotating and centering the points (see arguments for usage).
The plots are available in `Figures from face_landmarks_visualization.py` folder.
