YouTube Faces With Facial Keypoints

by user1
27 February, 2022

Videos of Celebrity Faces with Facial Keypoints for each Image Frame

Tagscomputer science, internet, image data, celebrities, popular culture

YouTube Faces Dataset with Facial Keypoints

This dataset is a processed version of the YouTube Faces Dataset, that basically contained short videos of celebrities that are publicly available and were downloaded from YouTube. There are multiple videos of each celebrity (up to 6 videos per celebrity). I’ve cropped the original videos around the faces, plus kept only consecutive frames of up to 240 frames for each original video. This is done also for reasons of disk space, but mainly to make the dataset easier to use.

Additionally, for this kaggle version of the dataset I’ve extracted facial keypoints for each frame of each video using this amazing 2D and 3D Face alignment library that was recently published. please check out this video demonstrating the library. It’s performance is really amazing, and I feel I’m quite qualified to say that after manually curating many thousands of individual frames and their corresponding keypoints. I removed all videos with extremely bad keypoints labeling. The end result of my curation process is approximately 2200 videos from more than 800 unique individuals.

Context

Kaggle datasets platform and its integration with kernels is really amazing, but it’s yet to have a videos dataset (at least that I’m aware of). Videos are special in the fact that they contain rich spatial patterns (in this case images of human faces) and rich temporal patterns (in this case how the faces move in time).

I was also inspired by the Face Images with Marked Landmark Points dataset uploaded by DrGuillermo and decided to create and share a dataset that would be similar but would also add something extra.

Acknowledgements

If you use The YouTube Faces Dataset, or refer to its results, please cite the following paper:
Lior Wolf, Tal Hassner and Itay Maoz
Face Recognition in Unconstrained Videos with Matched Background Similarity.
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2011. (pdf)

if you use the 2D or 3D keypoints, or refer to its results, please cite the following paper:
Adrian Bulat and Georgios Tzimiropoulos.
How far are we from solving the 2D & 3D Face Alignment problem?
(and a dataset of 230,000 3D facial landmarks), arxiv, 2017. (pdf)

Also, I would like to thank Gil Levi for pointing out YouTube Faces to me a few years back.

Inspiration

The YouTube Faces Dataset was originally intended to be used for face recognition across videos, i.e. given two videos, are those videos of the same person or not?

I think it can be used to serve many additional goals, especially when combined with the keypoints information. For example, can we build a face movement model and predict what facial expression will come next?

This dataset can also be used to test transfer learning between other face datasets (like Face Images with Marked Landmark Points that I mentioned earlier), or even other types of faces like cat or dog faces (like here or here). Also, using the pre-trained Keras models might be useful (example kernel).

Have Fun!

Size: 17003751 KB Price: Free Author: David Beniaguev Data source: kaggle.com