Challenge description

We had a great meet-up on February 14 where we beta-tested the platformbanner

February 14-April 2, 2017

Challenge in Video-Forecasting: replacing missing frames in tele-conferences


Aim and scope:

We are organizing a “warm up challenge” whose objective is to test the protocol and platform of a large-scale upcoming EU challenge to predict electricity flows in the French power network (2 million Euros in prizes). This is an opportunity to get a head start and learn about spatio-temporal forecasting.

We propose a different task, but of real practical interest: predicting upcoming frames in video data. The applications may include replacing missing frames in a tele-conference when data transmission is defective.  For this benchmark, we are making available a dataset of thousands of videos of speakers facing a camera, sampled at 25 frames per second. We limit the resolution to small 32x32 pixels frames in black and white to permit obtaining results in a short time, in the context of a hackathon.
The problem is far from trivial. The baseline method consisting of “freezing” the frame and making constant prediction (persitence) is very difficult to beat. While computer vision methods may help, it will be interesting to see whether generic methods applicable to other domains can significantly outperform the baseline, including deep learning methods. Classical signal processing methods such as ARIMA models should also be serious contenders. 

One day hackathon:

We will be having a launching event at  La Paillasse, 226 Rue St Denis, 75002 Paris. This event is co-sponsored by the Paris Machine Learning Meetups. The challenge will remain open until April 2, 2017.


We are releasing a new dataset of 1000 5 second video clips (released under a CC license). The clips portray a single person facing the camera and the faces are centered (the intended scenario is that of a teleconference). Thousands of additional clips are used for testing on our server, but not released. We chose to provide grey level low resolution (32x32 pixel) images and reasonably fast sampling rate (25 fps) to deliver a realistic task, which can be completed with the computational constraints imposed by a hackathon (training and testing on a large subset of videos in a few minutes).


Each video clip has 125 frames, cut into 101+8+8+8 frames.
You are given 101 first frames and have to predict the next 8 frames.
Then the 8 frames you needed to predict are revealed. This is repeated 3 times.

For evaluation, we use RMSE on pixel intensity equally weighted for all frames.


8:00: Breakfast and registration. Group forming.
8:30 am (10 min): Isabelle Guyon (UPSud Paris-Saclay). Welcome.
8:40 am (20 min): Florin Popescu (Fraunhofer). Project presentation. Sneak peek at the data.
9:00 am (20 min): Sergio Escalera, Xavier Baro, and Julio Jacques (U. Barcelona). Sneak peek at the data and deep learning benchmarks.
9:20 am (40 min): Gael Varoquaux (INRIA). Learning on very noisy spatio-temporal series: lessons from fMRI.
10:00 am (30 min): Break. Group discussions.
10:30 am (40 min): Ivan Laptev (INRIA). Weakly supervised learning from images and video.
11:10 am (40 min): Sebastien Treguer (La Paillasse). Hands-on Python tutorial for spatio-temporal time series prediction.
12:00 pm: Lunch break (free lunch for participants having completed the prerequisites). Group forming. Sneak peek at the challenge.
13:00 pm (40 min): Antoine Marot (RTE).Forecasting power flows to ensure grid security with increasing complexity.
13:40 pm (40 min): Nadine Peyrieras (CNRS). Analyzing 3D+time microscopy images of developing organisms and reconstruct their cell lineage.
14:10 pm (40 min): Balazs Kegl (CNRS, Paris-Saclay Center for Data Science). Data challenges with modularization and code submission: lessons learned.
15:00 pm: Hackathon.
18:00 pm: Stephane Ayache and Cecile Capponi (AMU). Announcement of the winners of the day and wrap up.
19:00 pm: Adjourn.

Hackathon organizers:

Florin Popescu (Fraunhofer Institute, Berlin, Germany)
Sergio Escalera, Xavier Baro, and Julio Jacques Jr. (University of Barcelona, Spain)
Cecile Capponi, Stephane Ayache, and Isabelle Guyon (Aix Marseille University)

Commitee and local arrangements:

Isabelle Guyon, Lisheng Sun, and Diviyan Kalainathan (UPsud Paris-Saclay and ChaLearn)
Igor Carron and Frank Bardol (Paris Machine-Learning Meetups)
Sebastien Treguer (La Paillasse and ChaLearn)
Balazs Kegl (CNRS, Paris-Saclay Center for Data Science)