TY  - JOUR
A1  - Huang, Jiahui
A1  - Chia, Yew Ken
A1  - Yu, Samson
A1  - Yee, Kevin
A1  - Küster, Dennis
A1  - Krumhuber, Eva G.
A1  - Herremans, Dorien
A1  - Roig Noguera, Gemma
T1  - Single Image Video Prediction with Auto-Regressive GANs
T2  - Sensors
N2  - In this paper, we introduce an approach for future frames prediction based on a single input image. Our method is able to generate an entire video sequence based on the information contained in the input frame. We adopt an autoregressive approach in our generation process, i.e., the output from each time step is fed as the input to the next step. Unlike other video prediction methods that use “one shot” generation, our method is able to preserve much more details from the input image, while also capturing the critical pixel-level changes between the frames. We overcome the problem of generation quality degradation by introducing a “complementary mask” module in our architecture, and we show that this allows the model to only focus on the generation of the pixels that need to be changed, and to reuse those that should remain static from its previous frame. We empirically validate our methods against various video prediction models on the UT Dallas Dataset, and show that our approach is able to generate high quality realistic video sequences from one static input image. In addition, we also validate the robustness of our method by testing a pre-trained model on the unseen ADFES facial expression dataset. We also provide qualitative results of our model tested on a human action dataset: The Weizmann Action database.
KW  - video prediction
KW  - autoregressive GANs
KW  - emotion generation
Y1  - 2022
UR  - http://publikationen.ub.uni-frankfurt.de/frontdoor/index/index/docId/71895
UR  - https://nbn-resolving.org/urn:nbn:de:hebis:30:3-718955
SN  - 1424-8220
N1  - This project was partially supported by the Singapore Ministry of Education, Grant no. MOE2018-T2-2-161.
VL  - 22
IS  - 9, art. 3533
SP  - 1
EP  - 14
PB  - MDPI
CY  - Basel
ER  -