Deep Video Prediction Network-ased Inter-Frame Coding in HEVC
- 주제(키워드) Virtual private networks , Encoding , Video coding , Kernel , Interpolation , Extrapolation , Convolution , Video coding , deep learning , convolutional neural network , video prediction network , inter-prediction , virtual reference frame , HEVC , VVC
- 주제(기타) Computer Science, Information Systems
- 주제(기타) Engineering, Electrical & Electronic
- 주제(기타) Telecommunications
- 설명문(일반) [Lee, Jung-Kyung; Kim, Nayoung; Kang, Je-Won] Ewha Womans Univ, Dept Elect & Elect Engn, Seoul 03760, South Korea; [Cho, Seunghyun] Kyungnam Univ, Dept Informat & Commun Engn, Chang Won 51767, South Korea
- 등재 SCIE, SCOPUS
- OA유형 gold
- 발행기관 IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- 발행년도 2020
- 총서유형 Journal
- URI http://www.dcollection.net/handler/ewha/000000169437
- 본문언어 영어
- Published As https://dx.doi.org/10.1109/ACCESS.2020.2993566
초록/요약
In this paper, we propose a novel Convolutional Neural Network (CNN) based video coding technique using a video prediction network (VPN) to support enhanced motion prediction in High Efficiency Video Coding (HEVC). Specifically, we design a CNN VPN to generate a virtual reference frame (VRF), which is synthesized using previously coded frames, to improve coding efficiency. The proposed VPN uses two sub-VPN architectures in cascade to predict the current frame in the same time instance. The VRF is expected to have higher temporal correlation than a conventional reference frame, and, thus it is substituted for a conventional reference frame. The proposed technique is incorporated into the HEVC inter-coding framework. Particularly, the VRF is managed in a HEVC reference picture list, so that each prediction unit (PU) can choose a better prediction signal through Rate-Distortion optimization without any additional side information. Furthermore, we modify the HEVC inter-prediction mechanisms of Advanced Motion Vector Prediction and Merge modes adaptively when the current PU uses the VRF as a reference frame. In this manner, the proposed technique can exploit the PU-wise multi-hypothesis prediction techniques in HEVC. Since the proposed VPN can perform both the video interpolation and extrapolation, it can be used for Random Access (RA) and Low Delay B (LD) coding configurations. It is shown in experimental results that the proposed technique provides & x2212;2.9 & x0025; and & x2212;5.7 & x0025; coding gains, respectively, in RA and LD coding configurations as compared to the HEVC reference software, HM 16.6 version.
more