Robust Spatial-Temporal Motion Coherent Priors for Multi-View Video Coding Artifact Reduction
- 주제(키워드) Video compression , Transform coding , Three-dimensional displays , Spatiotemporal phenomena , Rendering (computer graphics) , Quantization (signal) , High efficiency video coding , Multi-view video compression , video enhancement , motion vector , VVC , MPEG-immersive video , TMIV
- 주제(기타) Computer Science, Information Systems; Engineering, Electrical & Electronic; Telecommunications
- 설명문(일반) [Jeon, Gyulee; Lee, Yeonjin; Kang, Je-Won] Ewha Womans Univ, Grad Program Smart Factory, Seoul 03760, South Korea; [Lee, Jung-Kyung; Kang, Je-Won] Ewha Womans Univ, Dept Elect & Elect Engn, Seoul 03760, South Korea; [Kim, Yong-Hwan] Korea Elect Technol Inst, Seongnam Si 13509, South Korea
- 등재 SCIE, SCOPUS
- OA유형 Gold Open Access
- 발행기관 IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- 발행년도 2023
- 총서유형 Journal
- URI http://www.dcollection.net/handler/ewha/000000213625
- 본문언어 영어
- Published As https://doi.org/10.1109/ACCESS.2023.3329949
초록/요약
Multi-view video (MVV) data processed by three-dimensional (3D) video systems often suffer from compression artifacts, which can degrade the rendering quality of 3D spaces. In this paper, we focus on the task of artifact reduction in multi-view video compression using spatial and temporal motion priors. Previous MVV quality enhancement networks using a warping-and-fusion approach employed reference-to-target motion priors to exploit inter-view and temporal correlation among MVV frames. However, these motion priors were sensitive to quantization noise, and the warping accuracy was degraded, when the target frame used low-quality features in the corresponding search. To overcome these limitations, we propose a novel approach that utilizes bilateral spatial and temporal motion priors, leveraging the geometry relations of a structured MVV camera system, to exploit motion coherency. Our method involves a multi-view prior generation module that produces both unidirectional and bilateral warping vectors to exploit rich features in adjacent reference MVV frames and generate robust warping features. These features are further refined to account for unreliable alignments cross MVV frames caused by occlusions. The performance of the proposed method is evaluated in comparison with state-of-the-art MVV quality enhancement networks. Synthetic MVV dataset facilitates to train our network that produces various motion priors. Experimental results demonstrate that the proposed method significantly improves the quality of the reconstructed MVV frames in recent video coding standards such as the multi-view extension of High Efficiency Video Coding and the MPEG immersive video standard.
more