360 degrees Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real
- 주제(키워드) 360 degrees imagery , reference-based super-resolution , latitude-aware convolution , disparity estimation , synthetic-to-real transfer learning
- 주제(기타) Computer Science, Information Systems
- 주제(기타) Engineering, Electrical & Electronic
- 주제(기타) Telecommunications
- 설명문(일반) [Kim, Hee-Jae; Kang, Je-Won; Lee, Byung-Uk] Ewha Womans Univ, Dept Elect & Elect Engn, Seoul 03760, South Korea; [Kang, Je-Won] Ewha Womans Univ, Grad Program Smart Factory, Seoul 03760, South Korea
- 등재 SCIE, SCOPUS
- OA유형 gold
- 발행기관 IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- 발행년도 2021
- 총서유형 Journal
- URI http://www.dcollection.net/handler/ewha/000000191036
- 본문언어 영어
- Published As https://doi.org/10.1109/ACCESS.2021.3128574
초록/요약
High-resolution (HR) 360 degrees images offer great advantages wherever an omnidirectional view is necessary such as in autonomous robot systems and virtual reality (VR) applications. One or more 360 degrees images in adjacent views can be utilized to significantly improve the resolution of a target 360 degrees image. In this paper, we propose an efficient reference-based 360 degrees image super-resolution (RefSR) technique to exploit a wide field of view (FoV) among adjacent 360 degrees cameras. Effective exploitation of spatial correlation is critical to achieving high quality even though the distortion inherent in the equi-rectangular projection (ERP) is a nontrivial problem. Accordingly, we develop a long-range 360 disparity estimator (DE360) to overcome a large and distorted disparity, particularly near the poles. Latitude-aware convolution (LatConv) is designed to generate more robust features to circumvent the distortion and keep the image quality. We also develop synthetic 360 degrees image datasets and introduce a synthetic-to-real learning scheme that transfers knowledge learned from synthetic 360 degrees images to a deep neural network conducting super-resolution (SR) of camera-captured images. The proposed network can learn useful features in the ERP-domain using a sufficient number of synthetic samples. The network is then adapted to camera-captured images through the transfer layer with a limited number of real-world datasets.
more