Single Image Deraining Using Time-Lapse Data
- 주제(키워드) Rain , Training data , Task analysis , Convolutional neural networks , Rendering (computer graphics) , Training , Feature extraction , Single image deraining , convolutional neural networks (CNNs) , time-lapse dataset , dynamic fusion module
- 주제(기타) Computer Science, Artificial Intelligence
- 주제(기타) Engineering, Electrical & Electronic
- 설명문(일반) [Cho, Jaehoon; Sohn, Kwanghoon] Yonsei Univ, Sch Elect & Elect Engn, Seoul 120749, South Korea; [Kim, Seungryong] Ecole Polytech Fed Lausanne EPFL, CH-1015 Lausanne, Switzerland; [Kim, Seungryong] Korea Univ, Dept Comp Sci & Engn, Seoul 02841, South Korea; [Min, Dongbo] Ewha Womans Univ, Dept Comp Sci & Engn, Seoul 03760, South Korea
- 관리정보기술 faculty
- 등재 SCIE, SCOPUS
- 발행기관 IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- 발행년도 2020
- 총서유형 Journal
- URI http://www.dcollection.net/handler/ewha/000000172398
- 본문언어 영어
- Published As https://dx.doi.org/10.1109/TIP.2020.3000612
초록/요약
Leveraging on recent advances in deep convolutional neural networks (CNNs), single image deraining has been studied as a learning task, achieving an outstanding performance over traditional hand-designed approaches. Current CNNs based deraining approaches adopt the supervised learning framework that uses a massive training data generated with synthetic rain streaks, having a limited generalization ability on real rainy images. To address this problem, we propose a novel learning framework for single image deraining that leverages time-lapse sequences instead of the synthetic image pairs. The deraining networks are trained using the time-lapse sequences in which both camera and scenes are static except for time-varying rain streaks. Specifically, we formulate a background consistency loss such that the deraining networks consistently generate the same derained images from the time-lapse sequences. We additionally introduce two loss functions, the structure similarity loss that encourages the derained image to be similar with an input rainy image and the directional gradient loss using the assumption that the estimated rain streaks are likely to be sparse and have dominant directions. To consider various rain conditions, we leverage a dynamic fusion module that effectively fuses multi-scale features. We also build a novel large-scale time-lapse dataset providing real world rainy images containing various rain conditions. Experiments demonstrate that the proposed method outperforms state-of-the-art techniques on synthetic and real rainy images both qualitatively and quantitatively. On the high-level vision tasks under severe rainy conditions, it has been shown that the proposed method can be utilized as a pre-preprocessing step for subsequent tasks.
more