The Institute of Electrical and Electronics Engineers (IEEE) / Computer Vision Foundation (CVF) Computer Vision and Pattern Recognition Conference (CVPR) is considered one of the top conferences in computer vision alongside the European Conference on Computer Vision (ECCV) and the International Conference on Computer Vision (ICCV). The Mobile Intelligent Photography & Imaging (MIPI) workshop will be held with CVPR 2023, aiming to develop and integrate advanced image sensors with novel algorithms into the camera systems of mobile platforms. Samsung R&D Institute China - Beijing (SRC-B) achieved excellent results at the MIPI 2023, representing breakthroughs for the company in nighttime flare removal.

SRC-B participated and won the Best Visualization Award in the Nighttime Flare Removal Challenge track in the competition, which attracted many followers and participants from the industry and academia.

Proposed FF-Former’s Overall Architecture

In removing nighttime flare, it is crucial to have a large receptive field because flare can occupy a substantial portion of an image, potentially even the entire photo. However, the conventional window-based Transformer approaches restrict the receptive field within the window, limiting its ability to capture global features. In addition, the flare can cause the dark regions to become brighter, result in a loss of contrast, and alter the frequency characteristics of the image. To address these challenges, SRC-B introduced FF-Former, which is based on Fast Fourier Convolution (FFC) and designed to extract global frequency features for enhancing nighttime flare removal.

To achieve this, we incorporated a Spatial Frequency Block (SFB) with the Swin Transformer to form the Swin Fourier Transformer Block (SFTB). This configuration enables long-dependency establishments and global feature extractions. Unlike the traditional Transformer, which relies on global self-attention, the SFB module only performs convolution operation computation, making it both effective and efficient.

In addition, we improved the loss function to keep the light source points after removing nighttime flares during the training phase. Experimental results on both real-world and synthetic benchmarks demonstrate that the proposed FF-Former significantly improves the performance of nighttime flare removal.

SRC-B’s team members