WORKSHOP PAPER
Low-Light High Dynamic Range Single Frame Image Denoising for Quanta Image Sensors
Abstract
Imaging low-light high dynamic range (HDR) scenes in a single capture is challenging for conventional sensors when exposure bracketing is not feasible due to application constraints. Advancements in sensor technology have narrowed the gap, as split-pixel and dual conversion gain (DCG) enables single-frame HDR capture and Quanta Image Sensors (QIS) allow counting individual photons at low light. However, removing shot noise from a single HDR image remains a difficult task due to the spatially varying nature of noise. To address this issue, we propose a learnable pipeline with a modular design for processing high bit-depth QIS raw images. Compared to existing algorithmic solutions, our approach offers superior reconstruction performance and greater robustness to variations in illuminance and noise.Keywords
low-light, high dynamic range, single-frame, image denoising, Quanta Image Sensors, HDR capture,References
1) Stanley H Chan, "What does a one-bit quanta image sensor offer?", IEEE Transactions on Computational Imaging, 2022. https://doi.org/10.1109/tci.2022.3202012
2) Stanley H Chan, "On the insensitivity of bit density to read noise in one-bit quanta image sensors", IEEE Sensors Journal, 2023. https://doi.org/10.1109/jsen.2023.3235493
3) Stanley H Chan, Yiheng Chi, and Preston Rahim, "SLIM: Small and learnable image signal processing module for CMOS and quanta image sensors", International Image Sensor Workshop, 2023
4) Yiheng Chi and Stanley H Chan, "Fast and robust recursive filter for image denoising", 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, 2018. https://doi.org/10.1109/icassp.2018.8461887
5) Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian, "Image denoising by sparse 3-D transform-domain collaborative filtering", IEEE Transactions on Image Processing, 2007. https://doi.org/10.1109/tip.2007.901238
6) Omar A Elgendy, Abhiram Gnanasambandam, Stanley H Chan, and Jiaju Ma, "Low-light demosaicking and denoising for small pixels using learned frequency selection", IEEE Transactions on Computational Imaging, 2021. https://doi.org/10.1109/tci.2021.3052694
7) Abhiram Gnanasambandam and Stanley H Chan, "HDR imaging with quanta image sensors: Theoretical limits and optimal reconstruction", IEEE Transactions on Computational Imaging, 2020. https://doi.org/10.1109/tci.2020.3041093
8) Jiaju Ma, Dexue Zhang, Dakota Robledo, Leo Anzagira, and Saleh Masoodian, "Ultra-high-resolution quanta image sensor with reliable photon-number-resolving and high dynamic range capabilities", Scientific Reports, 2022. https://doi.org/10.1038/s41598-022-33059-y
9) Johannes Solhusvik, Sam Hu, Robert Johansson, Zhiqiang Lin, Siguang Ma, Keiji Mabuchi, Sohei Manabe, Duli Mao, Bill Phan, Howard Rhodes, et al., "A 1392x976 2.8 µm 120db CIS with per-pixel controlled conversion gain", Proceedings of the 2017 International Image Sensor Workshop, 2017
10) Johannes Solhusvik, Jiangtao Kuang, Zhiqiang Lin, Sohei Manabe, Jeong-Ho Lyu, Howard Rhodes, et al., "A comparison of high dynamic range CIS technologies for automotive applications", Proceedings of the 2013 International Image Sensor Workshop, 2013
11) Johannes Solhusvik, T Willassent, Sindre Mikkelsen, Mathias Wilhelmsen, Sohei Manabe, Duli Mao, Zhaoyu He, Keiji Mabuchi, and TA Hasegawa, "1280x960 2.8 µm HDR CIS with DCG and split-pixel combined", Proceedings of the International Image Sensor Workshop (IISW), 2019
12) Trygve Willassen, Johannes Solhusvik, Robert Johansson, Sohrab Yaghmai, Howard Rhodes, Sohei Manabe, Duli Mao, Zhiqiang Lin, Dajiang Yang, Orkun Cellek, et al., "A 1280x1080 4.2 µm split-diode pixel HDR sensor in 110 nm BSI CMOS process", Proceedings of the International Image Sensor Workshop, 2015
13) Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang, "Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising", IEEE Transactions on Image Processing, 2017. https://doi.org/10.1109/tip.2017.2662206