WORKSHOP PAPER
Understanding 3D Imaging Performance in Sensors with Angle-Sensitive Pixels
Abstract
Image sensors equipped with angle-sensitive pixels (ASPs) can extract depth images at a fraction of the cost, resources, and power requirements of current solutions, but some technical challenges remain to obtain high accuracy 3D imaging. Off-the-shelf CMOS image sensors are upgraded into ASP cameras by the addition of a transmissive diffraction mask (TDM) patterned directly on top of the sensor. This work models for the first time the depth sensitivity of ASP cameras. To demonstrate the validity of the model, the depth sensitivity is measured for diverse lenses and sensor architecture. This approach allows maximizing depth performances for a wide range of applications.Keywords
3D imaging, single-sensor, angle sensitive pixels, transmissive diffraction mask, PSF centroid,References
1) N. Kunnath, "Depth from Defocus using Angle Sensitive Pixels based on a Transmissive Diffraction Mask", M.S. thesis, McGill University, Montreal, 2018., 2018
2) G. Summy and J. Mihaychuk, "Diffraction mask design brings 3D imaging to standard CMOS image sensors", Laser Focus World, 2020., 2020
3) R. Garg, N. Wadhwa, S. Ansari, and J. T. Barron, "Learning Single Camera Depth Estimation using Dual-Pixels", ICCV, Apr. 2019., 2019. https://doi.org/10.1109/iccv.2019.00772
4) A. Punnappurath, A. Abuolaim, M. Afifi, and M. S. Brown, "Modeling Defocus-Disparity in Dual-Pixel Sensors", ICCP, 2020., 2020. https://doi.org/10.1109/iccp48838.2020.9105278
5) N. Wadhwa et al., "Synthetic depth-of-field with a single-camera mobile phone", ACM Trans Graph, vol. 37, no. 4, 2018., 2018. https://doi.org/10.1145/3197517.3201329
6) S. Xin et al., "Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image", ICCV, 2021., 2021. https://doi.org/10.1109/iccv48922.2021.00223
7) E. S. Shim et al., "All-Directional Dual Pixel Auto Focus Technology in CMOS Image Sensors", IEEE Symposium on VLSI Circuits, Digest of Technical Papers, Jun. 2021, vol. 2021 -June., 2021. https://doi.org/10.23919/vlsicircuits52068.2021.9492472
8) M. Kobayashi et al., "A Low Noise and High Sensitivity Image Sensor with Imaging and Phase-Difference Detection AF in All Pixels", IISW, 2015., 2015. https://doi.org/10.3169/mta.4.123
9) B. S. Choi et al., "Analysis of disparity information for depth extraction using CMOS image sensor with offset pixel aperture technique", Sensors (Switzerland), vol. 19, no. 3, Feb. 2019., 2019. https://doi.org/10.3390/s19030472
10) K. Fukuda, "A Compressed N×N Multi-Pixel Imaging and Cross Phase-Detection AF with N×1RGrB + 1×NGb Hetero Multi-Pixel Image Sensors", IISW, 2021., 2021. https://doi.org/10.60928/6145-mac0
11) E. W. Weisstein, "Convolution", MathWorld--A Wolfram Web Resource.
12) S. W. Smith, "The scientist and engineer's guide to digital signal processing", San Diego: California Technical Publishing, 1997, pp.400-402., 1997