WORKSHOP PAPER
Transporter: A 128×4 SPAD Imager with On-chip Encoder for Spiking Neural Network-based Processing
Abstract
Single-photon avalanche diodes (SPADs) are widely used today in time-resolved imaging applications, however traditional architectures rely on time-to-digital converters (TDCs) and histogram-based processing, leading to significant data transfer and processing challenges. Previous work based on recurrent neural networks has realized histogram-free processing. To further address these limitations, we propose a novel paradigm that eliminates TDCs by integrating in-sensor spike encoders. This approach enables preprocessing of photon arrival events in the sensor while significantly compressing data, reducing complexity, and maintaining real-time edge processing capabilities. A dedicated spike encoder folds multiple laser repetition periods, transforming phase-based spike trains into density-based spike trains optimized for spiking neural network processing and training via backpropagation through time. As a proof of concept, we introduce Transporter, a 128×4 SPAD sensor with a per-pixel D flip-flop ring-based spike encoder, designed for intelligent active time-resolved imaging. This work demonstrates a path toward more efficient, neuromorphic SPAD imaging systems with reduced data overhead and enhanced real-time processing.Keywords
Single-photon avalanche diodes, SPAD, Time-resolved imaging, Spike encoders, Neuromorphic SPAD imaging systems,References
[1]) Claudio Bruschini, Harald Homulle, Ivan Michel Antolovic, Samuel Burri, and Edoardo Charbon, "Single-photon avalanche diode imagers in biophotonics: review and outlook", Light: Science & Applications, 2019. https://doi.org/10.1038/s41377-019-0191-5
[2]) Peter Kapusta, Michael Wahl, and Rainer Erdmann, "Advanced photon counting", Springer Series on Fluorescence, 2015. https://doi.org/10.1007/978-3-319-15636-1
[3]) Tommaso Milanese, Jiuxuan Zhao, Brent Hearn, and Edoardo Charbon, "Histogram-less direct time-of-flight imaging based on a machine learning processor on FPGA", International Image Sensor Workshop (IISW), 2023
[4]) Yang Lin, Paul Mos, Andrei Ardelean, Claudio Bruschini, and Edoardo Charbon, "Coupling a recurrent neural network to SPAD TCSPC systems for real-time fluorescence lifetime imaging", Scientific Reports, 2024. https://doi.org/10.1038/s41598-024-52966-9
[5]) Jason K Eshraghian, Max Ward, Emre O Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D Lu, "Training spiking neural networks using lessons from deep learning", Proceedings of the IEEE, 2023. https://doi.org/10.1109/jproc.2023.3308088
[6]) Yang Lin and Edoardo Charbon, "Spiking neural networks for active time-resolved SPAD imaging", Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024. https://doi.org/10.1109/wacv57701.2024.00796