Current browse context:
eess.IV
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Image and Video Processing
Title: Pix2HDR -- A pixel-wise acquisition and deep learning-based synthesis approach for high-speed HDR videos
(Submitted on 24 Oct 2023 (v1), last revised 25 Apr 2024 (this version, v2))
Abstract: Accurately capturing dynamic scenes with wide-ranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera's frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern simultaneously captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds - both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.
Submission history
From: Caixin Wang [view email][v1] Tue, 24 Oct 2023 19:27:35 GMT (13884kb,D)
[v2] Thu, 25 Apr 2024 16:11:40 GMT (32586kb,D)
Link back to: arXiv, form interface, contact.