Extraction of Blood Volume Pulse Morphology from Facial Videos Using an LSTM-Based Temporal Encoder-Decoder Model

Abstract

This thesis introduces a method for extracting blood volume pulse (BVP) signals from facial videos, moving beyond basic heart rate estimation to capture full pulse waveforms. Our approach adapts techniques from audio signal separation and applies them to video, using a machine learning model capable of processing complex time-based data. By incorporating both regular RGB (red, green, blue) and infrared (850nm, 940nm) video, we enhance the quality of the extracted signals, making signal extraction more reliable under different lighting conditions. This method not only improves accuracy in measuring real-time heart rate but also captures unique heart patterns that could support biometric identification. Our findings show that this approach effectively recovers detailed BVP shapes from video, paving the way for advancements in health monitoring and identity verification technologies.