×
Dissertation Abstract
Modern existing surveillance systems are mostly reactive and possess limited capability to detect potential threats. They can identify threats that are visible, but struggle to detect weapons that are concealed (under clothing) or signs of emotional distress that could help in preventing harmful incidents. This thesis addresses that gap by proposing a real-time threat prediction system that fuses emotional detection and concealed weapon detection. The system has two main modules: an emotion recognition module and a concealed object detection module. The first module analyses facial video to classify emotion using remote photoplethysmography (rPPG). The purpose of this module is to predict whether the emotion is positive or negative (valence), and arousal, which reflects level of intensity, ranging from calm to highly active. The contactless design of this module makes it more suitable for surveillance purposes. The second module uses a millimeter-wave (mmWave) radar to detect scan weapons under clothing, allowing it to detect metallic as well as non-metallic threats even when they are not visible to the eyes. Individually, the radar-based concealed-object detector achieved 99.7% accuracy with a mean IoU of 0.70 for weapon localisation on the test set. The rPPG-based emotion classifier on the custom dataset reached 73% accuracy for valence and 75% for arousal. To perform reliable threat detection, these two modules were combined using an evidential reasoning fusion model. This fusion technique uses Dempster–Shafer evidence framework to assign weights to module's predicting values to produce a reliable threat score. Through this dynamic weighting, the system gives more priority to trustworthy input (emotion or object) at any moment, supporting it to predict potential threats more accurately. By hysteresis thresholds, the system avoids flickering problems, making it ideal for real-time threat detection.
Figure: Overview of Threat Detection System
