Lumisafe: Privacy-Preserving Fall Detection System using Time-of-Flight Cameras and Machine Learning

The Project

Every year, falls in bathrooms send millions of elderly people to the hospital, including my grandparents, who have fallen multiple times. Existing solutions force a painful tradeoff: RGB cameras in private spaces raise privacy concerns, while wearables often get forgotten or removed. Lumisafe provides reliable detection without harming privacy. It uses a Time-of-Flight (ToF) depth camera, which captures body movement using the reflection of IR light, not a recognizable image, to detect falls in real time. The hardest part was that no machine learning model existed for ToF sensor data. Pose estimators trained on RGB images failed completely when applied to depth maps. Therefore, I built a custom U-Net-style CNN optimized to run on a Raspberry Pi, trained on depth data to locate body joints accurately even in low-light environments. Additionally, a GRU-based classifier analyzes joint movement over a sliding 3-second window to distinguish a fall from normal motion. The result of 89.4% PCK@10% on pose estimation (a standard benchmark for joint-location accuracy), 97.7% classifier accuracy, and 95.7% precision marks the capability of the system. Furthermore, when a fall is detected, a paired mobile app receives an immediate alert. The mobile app also lets caregivers adjust sensitivity and other settings remotely. Lumisafe is the privacy-preserving and accessible AI fall detection solution for everyone. Elderly people should not have to choose between safety and privacy. In this project, AI tools, including Google Gemini and ChatGPT, were used for background research on methods of processing depth maps that Time-of-Flight sensors return. I chose to use AI tools for that part of the project because there are few resources and documentation on training models based on depth maps.

View this project on YouTube