Visually Impaired Gait-Based Detection Using LSTM Networks

Visual impairment limits individuals in doing daily tasks, particularly in recognizing their surroundings, which can lead to unpredictable dangers. This paper proposes a dangerous area warning system for the visually impaired to help them recognize dangerous area by utilizing action recognition. The proposed system works by classifying gait or walking patterns of a normal and visually impaired person, and when a visually impaired person is detected, the system will give a verbal warning. The proposed method utilized Mediapipe Pose Landmarker to extract skeletal features, including spatial data (object position) and temporal data (object motion). These features are then processed by an LSTM network for classification. After experimenting with six different LSTM models, results show that combining spatial and temporal data as detection features improves model performance. Additionally, models trained on datasets that underwent frame-shifting-based data augmentation demonstrate better generalization. Among all models, the multi-input LSTM trained on augmented data achieved the best performance. On the test set, the model achieved an accuracy of 0.99, a loss of 0.02, and a precision, recall, and F1 score of 0.99 for each class. The model's average execution time is 0.101 s per frame or ~10FPS.