Real-time Complex Hand Gestures Recognition Based on Multi-Dimensional Features

Abstract

Gesture recognition is broadly utilized within the field of sensing. There are basically three gesture recognition methods based on computer vision, depth sensor and motion sensor. Motion sensor-based gesture recognition has few input data, fast speed, and direct access to threedimensional information of the hand. The advantages of traditional motion sensor-based gesture recognition have gradually become a current research hotspot. The essence of traditional motion sensor-based gesture recognition is a pattern recognition problem, and its accuracy depends heavily on the feature dataset extracted from prior experience. Unlike traditional pattern recognition methods, deep learning can be used to a large extent, reducing the workload of artificial heuristic extraction of features. In order to solve the problems of traditional pattern recognition, this paper proposes a real-time recognition method of multifeature gestures based on a long short-term memory network (LSTM), which is verified by sufficient experiments. The method first defines a gesture library of five (5) basic gestures and seven (7) complex gestures. Based on the kinematic characteristics of the hand posture, the angle features and displacement features are further extracted, and then short-time Fourier transform (SFTF) is used. The frequency domain features of sensor data are extracted, and the three features are input into the deep neural network LSTM to train, classify and recognize the collected gestures. At the same time, to verify the effectiveness of the proposed method, a selfdesigned handheld experience stick is collected. The gesture data of six (6) volunteers is used as an experimental data set. The collected experimental results show that the proposed recognition method has a recognition accuracy of 93.50% for basic and complex gestures. Compared with other methods, the recognition accuracy has increased by nearly 2%.

Key words: Human-computer interaction; gesture recognition; inertial sensor; motion capture.