Head gesture recognition for a semi-autonomous powerchair

The author had been involved in a project, where a semi-autonomous powerchair which has obstacle avoidance system for the paraplegic who can not use their hands properly, have been developed. In the project, the author was tasked to design a head gesture recognition system which would allow the end user to convey control commands to the chair so that he can have full autonomy. There are many different ways to communicate with a machine from using sound to using flexing of a muscle which is, by the way, this is how Stephen Hawking interacts with his chair. Ultimately, an inertial measurement unit has been chosen for the project. There are two simple reasons for this selection. One of which is that inferring intend of the user from the head orientation is very basic. For example, if the end user is leaning his head forward, that probably means "go forward", on the other hand; if he is leaning his head left, this probably means "turn left". The second reason is that the sensor needed for the task is affordable and easy to find. Design choices such as this, help to make the chair a low cost alternative to commercially available traditional powerchairs which require further engineering that is specific to the person who can not control it manually. For example, Stephen Hawking uses a specific muscle in his face to interact with his chair, which is impractical for others who can use their other limbs. One may ask that why a semi-autonomous powerchair that uses head orientation as direction command, also needs a gesture recognition system. The answer may not be very obvious. However, there is a reason behind it. As mentioned in the previous paragraph, the end user needs full autonomy. For example; the user may want to cross a road. But the chaotic nature of crossing a road with a group of people, whose behaviour could not be easily predicted, may lead the chair's obstacle avoidance system to behave chaotically too. The chair may avoid humans towards an incoming car. Therefore, the user should be able to suspend the obstacle avoidance system. Or the user may be talking to his friends, and the chair may interpret one of his nods as a "go forward" command which may cause injuries or worse. Therefore, the user should also be able to prevent chair from listening to directional commands. For reasons as such, the semi-autonomous chair needs ways of communicating the intention of the user. This is achieved via head gesture recognition in the project. There are many methods for gesture recognition in the literature. However, all wrap around several ideas which are data collection, feature extraction and decision making. All of these ideas are detailed in the chapters. However it is worth noting that most of the publications are about feature extraction. The author suspects that one of the reasons for this difference in the number of publications between the three in gesture recognition relate to the recent advancements of the computing technology where signal processing gained relative importance and it advanced relatively more rapidly due to the need created by consumers. Consequently, there exist more mathematical tools for feature extraction, compared to the number of tools available for the other two. Three different method have been studied in the thesis. These methods are named as; "Fast Fourier Transform as a Feature Extraction Method", "Head Gesture Recognition via DTW and Threshold Optimization", "Gesture Recognition via Neural Network". The first method failed at recognition task. The focus of the second method was directed towards decision making of the gesture recognition algorithm, reasons for which are discussed in detail. As a result of our studies, a method has been proposed for recognition algorithms where dynamic time warping is used for signal comparisons, to increase the recognition rate via threshold optimization. Even though the method has been developed to be used with dynamic time warping, the intuitive idea behind the method for threshold optimization which comes from geometry, can be extended to other gesture or pattern recognition problems. In the proposed method, the optimization algorithm used was genetic algorithm. But anyone who is interested in replicating or implementing the work can use any other meta-heuristic optimization algorithm to optimize the thresholds. The thresholds found through the method, which are calculated/optimized with very simple geometric shapes, have achieved, on average, success rate of 85% which means that more than eight out of ten repetitions of gestures are recognised successfully. Nonetheless, it is important to understand that this modest success rate can be easily increased towards 100% (if not 100%) by using more complex polygons for optimization. The third and the last method have achieved around 97% success rate, however this success rate means there is a possibility of switching the mode of the chair from semi-autonomous to manual without the intention of the user.