Towards only-vision autonomous wheelchair: A deep learning obstacle detection and image-based avoidance

A rapidly increasing number of people need to use a wheelchair (WC). WC users face several challenges while using the WC. Obstacle avoidance is one of these challenges. Avoidance for some users cannot be done simply using manual control. We propose that the WC should be able to achieve the avoidance automatically. Previous systems offered solutions to similar problems using a fusion of expensive depth sensors. This system uses vision-only technology, via a single camera, to achieve detection and avoidance at a cost that makes it accessible to a large number of disabled users. Our approach integrates functionalities from deep learning, computer vision and mobile robotics fields into the standard powered wheelchair (PWC). A deep-learning model is adapted using learning-transfer techniques to detect obstacles. A dataset of sidewalks has been developed to be used in the learning-transfer process. Any obstacle detected in front of the WC is avoided using a developed image- space avoidance method. The system was deployed during experiments on a Hardware setup using a real PWC. Object detection accuracy is reported as 61% mAP, which is comparable to methods implemented using standard computers. The control module generated the required motor speeds to avoid the obstacle successfully. The overall system achieves a speed of 5 FPS. We conclude that our cost-effective system can work effectively without the need for redundant and costly depth sensors. Adopting our system will increase the mobility of people with disabilities both indoors and outdoors. This opens the way for a vision-only fully autonomous WC.