Surgical gesture recognition with machine learning

Automatic classification and recognition of surgical activities is an important step towards providing automatic feedback in surgical training, and preventing adversarial events and medical errors in surgeries. Kinematic data recorded from surgical robots contains information about surgical gestures. Using this data, we can model and recognize surgeons' gestures automatically. In this study, the Transformer model, which has shown better performance than Recurrent Neural Networks (RNNs) with time series data, has been used to recognize surgical gestures with kinematic data. The model learned in this study is compared with the Long Short-Term Memory (LSTM) model, which is widely used in the literature. The average accuracy of the Transformer model is %77. According to the results, Transformer model is comparable to the state of the art LSTM methods, and has outperformed the LSTM model we have developed in this study as part of the benchmark, and the standard variation is lower. To our knowledge, our study is the first to use Transformer model for surgical activity recognition with kinematic data. Our experiments show the promise of Transformer Network in this domain.