Fedopenhar: Federated multi-task transfer learning for sensor-based human activity recognition

Motion sensors integrated into wearable and mobile devices provide valuable information about the device users. Machine learning and, recently, deep learning techniques have been used to characterize sensor data. Mainly, a single task, such as the recognition of human activities, is targeted, and the data is processed centrally at a server or in the cloud. However, the same sensor data can be utilized for multiple tasks and distributed machine-learning techniques can be used without the requirement of transmitting data to a centre. In this thesis, we introduce the FedOpenHAR framework that explores Federated Transfer Learning in a Multi-Task manner for both sensor-based human activity recognition and device position identification tasks. It utilizes a transfer learning approach by training task-specific and personalized layers in a federated manner. The OpenHAR framework, which contains ten smaller datasets, is used to train the models. The challenge is obtaining robust model(s) applicable for both tasks in different datasets, which may include only some label types. Multiple experiments are carried out in the Flower federated learning environment using DeepConvLSTM, MLP and CNN architectures. Results are presented for federated and centralized versions under different parameters and restrictions. By utilizing transfer learning and training a task-specific and personalized federated model, we obtained a higher accuracy (72.4%) than a fully centralized training approach (64.5%) and a similar accuracy in a case where each client performs individual training in isolation (72.6%). However, the advantage of FedOpenHAR over individual training is that if a new client joins with a new label type (meaning a new task), it can start training from the common layer. Additionally, if a new client wants to classify a new class type in one of the existing tasks, FedOpenHAR can start training directly from task-specific layers. We also perform further experiments to evaluate the performance of FedOpenHAR under different settings.