Q-learning algorithm inspired objective function optimization for IETF 6TiSCH networks

Technologies such as the Internet of Things (IoT) and Wireless Sensor Networks (WSN) are currently at the forefront of popular scientific research areas and these are considered to be among the promising technologies. WSNs are effectively used in a wide range of application areas such as environmental monitoring, home automation, control of agricultural production processes, health monitoring systems, military surveillance, industrial automation systems, smart grids and smart cities. However, it is necessary to develop efficient solutions to the problems encountered in WSN applications such as low latency, low energy consumption, real-time operation, high performance, reliability, minimum packet loss and reliable routing. Protocol stacks developed by the IETF (Internet Engineering Task Force) are used to ensure the connectivity of WSNs to the Internet. The IETF 6TiSCH protocol stack, with efficient, secure and scalable features, is defined as an extension of the IPv6 Internet protocol to support low power lossy networks for industrial applications. The 6TiSCH protocol has a time-slotted/channel-hopping mechanism with dynamic resource requirements. However, these resources need to be reorganized in case of frequent path changes in the network. This leads to extra energy consumption due to additional computational overhead and higher communication cost. The routing layer of 6TiSCH networks uses the IPv6 Routing Protocol for Low Power and Lossy Networks (RPL). RPL protocol makes routing decisions based on objective functions. In this paper, a new solution to the problem of frequent path changes of nodes is presented by optimizing the RPL objective function inspired by the Q-Learning algorithm. With the developed method, real-time application problems such as high end-to-end packet delivery time and high packet losses in the buffer, which are encountered in routing processes, are also improved. The proposed algorithm, a more stable network structure is provided by utilising the previous states and critical metrics in the WSNs with limited power, limited memory and limited computational capacity and reliability of the network are increased.