Near real-time web-based mapping of live video streams from unmanned aerial vehicles

Unmanned Aerial Vehicles (UAVs) can be controlled autonomously or manually via a ground station and reduce the dependency of mission success on human capability. They can perform tasks longer than a pilot's ability to carry out while avoiding the risk of life loss. UAVs are used for the purposes of target determination, reconnaissance and observation, clash and logistics in the fields of military operations. For most of these use-cases the image/video stream taken by the digital camera payload on the UAV is very important for the success of the mission. Near-real time display of these images especially by georeferencing information brings additional advantages such as being able to monitor the operation area instantly and examine the current view which can greatly improve the situational awareness of the decision maker. They also can create a communication channel between people in distress and rescue teams in case of natural hazard. In most of the scenarios, images from multiple UAVs shall be consumed by multiple decision makers simultaneously. This creates a bottleneck on the infrastructure especially in the communication channels. In this study, a software pipeline that can display the images/streams coming from multiple UAVs on web-based 3D virtual globes as georeferenced video frames is proposed. In this way, it is aimed to transmit the images coming from operation area with the associated georeferencing information in near real-time and without the need for powerful servers by establishing a virtual network based on WebRTC technology. The proposed network topology and pipeline is implemented using CesiumJs as virtual globe and network and bandwidth utilization is tested. In addition direct georeferencing techniques and visualization of the georeferenced video is demonstrated. Besides, an application was developed that can push image and orientation information from an android phone to the same pipeline. According to the results, if high quality position and orientation information of video streams from UAVs are provided, the proposed pipeline can provide near real-time mapping of video streams on virtual globes.