Deep learning based log anomaly detection with time differences

With the ever-growing digital transformation in our lives and the new computing systems with the adaption of microservices, systems generated log records are increasing tremendously. Monitoring and evaluation of these "big" log records are real challenges due to the size and growing pace of system log generation. Most of the time, these records are not utilized efficiently for the benefit of increased system availability and reliability due to the lack of resources to process these records timely and efficiently. In this work, we propose a method for parsing and evaluating system logs based on the length of time between the occurrence events in logs and the utilization of these time periods in learning-based anomaly detection. We specifically use Seq2seq networks for anomaly detection. Results show that our method is successful at distinguishing between normal and anomaly events, even without any information about log keys.