Back to all available

Cosine similarity based hybrid attention mechanism for log anomaly detection

artificial-intelligence
computer-networks-and-communications
data-mining-and-machine-learning
natural-language-and-speech
neural-networks

Abstract

System logs contain rich information for monitoring system operation. Although existing deep learning models have achieved significant success in log anomaly detection, sequence-type log data have not yet clearly learned the local and global relationships within sequences, creating a bottleneck in the development of log anomaly detection. To address this issue, we propose a cosine similarity-based hybrid attention mechanism log anomaly detection algorithm, named cosHALog. Firstly, after log parsing, we select three keywords from every log template based on the IDF algorithm, use BERT to extract the semantic information in templates, and then apply PCA for dimensionality reduction. Subsequently, three parallel one-dimensional convolutional neural networks are employed to learn local relationships in the sequence, combined with a simple additive attention mechanism to assist CNNs in learning. Finally, inspired by cosine similarity and Knowledge Distillation, we optimize the Vaswani attention mechanism to make it a parameter-free attention mechanism and use cosine similarity to measure the relationships between sequence nodes, thereby facilitating GRU to capture global relationships. To evaluate the performance of the cosHALog model, we compare it with the other eight hybrid attention mechanisms to select the most suitable attention mechanisms for learning local and global relationships. Meanwhile, we also compare cosHALog with four benchmark models. The experimental results show that, compared with DeepLog, CNN, LogAnomaly, and LogRobust, our proposed method achieves an average improvement of 4.50%, 1.51%, 5.60%, and 3.09% in F1 score, respectively. All experiments in this paper are conducted on the HDFS and BGL datasets.

Our source code and datasets are freely available on https://github.com/Lxc2git/cosHALog.git

Ask to review this manuscript

Notes for potential reviewers

  • Volunteering is not a guarantee that you will be asked to review. There are many reasons: reviewers must be qualified, there should be no conflicts of interest, a minimum of two reviewers have already accepted an invitation, etc.
  • This is NOT OPEN peer review. The review is single-blind, and all recommendations are sent privately to the Academic Editor handling the manuscript. All reviews are published and reviewers can choose to sign their reviews.
  • What happens after volunteering? It may be a few days before you receive an invitation to review with further instructions. You will need to accept the invitation to then become an official referee for the manuscript. If you do not receive an invitation it is for one of many possible reasons as noted above.
  • PeerJ Computer Science does not judge submissions based on subjective measures such as novelty, impact or degree of advance. Effectively, reviewers are asked to comment on whether or not the submission is scientifically and technically sound and therefore deserves to join the scientific literature. Our Peer Review criteria can be found on the "Editorial Criteria" page - reviewers are specifically asked to comment on 3 broad areas: "Basic Reporting", "Experimental Design" and "Validity of the Findings".
  • Reviewers are expected to comment in a timely, professional, and constructive manner.
  • Until the article is published, reviewers must regard all information relating to the submission as strictly confidential.
  • When submitting a review, reviewers are given the option to "sign" their review (i.e. to associate their name with their comments). Otherwise, all review comments remain anonymous.
  • All reviews of published articles are published. This includes manuscript files, peer review comments, author rebuttals and revised materials.
  • Each time a decision is made by the Academic Editor, each reviewer will receive a copy of the Decision Letter (which will include the comments of all reviewers).

If you have any questions about submitting your review, please email us at [email protected].