Yuval Elovici

Senior Academic

Analyst intuition based Hidden Markov Model on high speed, temporal cyber security big data

T. T. Teoh, Y. Y. Nguwi, Yuval Elovici, N. M. Cheung, W. L. Ng

Hidden Markov Models (HMM) are probabilistic models that can be used for forecasting time series data. It has seen success in various domains like finance [1-5], bioinformatics [6-8], healthcare [9-11], agriculture [12-14], artificial intelligence[15-17]. However, the use of HMM in cyber security found to date is numbered. We believe the properties of HMM being predictive, probabilistic, and its ability to model different naturally occurring states form a good basis to model cyber security data. It is hence the motivation of this work to provide the initial results of our attempts to predict security attacks using HMM. A large network datasets representing cyber security attacks have been used in this work to establish an expert system. The characteristics of attacker's IP addresses can be extracted from our integrated datasets to generate statistical data. The cyber security expert provides the weight of each attribute and forms a scoring system by annotating the log history. We applied HMM to distinguish between a cyber security attack, unsure and no attack by first breaking the data into 3 cluster using Fuzzy K mean (FKM), then manually label a small data (Analyst Intuition) and finally use HMM state-based approach. By doing so, our results are very encouraging as compare to finding anomaly in a cyber security log, which generally results in creating huge amount of false detection.

Publication language English
Pages 2080-2083
Publication status Published - 21.06.2018

Keywords

Analyst Intuition
Big Data
Cyber security
Expectation Regulated
Fuzzy k-means (FKM)
Hidden Markov Model (HMM)
High Velocity
Multi-layer Perceptron (MLP)
Network Protocols
Principal Component Analysis (PCA)
Virus

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications
Hardware and Architecture
Information Systems
Information Systems and Management
Logic
Modeling and Simulation
Statistics and Probability
Access to Document
10.1109/FSKD.2017.8393092
Other files and links
Link to publication in Scopus