Outlier detection for keystroke biometric user authentication

Linear models for outlier detection

In the category of Linear models for outlier detection, three distinct methodologies were employed. Principal component analysis (PCA) utilizes a lower-dimensional space to generate principal components, assigning outlier scores based on distances from these components (Shyu et al., 2003). Mathematically, PCA involves the computation of principal components (PC_i) using the covariance matrix (Cov):

PC_i = Cov(X).

One-Class support vector machines (OCSVM) map data into a high-dimensional space, constructing a hyperplane for maximal separation between normal instances and the origin, serving as a boundary to distinguish normal from abnormal instances (Tax & Duin, 2004). The optimization problem for OCSVM involves finding the hyperplane parameters (w and b) that maximize the margin and minimize the classification error:

$min\frac{1}{2}\parallel w{\parallel }^2+\frac{1}{\nu n}{\sum }_{i=1}^n{\xi }_i-\rho$.

Linear model deviation-based outlier detection (LMDD) takes a unique approach, not relying on statistical analysis or distance-based metrics. It identifies outliers by assessing the primary properties of objects in a group, flagging those that ‘deviate’ from the group’s description, with the term ‘deviation’ characterizing outliers (Arning, Agrawal & Raghavan, 1996). While the specific mathematical formulation for LMDD is not provided here, it involves deviation-based scoring for outlier identification.

Proximity-based outlier detection models

Various approaches offer unique outlier detection methodologies within this category. Local Outlier Factor (LOF) assesses an object’s local density deviation from its neighbors (Breunig et al., 2000). It utilizes K-nearest neighbors to estimate local density and identify outliers with significantly lower density, calculated as:

$LOF\left(p\right)=\frac{{\sum }_{o\in N_k\left(p\right)}\frac{{\text{lrd}}_k\left(o\right)}{{\text{lrd}}_k\left(p\right)}}{k}$

where lrd_k(o) denotes the local reachability density of object o.

Optimizing computational costs, clustering-based local outlier factor (CBLOF) partitions data into clusters and applies LOF based on cluster size and distance to the nearest cluster (He, Xu & Deng, 2003).

KNN calculates the maximum distance of the kth neighbor as the outlier score (Angiulli & Pizzuti, 2002), given by:

$\text{Outlier Score}\left(p\right)={max}_{i=1}^k\text{distance}\left(p,{\text{neighbor}}_i\right)$

Mean KNN derives the score from the mean distance to the K-nearest neighbors as:

$\text{Outlier Score}\left(p\right)=\frac{{\sum }_{i=1}^k\text{distance}\left(p,{\text{neighbor}}_i\right)}{k}$

Assuming feature independence, the HBOS normalizes histograms, calculates the score for each feature, and flips values for outlier prioritization (Goldstein & Dengel, 2012), formulated as:

$\text{HBOS}\left(p\right)={\prod }_{i=1}^d\frac{1}{{\text{hist}}_i\left(b\left(p_i\right)\right)}$

where hist_i is the normalized histogram for feature i and (p_i) is the bin index of feature p_i. Choosing the most appropriate method depends on specific needs, considering the strengths and weaknesses of each approach.

Probabilistic models for outlier detection

This section explores three distinct probabilistic models employed for outlier detection: Angle-based outlier detection (ABOD), kernel density estimation (KDE), and Gaussian mixture model (GMM). Each model offers unique advantages and drawbacks, and their mathematical equations are presented below.

ABOD leverages the angle spectrum of data points to quantify their deviation from the majority (Kriegel, Schubert & Zimek, 2008), calculating the variance of weighted directional vectors. The angle between two points is given by $\angle \left(p_i,p_j\right)=arccos\left(\frac{p_i\cdot p_j}{\left|\right|p_i\left|\right|\cdot \left|\right|p_j\left|\right|}\right)$, where p_i and p_j are data points and ⋅ represents the dot product. The ABOD score for a point i is calculated using the following formula: ${\displaystyle \text{ABOD score for}{p}_i=\frac{\sum _{j\ne i}{w}_{ij}{\left(\angle \left(p_i,p_j\right)-\overline{\angle \left(p_i\right)}\right)}^2}{\sum _{j\ne i}{w}_{ij}}.}$

In this equation, w_ij represents the weight assigned to the angle between data points p_i and p_j, and $\overline{\angle \left(p_i\right)}$ is the mean angle for point p_i.

KDE constructs a density profile by summing smooth kernels around each data point, estimating the probability density function (Latecki, Lazarevic & Pokrajac, 2007). The kernel density estimate at point x is given by: ${\displaystyle {\hat{f}}_h\left(x\right)=\frac{1}{nh}\sum _{i=1}^{n}K\left(\frac{x-x_i}{h}\right).}$

Here, n is the number of data points, h is the bandwidth parameter, K is the kernel function, and x_i represents the data points.

GMM assumes the data originates from a mixture of k Gaussian distributions, using the expectation–maximization algorithm to estimate their parameters (Aggarwal, 2016). The mixture density model is given by: ${\displaystyle p\left(x\right)=\sum _{k=1}^K{\pi }_k\mathcal{N}\left(x|{\mu }_k,{\Sigma }_k\right).}$

In this equation, K is the number of Gaussian distributions, π_k represents the mixing coefficients, μ_k is the mean vector, and Σ_k is the covariance matrix for the kth Gaussian component.

Outlier ensembles and combination frameworks

This section explores diverse strategies employed within outlier ensembles and combination frameworks to improve outlier detection performance.

Isolation forest: This method leverages an ensemble of isolation trees, identifying outliers as data points with shorter path lengths within the trees. While effective, it can struggle with local outliers due to multiple normal instance clusters, impacting isolation efficiency (Liu, Ting & Zhou, 2008).

Feature bagging: This approach consolidates outputs from multiple outlier detection algorithms, where each detector randomly selects a subset of features. An example includes an ensemble of 10 LOF classifiers with varying n_neighbors parameters (Lazarevic & Kumar, 2005).

Locally selective combination of parallel outlier ensembles (LSCP): This method defines local regions around data points based on nearest neighbors in randomly chosen feature sub-spaces. It then selects top-performing base detectors within these regions and combines them into the final model (Zhao et al., 2019).

Isolation-based anomaly detection using nearest-neighbor ensembles (INNE): This technique partitions the data space into regions using a subsample. It then assigns isolation scores based on local distributions for both global and local anomaly detection (Bandaragoda et al., 2018).

By understanding these diverse methods and their underlying mechanisms, researchers and practitioners can select the most suitable approach for their specific outlier detection needs, achieving more robust and accurate results.

Comparison with previous works

Table 5 provides a comprehensive overview of recent keystroke dynamics-based authentication methods, focusing on their EER using the CMU keystroke dataset. Remarkably, the proposed HBOS not only exhibits competitive performance with an EER of 5.97% and an accuracy of 89.23%, but it also stands out for its distinctive characteristic of not requiring imposter data during training (× in the ‘Requires Imposter Data’ column). This sets HBOS apart from various existing methods, such as histogram gradient boosting (Ibrahim et al., 2023), deep secure (Maheshwary, Ganguly & Pudi, 2017), MLP (Andrean, Jayabalan & Thiruchelvam, 2020), autoencoder model (Patel et al., 2019), dependence clustering + KNN (Ivannikova, David & Hämäläinen, 2017), and X-means with QT (Hazan, Margalit & Rokach, 2021), all of which necessitate imposter data for training (indicated by ✓ in the same column). The results further emphasize the practical applicability and efficiency of HBOS, achieving comparable performance to methods requiring imposter data, while concurrently mitigating the challenges associated with imposter data collection. This underlines the significance of HBOS as a robust and viable solution in keystroke dynamics-based authentication.

In summary, the HBOS not only demonstrates competitive performance but also introduces a significant practical advantage by operating independently of an imposter dataset. This positions HBOS as a promising and efficient solution for real-world security applications, addressing critical challenges associated with imposter data collection. The findings underscore the potential of HBOS to enhance the reliability and applicability of keystroke dynamics authentication in diverse and challenging environments.

Limitations

The first limitation pertains to the dependency of user keystroke biometrics on the keyboard/device used and the user’s familiarity with it. In this study, we constrained the environment to a single device and password across all recorded iterations using the CMU dataset. An enhancement in this regard could involve enriching the dataset with multi-device and multi-password cases, either by collecting new data or by amalgamating multiple open-source datasets. This broader dataset would better reflect real-world scenarios, thereby improving the generalizability of the findings.

Another limitation lies in the focus solely on machine learning outlier detection approaches. While these approaches are well-established, recent studies, as evidenced in Table 1, have increasingly delved into deep learning outlier detection methods. Consequently, some of the models utilized in this study may be considered outdated in comparison. To address this limitation, future research should explore deep learning outlier detection approaches and compare their performance with the traditional machine learning models documented here. Such an investigation would entail evaluating these deep learning models using the same evaluation metrics applied in this study and considering factors such as training and inference time, which are crucial in authentication systems. By embracing the advancements in deep learning techniques, researchers can potentially uncover novel insights and improve the efficacy of keystroke biometric authentication systems.