Optimized hybrid SVM-RF multi-biometric framework for enhanced authentication using fingerprint, iris, and face recognition

Dataset used

In this study, three distinct datasets, including fingerprint, face, and iris data, are utilized to implement the proposed methodology. The datasets are processed uniformly to ensure comprehensive evaluation and validation, accounting for variations in biometric traits.

System configuration

The computing infrastructure used in this study included MATLAB software (The MathWorks, Natick, MA, USA) for algorithm implementation and data analysis. The experiments were conducted on a system running a Windows 10 operating system, equipped with an Intel Core i7 processor and 16GB of RAM. This setup provided adequate computational power to handle the data preprocessing, feature extraction, and classification tasks for the multi-biometric datasets, ensuring efficient execution of the proposed hybrid SVM-RF model. The MATLAB environment facilitated integration of various ML and optimization algorithms.

Data pre-processing

The datasets undergo a comprehensive pre-processing phase designed to standardize inputs, enhance quality, and remove noise for effective feature extraction. The pre-processing techniques includes:

Feature extraction using the Gabor filter method

Gabor filters (GF) are employed to extract features, leveraging their capability to analyze spatial and frequency domain information (Shroff & Maheta, 2015). The convolution process with Gabor filter banks is used to identify textural patterns unique to each biometric trait. The extracted Gabor features are multi-dimensional and expressive, reducing redundancies through dimensionality reduction techniques (Muthukumar & Kavipriya, 2017). The mathematical formulation of Gabor filters ensures robust extraction of discriminative features, suitable for handling diverse biometric traits like fingerprint ridges, iris textures, and facial structures (Patro et al., 2020). The convolution process of an input picture with a GF bank primarily estimates the GF as follows:

(1) $$G_{u,v}\left(p,q\right)=I\left(p,q\right)\ast \psi \left(p,q\right).$$

These Gabor characteristics are further discernible based on a filtering action of the Gabor with size “u” and orientation (Garg, Vig & Gupta, 2016; Purohit & Ajmera, 2021). This convolution is frantically carried out for the actual and imagined parts. The initial definition of the Gabor feature representation is:

(2) $$Re{\left(O\left(p,q\right)\right)}_{m,n}=I\left(p,q\right)\ast Re\left(\psi \left(p,q,{\lambda }_m,{\theta }_n\right)\right)$$

(3) $$Im{\left(O\left(p,q\right)\right)}_{m.n}=I\left(p,q\right)\ast Im\left(\psi \left(x,y,{\lambda }_m,{\theta }_n\right)\right).$$

As a result, calculating amplitude incorporates both imaginary and genuine parts,

(4) $$O{\left(p,q\right)}_{m,n}={\left({\left(Re{\left(O\left(p,q\right)\right)}_{m,n}\right)}^2+{\left(Im{\left(O\left(p,q\right)\right)}_{m,n}\right)}^2\right)}^{\frac{1}{2}}$$when utilizing GF, the multi-dimensional and highly expressive Gabor features are crucial in dimensionality reduction. These Gabor features, once filtered, form a well-defined feature vector that encapsulates the essential characteristics of the input data. This feature vector is employed to identify the class of the test data for the classification task. This process is facilitated by a supervised learning algorithm trained on labeled data to accurately classify new, unseen data based on the extracted Gabor features. The use of Gabor characteristics not only enhances the efficiency of the feature extraction process but also significantly improves the accuracy of the classification task.

Classification using SVM and RF

The choice of SVM and RF classifiers for this study was made based on their complementary strengths in handling different aspects of classification tasks. These algorithms were selected to achieve robust and accurate classification in the multi-biometric system, considering the nature of the data and the need for efficient processing. Classifiers are central to ML tasks, where their role is to categorize data based on previously learned patterns (Ulhe et al., 2024). In this study, classification is performed by first organizing the extracted features into feature vectors, which serve as input for the classifiers. The process involves dividing the dataset into training and testing sets to evaluate the performance of the classifiers. The selection of SVM and RF was driven by their distinct characteristics, which together enhance the system’s ability to identify individuals accurately (Prakash, Krishnaveni & Dhanalakshmi, 2020). SVM is employed in this study due to its effectiveness in finding the optimal decision boundary, or hyperplane, that separates different classes in a high-dimensional feature space (Kulkarni et al., 2024). It is particularly suitable for datasets with many features, where it aims to maximize the margin between data points of different classes. The objective function of SVM can be expressed as (Santoso, Safitri & Samidi, 2024):

(5) $$min{\displaystyle \frac{1}{2}||\omega |{|}^2+C\sum _{i=1}^N{\xi }_i}$$subject to:

(6) $$y_i\left(\omega \cdot x_i+b\right)\ge 1-{\xi }_i,{\xi }_i\ge 0,{\mathrm{\forall }}_i$$where, weight vector is denoted by $\omega$, b represents the bias term, ${\xi }_i$ are slack variables that allow for some misclassification, C is the regularization parameter that controls the trade-off between maximizing the margin and minimizing classification errors, $y_i$ and $x_i$ represents the class labels and input feature vectors, respectively. In this study, a linear kernel is used, which computes the decision boundary directly based on the feature space, making it computationally efficient for classification tasks.

RF was selected to complement SVM, particularly for its ability to handle larger datasets and imbalanced data distribution effectively (Reddy et al., 2024). RF is an ensemble learning method that builds multiple decision trees during training and merges their outputs for a final prediction. The method constructs each decision tree using a randomly selected subset of features, which reduces overfitting and enhances generalization. The classification process in RF can be described as follows (Shi et al., 2024):

Each decision tree is trained using a different subset of the data. For each node, the splitting criterion is chosen to maximize the separation between classes, commonly using metrics like Gini impurity:

(7) $$G=1-\sum _{i=1}^N{p}_i^2$$where $p_i$ is the probability of class i.

For classification tasks, the final prediction is determined by the majority vote from all decision trees,

(8) $$\hat{y}=mode\left(y_1,y_2,\dots \dots ..,y_T\right)$$where $\hat{y}$ is the predicted output and T is the total number of trees.

The hybrid algorithm combines the strengths of SVM and RF, leveraging SVM’s capacity for precise decision boundaries and RF’s ability to handle large, complex datasets. The approach involves initially classifying the data using SVM to achieve a high level of accuracy, followed by using RF to further refine the predictions, especially in cases of overlapping classes or noisy data. The integration of these classifiers was chosen to improve the overall performance, as SVM efficiently handles linear separability while RF’s ensemble learning addresses non-linear relationships and reduces overfitting. This combination enhances the identification process, resulting in superior classification outcomes compared to using either technique alone.

Optimization using GA and BFO

The selected features are optimized with the help of two different algorithms, the GA and the BFO. This phase is one of the methodology’s most critical phases because it helps optimize the results compared to the previous methods. Artificial intelligence uses GA as a search tool, which uses the natural selection process. With the help of natural evolution, various beneficial solutions to optimization problems are developed (Gururaj et al., 2024). In this study, GA selects the best feature sets by considering the mutual information between the features and the output. Population (N = 72) is first created from the retrieved feature set. Using the mutual information-based fitness function, GA determines each feature’s fitness and transforms the results into a more usable set of benefits for ML models. Based on fitness levels, it chooses the members designated as parents. In order to create a child for the following generation, crossover and mutation strategies are used. This procedure is repeated till the target satisfies the requirements. The acquired optimal feature set is iteratively tested with the classifier until a low error rate and high accuracy are achieved, with BFO and GA employed in tandem (Shanmugasundaram, Mohamed & Ruhaiyem, 2017).

The BFO algorithm was employed to discover the local best value (P_best). However, the BFO algorithm exhibited slow convergence, primarily due to the fixed step size during the tumbling stage (Al-Dabbas, Azeez & Ali, 2024a). Despite this limitation, BFO demonstrated a solid capability to avoid local optima. Consequently, BFO was utilized to identify the local best value (P_best), while GA was used to determine the global best search (G_best). The combination of these methods effectively mitigated the issue of sluggish convergence, as illustrated by the following equations.

(9) $${\theta }_i\left(j+1,k,l\right)={\theta }_i\left(j,k,l\right)+C\left(i\right)\ast {\theta }_j$$

(10) $$P_{best}=f\left({\theta }_i\left(j+1,k,l\right)\right)$$where ${\theta }_i$ is the new position of the i^th position, ${\theta }_j$ is the previous position, $C\left(i\right)$ is the step size, and $P_{best}$ is the local best fitness value. This combination ensures both global exploration and local exploitation, resulting in optimized feature subsets that improve classification accuracy. Experimental results confirm the efficacy of this optimization strategy, reducing the error rate and improving the system’s precision and recall metrics.

Performance metrics and evaluation

The assessment metrics used in this study were carefully selected to evaluate the performance of the multi-biometric recognition system accurately and comprehensively. These metrics include accuracy, precision, recall, F1-score, and specificity. Each metric serves a distinct purpose in assessing various aspects of the system’s effectiveness in classification tasks, ensuring a balanced evaluation.

TP: true positive, TN: true negative, FP: false positive and FN: false negatives.

1. Accuracy: This metric measures the proportion of correctly classified instances out of the total number of samples. However, in cases where there is class imbalance, accuracy alone may not fully capture the system’s effectiveness, as it can be biased towards the majority class. Therefore, while accuracy is useful, other metrics are also considered to give a more complete evaluation.

(11) $$Accuracy={\displaystyle \frac{TN+TP}{TN+FP+TN+FN}.}$$

2. Precision: Precision assesses the accuracy of the positive predictions made by the model, indicating how many of the samples labelled as positive are truly positive. This metric is particularly important in scenarios where false positives need to be minimized, such as in biometric security systems where incorrectly granting access could lead to security breaches.

(12) $$Precision={\displaystyle \frac{TP}{TP+FP}.}$$

3. Recall: Recall, or sensitivity, measures the model’s ability to identify all relevant instances within the dataset. It indicates the proportion of actual positives that are correctly classified by the system. High recall is crucial in cases where failing to identify a positive instance (FN) could have significant consequences, such as in authentication where genuine users must not be denied access.

(13) $$Recall={\displaystyle \frac{TP}{TP+FN}.}$$

4. F1-score: The F1-score is the harmonic mean of precision and recall, providing a single metric that balances both concerns. The F1-score helps to ensure that the system performs well across different conditions, rather than optimizing for one metric at the expense of the other.

(14) $$F-Measure=2\ast {\displaystyle \frac{Precision\ast Recall}{Precision+Recall}.}$$

5. Specificity: This metric measures the proportion of TN that are correctly identified, providing an indication of the system’s ability to recognize negative instances accurately. Specificity is important in biometric systems where it is necessary to reduce the number of false alarms (FP), thus increasing the reliability of the system.

(15) $$Specificity={\displaystyle \frac{TN}{TN+FP}.}$$

The combination of these metrics ensures a comprehensive evaluation of the model’s performance across different dimensions. While accuracy gives an overall picture, precision and recall provide insights into the handling of positive cases, and the F1-score balances precision and recall. Specificity adds another layer by evaluating the correct identification of negative cases. This multi-faceted approach allows for a thorough and balanced assessment of the multi-biometric recognition system’s effectiveness.

Performance metrics for various datasets

The performance metrics of individual biometric datasets (face, finger, and iris) as well as a combined multibiometric dataset is studied as it is crucial for understanding the strengths and limitations of different models in a comprehensive manner. By examining precision, recall, accuracy, and F1-score across these datasets, the model’s effectiveness and robustness in various biometric recognition tasks can be determined. This analysis not only helps identify the most suitable model for specific biometric traits but also underscores the potential improvements achieved through the integration of multiple biometric modalities.

Figure 2 illustrates the performance metrics of three different models including SVM, RF and the proposed hybrid SVM-RF, when applied to separate face, finger, iris, and combined multibiometric datasets. Figure 2A depicts the SVM model performance for various datasets, suggesting the performance is relatively consistent around 50–60% for all face datasets. However, there is a noticeable drop in performance, particularly in accuracy, which falls below 40%, when model is applied to the finger biometric dataset. The iris dataset performs better, with scores around 60% whereas the combined dataset yields the highest scores across all metrics, with values approaching 70%. Figure 2B displays the RF model performance showing the scores are fairly even, with all metrics hovering around 60–70% for face dataset. The finger dataset shows slightly better performance compared to the SVM model, with scores above 50%. The iris dataset maintains high scores, like the face dataset. The combined dataset once again produces the best results, with all metrics scoring around 70%. Figure 2C presents the proposed hybrid SVM-RF model performance, estimating high scores around 70% for the face dataset. The finger dataset shows an improvement over the previous models, with all metrics around 60%. The iris dataset maintains strong performance, with scores above 70%. The combined dataset significantly outperforms the individual datasets, with all metrics scoring close to 100%, indicating superior accuracy, precision, recall, and F1-score. This indicates that the hybrid SVM-RF model achieves superior performance when integrating multiple biometric modalities, thus validating the effectiveness of the hybrid approach in a multibiometric system. The model assessment parameters for SVM, RF and hybrid SVM-RF models towards various datasets is illustrated in Table 2.

The results underscore the importance of using combined multibiometric datasets for enhanced performance in biometric recognition systems. The proposed hybrid SVM-RF model consistently outperforms the individual SVM and RF models, particularly on the combined dataset, showcasing its effectiveness and potential for real-world biometric applications. This study demonstrates that leveraging a hybrid approach and integrating multi-biometric modalities can significantly enhance the performance accuracy of biometric systems.

Performance metrics of the suggested hybrid SVM-RF model

The hybrid SVM-RF model has demonstrated superior performance compared to the individual SVM and RF models, this hybrid model is now employed on a multibiometric dataset for further analysis. The rationale behind using a multibiometric dataset lies in its potential to provide more accurate and robust results. By integrating multiple biometric traits such as face, finger, and iris, the multibiometric approach capitalizes on the strengths of each individual modality, thereby reducing the overall error rate and enhancing the system’s reliability. This comprehensive evaluation allows for a more nuanced understanding of the hybrid model’s capabilities, ensuring that the resulting biometric system is both precise and resilient.

As shown in Fig. 3, the training phase of the model exhibits a clear trend: as the number of epochs increases, the training accuracy improves while the error loss decreases. Figure 3A illustrates the accuracy of the models over 100 epochs. The hybrid SVM-RF model shows a consistent improvement in accuracy, starting at approximately 60% and reaching 97.56% by the 100th epoch. This suggests the effective learning and adaptation capabilities of the model. The RF and SVM model exhibit the lowest initial accuracy which has increased roughly to 54.52% and 66% by the 100th epoch, respectively. Figure 3B depicts the loss values obtained for the models across 100 epochs. The hybrid SVM-RF model shows a significant decrease in loss, starting at around 0.6 and reducing to nearly 0.02, indicating a substantial improvement in the model’s prediction accuracy. The RF model starts with the highest loss at about 0.8, which gradually decreases to around 0.64, while the SVM model begins with a loss of approximately 0.7, decreasing to about 0.49 by the 100th epoch. The results demonstrate that the hybrid SVM-RF model beats the existing models in terms of both accuracy and loss reduction when applied to combined multibiometric datasets.

The performance parameters such as precision, F1 score, and recall obtained for the models is presented in Fig. 4. Figure 4A illustrates the precision of each model over 100 epochs. Among all the models, the hybrid SVM-RF model shows a consistent improvement in precision and evaluated around 98.89% by the 100th epoch. However, the RF and SVM model shows a steady increase in precision up to 67.45% and 65.35%, respectively by the 100th epoch. Figure 4B depicts the F1 score for each model and showed a gradual increase till 67.45% and 65.35% for RF and SVM model, respectively. Although, hybrid SVM-RF model shows a significant improvement with the F1 score increasing from around 50% to 98.12% by the 100th epoch. Similarly, the recall values for the models over 100 epochs is presented in Fig. 4C suggesting the hybrid SVM-RF model exhibits a substantial increase in recall up to 97.37% till the training period ending as compared to RF and SVM models. Therefore, the proposed hybrid SVM-RF model came out to be superior compared to both the SVM and RF models in terms of precision, F1 score, and recall when applied to combined multibiometric datasets. This highlights the robustness and effectiveness of the hybrid approach in achieving higher performance metrics, thus validating its suitability for multibiometric recognition systems.

Figure 5 presents the confusion matrix for the proposed hybrid SVM-RF multi-biometric system, highlighting the classification performance across different classes. The matrix is divided into two target classes, with the actual class labels on the vertical axis and the predicted class labels on the horizontal axis. As shown in Fig. 5, the model correctly identified 446 instances of Class 0 whereas 5 instances of Class 0 were incorrectly classified as Class 1, representing false positives. Furthermore, the model accurately predicted 236 instances of Class 1 while 12 instances of Class 1 were misclassified as Class 0, indicating false negatives. The confusion matrix demonstrates that the hybrid SVM-RF model exhibits strong classification capabilities, achieving high accuracy rates and maintaining low misclassification rates across all classes. The relatively low numbers of false positives and false negatives further underscore the model’s effectiveness in distinguishing between the two classes, thereby validating its robustness and reliability for classification tasks in multibiometric systems. The high accuracy and low error rates indicate the model’s potential for practical applications where reliable biometric authentication is crucial.

Table 3 presents a detailed set of parameters obtained during implementing the proposed hybrid SVM-RF multi-biometric system. These parameters are essential for analyzing the characteristics and performance of biometric data. The measured values include a contrast of 0.3059, indicating the difference in luminance that makes objects distinguishable, and a correlation of 0.1421, reflecting the linear relationship between pixels. The energy value is 0.78623, representing the uniformity in the image’s texture. At the same time, homogeneity is measured at 0.93793, showing the closeness of the distribution of elements to the diagonal in the gray level co-occurrence matrix (GLCM). The mean intensity of the pixel values is 0.0063091, and the standard deviation is 0.089593, indicating the variation in pixel values. Entropy, which reflects the randomness in the image texture, is 3.2051. The pixel intensity’s root mean square (RMS) is 0.089803, and the variance measures the spread of pixel intensity values, is 0.0080177. The smoothness of the image texture is indicated by a value of 0.95913. The kurtosis, describing the sharpness of the distribution, is 12.2408, while skewness, reflecting the asymmetry of the intensity distribution, is 1.1048. Finally, the inverse difference moment (IDM), which indicates texture uniformity, is 1.2156. These comprehensive parameters help evaluate and optimize the biometric data, contributing significantly to the accuracy and effectiveness of the multi-biometric system for robust identification and authentication.

Figure 6 showcases the performance metrics of the proposed hybrid SVM-RF multi-biometric system, detailing its effectiveness in terms of accuracy, precision, recall, F1-score, and specificity. The system achieved an impressive accuracy of 97.56%, indicating a high level of correctness in identifying individuals across the biometric datasets. The precision rate is 98.89%, reflecting the system’s ability to identify true positives among the predicted positive results correctly. The recall, also at 97.37%, demonstrates the system’s efficiency in detecting true positives from the actual positive instances, ensuring minimal false negatives. The F1-score, a harmonic mean of precision and recall, is recorded at 98.12%, underscoring the balanced performance of the system in both precision and recall aspects. Lastly, the specificity is 97.92%, highlighting the system’s ability to identify true negatives, thereby correctly reducing false positives. These metrics collectively illustrate the robustness and reliability of the hybrid SVM-RF system in multi-biometric identification, showcasing its superior performance compared to conventional methods.

Figure 7 presents a bar graph comparing the performance metrics of the existing and proposed models. The graph demonstrates that the proposed hybrid algorithm outperforms SVM and RF across all key metrics. From the visualized data, it is evident that the proposed hybrid SVM-RF model consistently outperforms both the SVM and RF models across all metrics. Specifically, the hybrid algorithm shows an accuracy of 97.56%, superior to that obtained for SVM (54.52%) and RF (66%) models. Regarding loss, the proposed method significantly reduces the error to 0.02, compared to 0.49 for SVM and 0.61 for RF. Precision is highest for the hybrid system at 98.89%, with SVM and RF at 65.35% and 67.45%, respectively. Recall rates for the hybrid method reach 97.37%, while SVM and RF are at 68.90% and 68.42%. Lastly, the F1-score of the proposed system is 98.12%, outperforming SVM (65.35%) and RF (67.45%). This clear dominance of the hybrid SVM-RF model suggests that the integration of the strengths of both SVM and RF into a single framework result in a more powerful and reliable classification tool, making it a preferable choice for applications where high performance across multiple evaluation criteria is critical. Table 4 compares the performance metrics for existing and the proposed models, effectiveness of the hybrid SVM-RF algorithm in enhancing multi-biometric identification accuracy and reliability.

Figure 8 illustrates a comparison of the receiver operating characteristic (ROC) curves and the corresponding area under the curve (AUC) values for the existing SVM and RF and the proposed SVM-RF model. In the comparison, the hybrid SVM-RF model significantly outperforms the SVM and RF models. The AUC for the hybrid model is 0.94, indicating excellent discriminative ability. In contrast, the SVM and RF models achieve lower AUC values of 0.68 and 0.69, respectively, reflecting their comparatively weaker performance. The ROC curve for the hybrid model signifies its superior capability to correctly classify positive instances while minimizing false positives. This enhanced performance highlights the effectiveness of combining SVM and RF in a hybrid approach, which leverages the strengths of both models to achieve higher accuracy and reliability in classification tasks. Therefore, the hybrid SVM-RF model is demonstrated to be the most robust and effective among the models compared, making it a preferable choice in scenarios where precise classification is essential. The comparative analysis of the proposed model with the state-of-the-art approaches is tabulated in Table 5.

While the proposed hybrid multi-biometric system demonstrates significant improvements in accuracy and robustness, it is essential to address computational cost and multimedia data handling to ensure practical applicability. The integration of ML classifiers (SVM and RF) and optimization algorithms (GA and BFO) naturally increases computational demands, particularly during the training phase. To mitigate this, optimization processes are confined to the training stage, while real-time operations leverage pre-trained models to minimize latency. Additionally, parallel processing and dimensionality reduction techniques are employed to reduce runtime computational overhead without compromising performance. The system’s modular design supports the integration of diverse biometric modalities, including fingerprint, face, and iris data, and can accommodate multimedia data such as images and video streams. However, the current implementation focuses primarily on static images. Future enhancements will aim to expand the system’s capacity for handling high-resolution and dynamic multimedia datasets. Techniques such as batch processing, adaptive memory management, and hardware accelerations, such as GPU support, will be explored to enhance scalability and processing speed. These considerations ensure the system remains adaptable and scalable, making it a strong candidate for deployment in real-world, high-demand biometric applications. Future work will focus on refining these aspects to fully realize the system’s potential for multimedia data handling and computational efficiency.