FV-EffResNet: an efficient lightweight convolutional neural network for finger vein recognition

Convolution block

The core component of deep CNNs is the convolutional layer. A standard convolutional layer is designed to extract features from an input three-dimensional volume, comprising spatial dimensions (width and height) and a depth/channel dimension. As a result, each kernel in the convolutional layer is tasked with simultaneously learning mappings for cross-channel correlations and spatial correlations. Executing this operation with multiple kernels involves quadruple-component multiplication resulting in quadratic complexity (Chollet, 2017; Freeman, Roese-Koerner & Kummert, 2018), which comes with nontrivial computational cost. A common trend observed in several early architectures for finger vein recognition (Das et al., 2018; Boucherit et al., 2022; Gumusbas et al., 2019; Tamang & Kim, 2022; Yang et al. 2020; Hao, Fang & Yang, 2020) is the performance of feature extraction through multiple successive layers without any form of spatial downscaling. This approach leads to networks with a significant number of parameters and, consequently, a high computational cost. In more recent works, there has been a shift towards utilizing various types of optimized convolutional layers to give rise to lightweight finger vein recognition networks (Liu et al., 2022; Chai et al., 2022; Yang et al., 2022). Our primary contribution in this work is also in this aspect of optimization through (re)design of the convolutional blocks to enhance network performance and reduce parameter count. A detailed description of our proposed method is presented in a subsequent section.

Activation function

Activation functions are an essential part of neural networks. An activation function is a mathematical function that takes in the weighted sum of inputs to a neuron, applies a non-linear transformation to it, and outputs the results, which are then used as input for the subsequent layer in the network (Sharma, Sharma & Athaiya, 2020). The purpose of an activation function is to add non-linearity to the output neuron. Without an activation function, the output of a neuron would simply be a linear function of its input, regardless of the number of layers in the network. Adding non-linearity to the output of a neuron allows a neural network to learn more complex functions and better represent non-linear relationships in the data it is trained on. The hidden layers of a neural network employ a uniform activation function, while the output layer’s activation function is typically distinct and relies on the model’s prediction objectives. Typically, differentiable non-linear activation functions are used in the hidden layers of a neural network. This allows the backpropagation strategy to be utilized for calculating weight losses and ultimately optimizing the weights through gradient descent or other optimization techniques to minimize errors (Ponti et al., 2017). Among the most popular activation functions proposed is the rectified linear unit (ReLU) (Nair & Hinton, 2010). ReLU is the most widely adopted activation in hidden layers of deep neural networks due to its simplicity and is also effective in addressing the problem of vanishing gradients, which is a common issue in deep networks. However, in a more recent work (Ramachandran, Zoph & Le, 2017), a non-linear function that combines a linear component with a sigmoid activation named Swish was proposed. The Swish function is a smooth and continuous function, unlike the ReLU which is a piecewise linear function. It allows a small number of negative weights to be propagated through, which has been shown to have a significant improvement on the performance of the system over the conventional ReLU, especially with deeper networks or challenging datasets and can be easily incorporated into deep networks using backpropagation. The Swish function is represented mathematically by Eq. (1), with its plot shown graphically in Fig. 1. (1)${\displaystyle f\left(x\right)=x.sigmoid\left(\beta x\right).\beta denotesalearnableparameter.}$

Cyclical learning rate

Learning rate (LR) is a crucial hyperparameter in the training of neural networks, as it plays a vital role in determining the speed and efficacy of the training process. Essentially, the learning rate determines the extent to which the loss gradient will be applied to the current weights, guiding them toward a lower loss. In a prior work Smith (2017), described a technique for setting and tweaking learning rate during the training of a neural network named cyclical learning rate (CLR). The basic idea of CLR is to vary the learning rate cyclically during training, which has been shown to help a network converge faster and to achieve improved classification accuracy of a network without a need to tune and often in fewer iterations. The cycle is defined by two hyperparameters: the maximum and minimum learning rate and the step size which is the number of iterations it takes to complete a half cycle. In a complete cycle, the learning process starts with a small LR and gradually increases over a few epochs until it reaches a maximum value, then decreases back to the minimum value. This cycle is repeated until the end of the training process. The logic behind this technique is that periodic higher learning rates within each epoch help the network come out of any saddle points or local minima if it encounters one during training. The CLR technique also described three mode policies: triangular, triangular2, and exp, for specifying the pattern in which the LR varies within the maximum and minimum bounds. The maximum and minimum LR values are estimated by conducting an “LR range test”, where the network is initially run for a few epochs during which the LR is increased linearly within a large boundary. The values of when the accuracy starts to increase and when it starts to fall are taken to be the max and min LR value rates. The LR test is of huge benefit especially when dealing with a new architecture or dataset as it allows for the estimation of the initial learning rate, which practically eliminates the need to experimentally find the best values and schedule for the global learning.

Finger vein ROI extraction

Finger vein ROI extraction involves isolating and clipping out the finger region with the most abundant vein pattern and eliminating undesirable areas from the raw images. In this work, we utilized the method proposed by Lu et al. (2021), which is based on the characteristics of the original finger vein image. Their algorithm starts by searching for the finger edges based on the finger contour imaging characteristics. This method combines the original vein image with the improved edge detection operator to achieve accurate detection of the finger edges. In the case of rotations, correction angles are calculated using least square estimation and corrected by affine transformation. The joint cavity closest to the tip of the finger is then located and used as a reference. Finally, the ROI images are segmented from the original images using the detected edges and the joint cavity as boundaries. It is noteworthy to state that our method does not require any additional preprocessing or enhancements other than standard ROI extraction which is also used by all other competing models as it is essential in eliminating undesirable areas.

EffRes block

The basic idea behind the proposed EffRes block is to enhance the efficiency of feature extraction while simultaneously minimizing the parameter count. This is accomplished by explicitly decomposing the convolution process into a sequence of operations. Specifically, the block begins similarly to Iandola et al. (2016) with a pointwise convolution, aiming to distill relevant information from inter-channel relationships in the input data. Simultaneously, it functions as a squeezing unit, reducing the number of input filters to the next layer by half.

The output of the preceding operation then undergoes depthwise convolution, realized by two consecutive layers using n × 1 and 1 × m convolutions respectively, rather than the conventional n × m convolution. Notably, a rectangular dimension (3 × 7 decomposed into 3 × 1 and 1 × 7) is employed, which is found to be more suitable for finger vein data, as indicated in Chai et al. (2022). A key distinction lies in our proposed rectangular convolution block kernel, which is implemented through two consecutive layers, deviating from Chai et al. (2022), where the conventional convolution layers with a (3 × 9) kernel are used. Both layers include a nonlinear activation followed by a batch norm layer. We adopt depthwise convolution to significantly reduce the number of computations compared to standard convolution, with further decomposition into two consecutive layers providing additional computational efficiency. The latter operation alone does not drastically reduce the computational complexity. However, when combined with the pooling strategy introduced by Freeman, Roese-Koerner & Kummert (2018) (which performs pooling after the first spatial layer), significant computational savings are achieved from the second layer and subsequent layers, contributing to an overall reduction in computation cost.

Each block being converted into an EffRes block may or may not contain a subsampling operation. In the former case, we split the subsampling operation into two operations: the first will be performed (via a 1 ×2 max-pooling operation) after the first/initial depthwise convolution, while the other is realized using a strided convolution (with a 1 ×2 kernel) after the last depthwise convolution in the block. In the latter case (i.e., no subsampling), max pooling is removed, and the last convolution is realized using a pointwise convolution. We also include residual connections over each block—from before the initial 1 × 1 convolution to just after the final 1 × 1 output—to improve gradient flow. The overall architecture of the proposed EffRes block both with and without subsampling is shown in Fig. 2.

Overall architecture

The overall architecture of the proposed model is illustrated in Fig. 3. The EffRes blocks described above constitute the most critical components of the entire model. FV-EffResNet is deliberately designed to be lightweight. The architectural design follows the classical deep-narrow approach to enhance multi-level feature extraction capability and alleviate the computational burden. As depicted in Fig. 3. as the network deepens, spatial dimensions’ decrease, and channel dimensions gradually increase. The network receives extracted ROIs with dimensions of (64 × 192 × 3) as input. A sequence of the EffRes blocks progressively transforms the input, resulting in a feature map of dimensions (2 × 6). This transformed feature map is then fed to the classification block. The classification block itself consists of an adaptive average pooling layer, followed by a flatten layer, and concluded with a softmax layer which yields probability values for each subject, facilitating classification.

Finger vein identification

Finger vein identification involves the comparison of a query image with a set of training images to assign it to a class label. Rank-1 accuracy and the cumulative match characteristic curve (CMC) are commonly employed to evaluate the identification performance of a biometric model. The comparison of Rank-1 accuracy results of FV-EffResNet and other methods is presented in, Tables 6, 7, 8 and 9. Figure 5 shows the CMC curves for the comparison across each considered database, utilizing different train-test split ratios. As shown in Table 6, for the FV_USM dataset, we conducted various comparisons based on the number of sessions utilized and the train-test split ratio. Specifically, when only the first session was employed, as in Boucherit et al. (2022) and Tamang & Kim (2022), for each subject, they utilized four images for training and two for testing. Employing the same configuration with our method resulted in superior performance. When both the first and second sessions were utilized, the compared methods employed different strategies. Boucherit et al. (2022), Chai et al. (2022) utilized the first session for training and tested with second session data. Adopting a similar approach, our method achieved a significant increase compared to Boucherit et al. (2022), with an almost 15% increase in accuracy. When data from both sessions are mixed, as in Das et al. (2018), Liu et al. (2022), Ma, Wang & Hu (2023) configuration, our method achieves similar results.

As the MMCBNU_600 and SDUMLA datasets contain images obtained in a single session, we present, in both cases, comparisons based on the various train-test split ratios employed in the existing literature. As shown in Table 7, three comparisons are made when considering the MMCBNU_600, In a scenario where for each subject, training with five and testing with five, our method exhibits better performance than Gumusbas et al. (2019), showing a 2% improvement. When the training set is increased to six and testing with four, our method could not outperform (Liu et al., 2022) but still achieves superior performance compared to Gumusbas et al. (2019). Lastly, employing eight for training and two for testing, our method could not match the 100% reported by Gumusbas et al. (2019). Similarly, for SDUMLA, three split ratios are compared, as presented in Table 8, in the scenario where four images are utilized for training and two for testing, our method outperforms the compared literature. However, in the other two settings, it was marginally surpassed by the compared methods. In the case of NUPT-FV, we applied a similar approach to that of Guo et al. (2022) utilizing only the data from the first session. The dataset was divided into six training and four testing images for each subject. Our approach achieved higher accuracy despite solely utilizing the finger vein modality, in contrast to Guo et al. (2022), which incorporated both finger vein and fingerprint modalities, as presented in Table 9.

Finger vein verification

Finger vein verification involves a procedure of evaluating whether two compared images are from the same subject. If the two images are evaluated to belong to the same identity, the match is categorized as genuine; otherwise, it is considered an imposter match. In this context, the equal error rate (EER) and the receiver operating characteristic (ROC) curve are employed as evaluation metrics. ROC curves depict the performance of binary classification, illustrating the relationship between the false acceptance rate (FAR) and the genuine acceptance rate (GAR). EER represents the FAR value at the point on the ROC curve where both FAR and GAR are equal. FAR and GAR can be expressed as follows:

(2)${\displaystyle FAR=\frac{\text{Number of misclassified negative samples}}{\text{Number of negative samples}}}$ (3)${\displaystyle GAR=\frac{\text{Number of correctly classified positive samples}}{\text{Number of positive samples}}}$

As noted earlier, this study specifically concentrates on close-set evaluation, where the class labels of finger vein images in any database are known. In the context of test image I, images from different classes are designated as ‘negative samples,’ while images from the same class are categorized as ‘positive samples.’ If the predicted label of a negative sample aligns with the label of image I, it is considered a misclassified negative sample. Conversely, if the predicted label of a positive sample matches the label of image I, it is considered a correctly classified positive sample. The comparison results presented are based on the six-four train test split ratio for MMCBNU_600, and first session NUPT_FV dataset, four-two split ratio for the SDUMLA dataset, and eight-four for FV_USM two sessions combined. EER results of our method and other methods are compared in Table 10. Moreover, in Fig. 6, the ROC curves depict the performance of our network across each of the utilized databases. As presented in, Table 10, on the MMCBNU_600 dataset, although our method is outperformed by Tang et al. (2019), Huang et al. (2021) and Huang et al. (2023), it still achieved a lower EER compared to Hao, Fang & Yang (2020) and Yang et al. (2020). On the FV_USM dataset, our method achieved the third-lowest EER, despite being outperformed by Huang et al. (2021), Huang et al. (2023). Notably, our method attained the lowest EER on the NUPT_FV dataset. It is worth noting that although some of the compared methods may have outperformed our proposed method, ours still maintains superiority in terms of parameter count, being the lowest among them.

Network execution time and parameter count comparison

Tables 11 and 12 present a comparison of parameter counts and processing times between the proposed network and other deep learning models respectively. The mean execution time represents the average processing time required to evaluate a single image input from the benchmark databases under consideration. It is important to highlight that our comparison of network parameter counts includes literature explicitly stating this information with the exception of Tamang & Kim (2022) and Boucherit et al. (2022) where we implemented their algorithms based on the specifications provided. As indicated in Table 11, the proposed network boasts the lowest parameter count compared to other methods, implying a more compact and resource-efficient model. Also as shown in Table 12, although the network suggested by (Ma, Wang & Hu, 2023) is marginally faster than ours, our proposed network outperforms the one suggested by Chai et al. (2022) by a significant margin in terms of processing speed.

Discussion

This study aims to provide improvements in the design and performance of deep networks for finger vein recognition. To this end, we developed an efficient CNN network named FV-EffResNet by exploring three main network design parameters. First, our network employed a domain-specific CNN block EffRes, which has significantly fewer parameters compared to other methods in the literature, rectangular kernels are employed in the proposed method, implemented through two consecutive layers using n × 1 and 1 × m convolutions, as opposed to the conventional n × m convolution. Importantly, experiments have shown that this choice of rectangular kernels is found to be more suitable for handling finger vein data. Through the incorporation of the proposed EffRes block, we have achieved a lightweight architecture with significantly fewer parameters in comparison to those put forth in the existing literature.

Secondly, the integration of the Swish activation function specifically in the hidden layers of networks dedicated to finger vein recognition has been shown here to offer unique benefits. This is because it allows for a small number of negative weights to be propagated through, which may encourage better information flow in challenging datasets like finger vein images, making it highly suitable for processing such data. Lastly, we accomplished faster convergence by implementing the concept of cyclical learning, which not only facilitated estimating the most efficient learning rate for training the architecture but also improved the efficiency and effectiveness of the network in handling finger vein image data. These results highlight the effectiveness of our method in achieving state-of-the-art performance while minimizing computational requirements, thereby contributing to the practical implementation of finger vein recognition systems.