Fast and accurate face recognition system using MORSCMs-LBP on embedded circuits

Chebyshev rational moments for gray level images

The Chebyshev OMs computations were illustrated by Guo et al. (2017) as:

MORSCMs was derived by Guo et al. (2017), For a color image, the multi-channel moments are a set OMs of each color channel, so the =MORSCMs= for the (R, G, and B) channels are as:

where m, n are non-negative integers for order and repetition, ἱ = $\surd -1$, the equation of weight function, Wi (r) is:

(2) $$\mathrm{W}\mathrm{i}(\mathrm{r})=\frac{1}{(1+\mathrm{r})\surd \mathrm{r}}$$

The Rm (r) is the Chebyshev rational polynomials defined as:

(3) $$\mathrm{R}\mathrm{m}(\mathrm{r})=\sum _{k=0}^m{\alpha }_{mk}\frac{1}{{(1+r)}^k}$$where ${\alpha }_{mk}$ defined as:

(4) $${\alpha }_{mK}=\frac{{(k!)}^2}{(2k)!}(\begin{array}{c}m+k-1\\ k\end{array})(\begin{array}{c}m\\ k\end{array})(-4{)}^{\kappa }$$

The orthogonality condition of Chebyshev rational moments can be expressed by multiplying polynomials $R_m\left(r\right)$, $R_n\left(r\right)$ and W (r) over the interval [0, $\mathrm{\infty })$ as:

(5) $$\underset{0}{\overset{\mathrm{\infty }}{\int }}{R}_m(r)R_n(r)W(r)dr={\delta }_{mn}\frac{A\pi }{2}$$

(6) $$\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e},{\delta }_{\mathrm{m}\mathrm{n}}=\{\begin{array}{c}1,form=n\\ 0,otherwise\end{array}$$and $\frac{{\displaystyle x_m\pi }}{2}$ is the normalization constant.

(7) $$A_m=\{\begin{array}{c}1,ifm=0\\ 2,ifm\ge 1\end{array}$$

B. Multi-channel radial substituted Chebyshev moments.

This method was introduced by Hosny (2019), where Rm (r) is replaced with R $\overline{m}$ ( $\stackrel{⌢}{r}$)

(8) $$\hat{r}=\frac{\hat{r}}{\frac{}{}}1+\hat{r}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{d}\mathrm{r}\to \frac{d\hat{r}}{\frac{}{}}(1-\hat{r}{)}^2$$and radial substituted Chebyshev moment function R $\overline{m}$ ( $\stackrel{⌢}{r}$) defined as:

(9) $$\mathrm{R}\overline{\mathrm{m}}(\hat{\mathrm{r}})={\sum }_{k=0}^m\frac{(k!{)}^2}{(2k)!}\left(\begin{array}{c}m+k-1\\ k\end{array}\right)\left(\begin{array}{c}m\\ k\end{array}\right)(-4{)}^k\frac{1}{(1+\hat{r}{)}^k}$$

for m $\ge 1$

And by using recurrence relation $\mathrm{R}\overline{\mathrm{m}}(\hat{\mathrm{r}})$ is defined as:

(10) $$\text{R}\overline{\text{m}}+1(\widehat{r})=2(2\widehat{r}-1)\text{\hspace{0.17em}R}\overline{\text{m}}(\widehat{\text{r}})-\text{R}\overline{\text{m}}-1(\widehat{r})\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}for\hspace{0.17em}m}\ge \text{1}$$

And $\overline{R}$0 ( $\stackrel{⌢}{r}$) = 1, $\overline{R}$1 ( $\stackrel{⌢}{r}$) = 2 $\stackrel{⌢}{r}$ − 1

Then substituted weight function Wi ( $\stackrel{⌢}{r}$)

(11) $$\mathrm{W}\mathrm{i}(\hat{\mathrm{r}})=\frac{1}{(1=\hat{r})\sqrt{\hat{r}}}=\frac{(1-\hat{r}{)}^{1.5}}{\frac{}{}}(\sqrt{\hat{r}}+\hat{r}{)}^2=\frac{1}{(\hat{r}-{\hat{r}}^2{)}^{0.5}}$$

So, the R $\overline{m}$ ( $\stackrel{⌢}{r}$) is orthogonal over unit disk on the interval [0, 1] as:

(12) $${\int }_0^1{\mathrm{R}}_m(\hat{r}){\mathrm{R}}_n(r)\mathrm{W}(\hat{r})d\hat{r}=(\frac{x_m\pi }{\frac{}{}}2){\delta }_{mn}$$

For a color image, the multi-channel moments are described as set OMs of each color channel (Hosny & Darwish, 2019), so the MORSCMs for the (R, G, and B) channel areas:

where the $f_r(\hat{r},\theta ),$ $f_g(\hat{r},\theta ),f_b(\hat{r},\theta )$ respectively, show intensity values of the three red, green, and blue channels.

1) Rotation invariant

The MORSCMs method is invariance to rotation, which means if the image is rotated by an angle ( $\alpha$), then:

(16) $$f_{color}^{rot}\left(\hat{r},\theta \right)=f_{colur}\left(\hat{r},\theta +\alpha \right)$$consider $\hat{\theta }$ =( $\theta +\alpha )$so $\theta =(\hat{\theta }-\alpha )$ and d $\hat{\theta }=$ d $\theta$

where color ∈ {red, green, blue}

so, depending on Eq. (17):

then,

(18) $$|x_{m,n}(f_{color}^{rot})|=|x_{m,n}(f_{color}^{rot})|$$

2) Scaling invariant

OMs values are invariant to scaling, as the calculation region can cover the whole contents of the image as explained in Guo et al. (2017), so the original image is defined on the unit circle as shown in Fig. 1.

3) Translation invariant

(19) $$\begin{array}{r}{\mathrm{A}}_c=({\mathrm{p}}_{10}(f_r)+{\mathrm{p}}_{10}(f_g)+{\mathrm{p}}_{10}(f_b))/{\mathrm{p}}_{00}\\ {\mathrm{B}}_c=({\mathrm{p}}_{01}(f_r)+{\mathrm{p}}_{01}(f_g)+{\mathrm{p}}_{01}(f_b))/{\mathrm{p}}_{00}\\ {\mathrm{p}}_{00}={\mathrm{p}}_{00}(f_r)+{\mathrm{p}}_{00}(f_g)+{\mathrm{p}}_{00}(f_b)\end{array}$$where (Ac, Bc) illustrates the color image centre point and P01( ft), P00( ft), P10( ft) represent the zero and first order of orthogonal moment for each channel sequentially. So, the MORSCMs is invariance to translation and could be calculated as:

where the ( $\overline{r},\overline{\theta }$) is the image pixel after moving it to the centroid (Ac, Bc).

Local binary pattern

LBP considered texture descriptor. It has become a widespread approach because of its simplicity and discriminative power. It was first described for texture classification by Ojala, Pietikainen & Maenpaa (2002). It can be computed using the following formula: It works by thresholding the neighbourhood of each pixel in the image, and then it converts them to binary numbers, as shown.

(21) $$LBP(x,y)={\sum }_{n=0}^{7}C(I_n-I_{thrsh})2^n$$where,

$$C(r)=\{\begin{array}{cc}1& r\ge 0\\ 0& otherwise\end{array}$$and I_thrsh = I(x, y), is the central pixel.

Then the LBPH is used and calculated as:

(22) $$\mathrm{L}\mathrm{B}\mathrm{P}\mathrm{H}(\mathrm{t})=\sum \delta \{\mathrm{t},\mathrm{L}\mathrm{B}\mathrm{P}(\mathrm{x},\mathrm{y})\},\mathrm{t}=\{0,....,7\}$$

But it is improved by Hosny & Darwish (2019) to use neighbors of different sizes to take any radius and neighbors that surround the central pixel. It is called a uniform local binary pattern, and it presents less feature vector length. It is computed as:

(23) $$\mathrm{L}\mathrm{B}{\mathrm{P}}_{\mathrm{n},\mathrm{r}}({\mathrm{I}}_{\mathrm{t}\mathrm{h}\mathrm{r}\mathrm{s}\mathrm{h}})=\{\begin{array}{c}{\sum }_{n=0}^{n-1}C(I_n-I_{thrsh})2^{n}ifU(LB{P}_{n,r})>2\\ n+1otherwise\end{array}$$

(24) $$\mathrm{L}\mathrm{B}\mathrm{P}{\mathrm{H}}_{\mathrm{n}}=\sum f\{\mathrm{I}(\mathrm{x},\mathrm{y})=\mathrm{n}\},\mathrm{n}=\{0,....,\mathrm{n}-1\}$$where r is the radius, n is the number of neighbours.

Figure 2 shows a sample of how LBP works.

System architecture

The proposed method presents an efficient face detection and recognition system. The computational power is reduced and storage is done with no effect on the efficiency and accuracy. In this work, the face recognition system comprises four basic steps, arranged as:

1. Face detection for the input image using Viola-Jones, the most widely used operator.

2. Cropping the face to facilitate feature extraction.

3. MORCMs and LBP are used to extract features.

4. Finally, the classification step on the extracted features is done using three popular standard classifiers. Figure 3 shows how the proposed algorithm works.

Viola-Jones algorithm

Viola-Jones was first proposed in 2001 by Viola (2001). There are many algorithms for face detection; detailed illustrations of these discussed approaches are available (Fasel & Movellan, 2002; Srivastava et al., 2017; Seredin, 2010). As proved by Dang & Sharma (2017), Viola-Jones is the most widely used and efficient approach for detecting faces. The Viola-Jones algorithm can detect faces in various conditions like illumination, scaling, expression, pose, and makeup. It has been used in various works (SivaKumar et al., 2021; Fatima et al., 2020; Ismael & Irina, 2020; Kirana, Wibawanto & Herwanto, 2018; Kumar, 2021). After locating the region of the face, it is cropped to minimize the region of interest. Then, the features of the face area are extracted using MORCMs and LBP. The three popular standard classifiers (KNN, SVM, and simple logistic) are employed to classify the extracted features (see Fig. 4).

Feature extraction using MORSCMs-LBP

MORSCMs method was introduced by Hosny & Darwish (2019) as a global feature descriptor for color image representation and recognition. It has proven to be effective in terms of recognition accuracy. In the proposed approach MORSCMs-LBP, we combined both methods: MORSCMs and LBP. LBP was used as a local texture features descriptor for the image. The global shape features are extracted using MORCMs. The MORSCMs method was introduced by Hosny & Darwish (2019) as a global feature descriptor for color image representation and recognition. It has proven to be effective in terms of recognition accuracy. In the proposed approach (MORSCMs-LBP), we combined both methods: MORSCMs and LBP. LBP was used as a local texture feature descriptor for the image. The global shape features are extracted using MORCMs. The two feature vectors are merged to get one feature vector, which is used for classification. See Fig. 3.

The MORSCMs-LBP implemented over three distinct challenging datasets (faces95, face96, grimace). A detailed explanation of these datasets is introduced in the next section. The most significant advantage of the proposed approach is that it requires less storage capacity and less computational cost, suitable for modern systems such as raspberry pi, compared to other methods.

System configuration

The experiments were executed on a Raspberry Pi, which is used to acquire the input face image. The used Raspberry Pi contains a Broadcom BCM2711 system on a chip with a 1.5 GHz processor and an ARM Cortex-A72. The GPU of the Raspberry Pi is the VI GPU. OpenGL ES2.0 and OpenCV libraries are used to access fast 3D cores. Several ports are present on the Raspberry Pi 4 Model B board, including two micro-HDMI ports, two USB 3.0 ports, two USB 2.0 ports, and a Gigabit Ethernet port. A total of 1 GB RAM on the B1 computer the default operating system for the Raspberry Pi (aka Raspbian) is built on top of Debian. We also tested a dual Intel (R) Corei7 7700 QH (2.80 GHz) CPU. The used operating system is Windows 10, and the IDE is Visual Studio 2019. Table 1 shows more details about the configuration and hardware specifications of the used device.

Face-95, faces-96, and grimace dataset

We used three different benchmark datasets to perform our experiments and will show some sample images of them. A sequence of 20 images was taken for each person using a fixed camera and 0.5 s between each taken image. During taking the sequence, the person moves one step toward the camera to get different scales. The images in each dataset vary in background, head scale, translation, facial emotions, and image lighting. See Table 2. The used dataset was introduced by Spacek (2009).

Experiment evaluation

In the proposed method, we used MORSCMs-LBP to extract facial features. We used the LBP with 59 features and different moment orders of five, 10, and 15. The number of extracted features is 31, 111, and 241, respectively. The LBP features vector was merged with MORSCMS's features vector to produce a single feature vector as shown in Fig. 3, so the total number of features extracted was 90, 170, and 300, respectively for each moment order. The proposed face recognition system is tested over a very challenging dataset, as illustrated in the earlier section. The accuracy of the obtained results has been confirmed by popular and standard classifiers; the k-nearest neighbor with k = 1, support vector machine, and simple logistic.

The experiments proved that our approach is more efficient than other currently proposed approaches. Hassan et al. (2021) proposed a descriptor for face recognition called (QKrMs); Pan, Li & Zhu (2015) proposed (QALMs); Pan, Li & Zhu (2020) proposed another descriptor called (QCMs); and Sukhjeet et al. proposed (QMV) approach for color face recognition (Ranade & Anand, 2021); Vinay et al. (2017) presented two methods for facial recognition based on convolution neural network called (FCNN & GCNN); Girdhar, Virmani & Kumar (2019) proposed a hybrid Fuzzy based Behavior Prognostic System for Disparate traits (FBPSDs) which uses face recognition to identify the contextual faces; Hosny, Abd Elaziz & Darwish (2021) proposed a novel color face recognition method that depends on a new family of fractional-order orthogonal functions called (FrMEMs); Virmani et al. (2019) presented face detection and recognition approach based on deep learning called (FDREnet); Ayoub et al. (2021) presented Deep learning models “Openface via PSO and introduced customized Inception-V3 model via PSO.

The recognition rates of the proposed approach (MORSCMs-LBP) against the compared methods are depicted in Tables 3, 4, 5. The obtained results showed that MORSCMs-LBP was superior over the compared approaches.

And Table 6 shows a comparison between the number of features for different moment-based approaches.

Time analysis

Many factors affect the recognition time, such as the size of the image, the dataset’s size, and the size of the feature vector. The length of the feature vector of the proposed approach was very small compared to the other compared approaches, where at moment order 15 (where the best recognition rate was obtained), it was 300, which is so small compared to the other approaches (Sajjad et al., 2020; Karami, Prasad & Shehata, 2017). It takes only 2 s to recognize the image, which is less than the time taken by the ORB, SIFT, and SURF approaches. Figure 8 shows the comparison between the recognition time of the proposed approach and the other compared methods.