A generalized fuzzy clustering framework for incomplete data by integrating feature weighted and kernel learning

FCM algorithm with whole data strategy

In the WDS-FCM algorithm, a simple method is used to directly discard the samples with missing attributes. Then, the data samples in the sample set $X_P=\left\{x_{jk}\left|x_{jk}\text{is the complete attribute}\right.\right\}$ are directly clustered by FCM.

The dealing strategy of WDS-FCM algorithm will cause data samples with missing attributes to discard other complete attributes. This can result in a large amount of wasted data information. When the missing rate in the dataset is low, it has little effect on the overall dataset. With the remaining complete sample for fuzzy clustering, the calculated clustering center is not much different from the original data clustering center. Due to the absence of a large number of attributes, the clustering accuracy will be significantly impacted by an increase in the missing rate. Therefore, Hathaway & Bezdek (2001) suggest that WDS-FCM algorithm is more suitable for clustering analysis of datasets, as the proportion of missing attribute information in incomplete datasets is less than 0.25.

The WDS-FCM algorithm proceeds as follows:

(1) Split data X: the incomplete data set is separated into two sections: the complete part X_P, the missing part X_M, and X = X_P ∪ X_M. In the experiment, X_P instead of X, X_M in FCM algorithm does not participate in the calculation.

(2) Initialization: iterative convergence threshold ɛ, fuzzy parameter m, cluster number $c\left(2\le c\le \sqrt{n}\right)$, maximum number of iterations G, initial membership matrix U⁽⁰⁾.

(3) Updating the cluster center: when the algorithm performs L (L = 1, 2, …) iterations, cluster center V^(l) is updated according to U^(l−1) and the cluster center calculation formula (Eq. 2.4).

(4) Calculation of membership matrix: according to V^(l) and (Eq. 2.3), solve membership matrix U^(l).

(5) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, WDS-FCM algorithm iteration stops, the algorithm ends, the output membership U and cluster center V ; or else L = L + 1, return (3) to continue.

FCM algorithm with partial distance strategy

On the basis of WDS-FCM, PDS-FCM in terms of attributes, the attributes participate in calculating local distances as long as they exist. When the attribute is missing, the complete attribute participation is converted. The distance between missing data sample x_k and cluster center v_i is determined according to attribute ratio.

(2.12)${\displaystyle D_{ik}=\frac{s}{{\sum }_{j=1}^s{I}_{jk}}{\sum }_{j=1}^s{\left(x_{jk}-v_{ji}\right)}^2{I}_{jk}}$

Among them,

(2.13)${\displaystyle I_{jk}=\left\{\begin{array}{c}0,if{x}_{jk}\in {\tilde{X}}_M\hfill \\ 1,if{x}_{jk}\in {\tilde{X}}_P\hfill \end{array}\right.,1\le j\le s,1\le k\le n}$

The clustering center at the extremum point is as follows.

(2.14)${\displaystyle v_{ji}=\frac{{\sum }_{k=1}^n{\mu }_{ik}^m{I}_{jk}{x}_{jk}}{{\sum }_{k=1}^n{\mu }_{ik}^m{I}_{jk}},1\le j\le s,1\le i\le c}$

The membership formula is shown as (Eq. 2.15).

(2.15)${\displaystyle u_{ik}={\left[{\sum }_{t=1}^c{\left(\frac{{∥x_k-v_i∥}_2^2}{{∥x_k-v_t∥}_2^2}\right)}^{\frac{1}{m-1}}\right]}^{-1},i=1,2,\dots ,c;k=1,2,\dots ,n}$

The PDS-FCM algorithm proceeds as follows:

(1) Initialization: iterative convergence threshold ɛ, fuzzy parameter m, cluster number $c\left(2\le c\le \sqrt{n}\right)$, maximum number of iterations G, initial membership matrix U⁽⁰⁾.

(2) Updating the cluster center: when the algorithm performs L (L = 1, 2, …) iterations, the cluster center V^(l) is updated according to U^(l−1) and (Eq. 2.14).

(3) Calculating the membership matrix: according to V^()l() and (Eq. 2.15), solving membership matrix U^(l).

(4) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, PDS-FCM algorithm iteration stops, the algorithm ends, the output membership U and cluster center V; or else L = L + 1, return (3) to continue.

FCM algorithm with optimal completion strategy

The OCS-FCM algorithm assigns the lacking attributes as variables and incorporates variables into the objective function calculation of the FCM algorithm. Iterative clustering is performed with variables instead of missing attributes.

The variable membership U and the cluster center V are iteratively updated in the clustering iteration process to find the optimal value. The objective function formula established by OCS-FCM is (Eq. 2.16).

(2.16)${\displaystyle min{\sum }_{i=1}^c{\sum }_{k=1}^n{u}_{ik}^m{∥{\tilde{x}}_k-v_i∥}_2^2.}$

Using the Lagrange multiplier method to locate the extremum of objective function (Eq. 2.16), the missing attribute update formula (Eq. 2.17) is obtained.

(2.17)${\displaystyle x_{jk}=\frac{{\sum }_{i=1}^c{u}_{ik}^m{v}_{ji}}{{\sum }_{i=1}^c{u}_{ik}^m}.}$

The main steps of OCS-FCM algorithm are:

(1) Initialization: Set the fuzzy parameter m, number of clusters $c\left(2\le c\le \sqrt{n}\right)$, utmost allowed iterations G, iterative convergence threshold ɛ, the missing attribute matrix ${\tilde{X}}_M^{\left(0\right)}$, and the membership matrix U⁽⁰⁾ combined with the constraint conditions.

(2) Updating the cluster center matrix: when the algorithm performs L (L = 1, 2, …) iterations, the cluster center V^(l) is updated according to U^(l−1) and (Eq. 2.3).

(3) Calculate the membership matrix: according to V^(l), and (Eq. 2.4) solving membership matrix U^(l).

(4) Update the missing value: calculate the missing value ${\tilde{X}}_M^{\left(l\right)}$ according to the membership partition matrix U^(l) and cluster center matrix V^(l) and (Eq. 2.17).

(5) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, OCS-FCM algorithm iteration stops, and the program outputs membership U and cluster center V; or else L = L + 1, return (3) to continue.

FCM algorithm with the nearest prototype strategy

The NPS-FCM algorithm is an estimation method. In the NPS-FCM algorithm, the missing data attributes in the NPS-FCM algorithm participate in clustering with the nearest neighbor center instead. The missing data no longer remain constant after pre-population. During the iterative process, the corresponding attribute values of the clustering centers are continuously followed and adjusted. The filling method for missing attributes is as follows.

(2.18)${\displaystyle x_{jk}^{\left(l\right)}=v_{ji},D_{ik}=min\left\{D_{1k},D_{2k},\dots ,D_{ck}\right\}.}$

The NPS-FCM algorithm is based on the OCS-FCM algorithm. In the process of iteration, the missing data attribute is replaced by (Eq. 2.18), and then the clustering analysis is performed according to the implementation steps of the OCS-FCM algorithm.

The main steps of NPS-FCM algorithm are:

(1) Initialization: Set the fuzzy parameter m, the number of clusters $c\left(2\le c\le \sqrt{n}\right)$, the maximum number of iterations G, the iterative convergence threshold ɛ, the missing attribute matrix ${\tilde{X}}_M^{\left(0\right)}$, and the membership matrix U⁽⁰⁾ combined with the constraint conditions.

(2) Updating the cluster center matrix: when the algorithm performs L (L = 1, 2, …) iterations, the cluster center V^(l) is updated according to U^(l−1) and (Eq. 2.3).

(3) Calculate the membership matrix: according to V^(l), and (Eq. 2.4) solving membership matrix U^(l).

(4) Update the missing value: calculate the missing value ${\tilde{X}}_M^{\left(l\right)}$ according to the membership partition matrix U^(l) and cluster center matrix V^(l) and (Eq. 2.18).

(5) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, NPS-FCM algorithm iteration stops, and the program outputs membership U and cluster center V; or else L = L + 1, return (3) to continue.

Feature weighted FCM algorithm with OCS

In order to solve the defects of FCM in practical application, the different contributions of FCM and sample attribute vectors to classification are considered. The sample attribute weight is introduced into the objective function, which can obtain more effective clustering analysis results. This method is called the feature weighted FCM algorithm (WFCM).

In the optimization of the complete strategy, the sample data x_jk is composed of two segments, the complete attribute part x_jk (o_jk), and the missing attribute part x_jk (m_jk). Then x_jk (o_jk) ∪ x_jk (m_jk) = x_jk, x_jk (o_jk) remain unchanged in the clustering process. Assuming that u_ij represents the degree of the j sample data x_j belonging to the i cluster (the cluster center is v_i), v_ik represents the i feature of the k cluster center, w_ik represents the weight of the i feature of the k cluster center, the objective function that OCS-WFCM needs to minimize is:

(3.1)${\displaystyle \begin{array}{c}min{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }{∥x_{jk}-v_{ik}∥}^2\hfill \\ s.t.{\sum }_{i=1}^c{u}_{ij}=1,u_{ij}\in \left[0,1\right],\hfill \\ {\sum }_{k=1}^l{w}_{ik}=1,w_{ik}\in \left[0,1\right],\hfill \\ i=1,2,\dots ,c\hfill \\ j=1,2,\dots ,n\hfill \\ k=1,2,\dots ,l\hfill \\ \beta >1\hfill \end{array}}$

Furthermore, because of $x_{jk}=\left[x_{jk}\left(o_{jk}\right),x_{jk}\left(m_{jk}\right)\right]$, (Eq. 3.1) is equivalent to

(3.2)${\displaystyle min{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }\left({∥x_{jk}\left(o_{jk}\right)-v_{ik}∥}^2+{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2\right)}$

Because the complete attribute x_jk (o_jk) remains unchanged during the clustering process and is a fixed constant, the minimum value of (Eq. 3.2) can be simplified as

(3.3)${\displaystyle min{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}$

The optimal solution of (Eq. 3.3) can be further analyzed as

(3.4)${\displaystyle x_{jk}\left(m_{jk}\right)=\frac{{\sum }_{i=1}^c{u}_{ij}^m{\omega }_{ik}^{\beta }{v}_{ik}}{{\sum }_{i=1}^c{u}_{ij}^m{\omega }_{ik}^{\beta }}}$

In order to obtain the membership degree, cluster center and weight matrix, the Lagrange method is used to solve (Eq. 3.3).

If x is known, then

(3.5)${\displaystyle {\sum }_{j=1}^n{\lambda }_j\left({\sum }_{i=1}^c{u}_{ij}-1\right)=0,}$

where λ_j is the Lagrange multiplier, and λ_j is a vector composed of the Lagrange multiplier λ₁, λ₂, …λ_n.

Combining (Eq. 3.3) and (Eq. 3.5), we can get

(3.6)${\displaystyle \begin{array}{c}{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2\hfill \\ ={\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2-{\sum }_{j=1}^n{\lambda }_j\left({\sum }_{i=1}^c{u}_{ij}-1\right)\hfill \end{array}.}$

Let $Q_{ij}={\sum }_{k=1}^l{\omega }_{ik}^{\beta }{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2$, further obtain

(3.7)${\displaystyle J_{OCS-WFCM}={\sum }_{i=1}^c{\sum }_{j=1}^n{u}_{ij}^m{Q}_{ij}-{\sum }_{j=1}^n{\lambda }_j\left({\sum }_{i=1}^c{u}_{ij}-1\right).}$

Get the partial derivative of u_ij and get

(3.8)${\displaystyle \frac{\partial J_{OCS-WFCM}}{\partial u_{ij}}=m{u}_{ij}^{m-1}{Q}_{ij}-{\lambda }_j=0.}$

Therefore,

(3.9)${\displaystyle u_{ij}={\left(\frac{{\lambda }_j}{m{Q}_{ij}}\right)}^{\frac{1}{m-1}}.}$

And ${\sum }_{i=1}^c{u}_{ij}=1$ is known, combined with (Eq. 3.9), we get

(3.10)${\displaystyle \lambda j\frac{1}{m-1}={\sum }_{i=1}^c{\left(\frac{1}{m{Q}_{ij}}\right)}^{\frac{-1}{m-1}}.}$

Further obtained

(3.11)${\displaystyle \begin{array}{c}{u}_{ij}=\frac{{\sum }_{r=1}^c{\left(\frac{1}{m{Q}_{rj}}\right)}^{\frac{-1}{m-1}}}{m{Q}_{ij}^{\frac{1}{m-1}}}={\left({\sum }_{i=1}^c\frac{Q_{ij}}{Q_{rj}}\right)}^{\frac{1}{1-m}}\hfill \\ ={\left({\sum }_{i=1}^c\frac{{\sum }_{k=1}^l{\omega }_{ik}^{\beta }{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sum }_{k=1}^l{\omega }_{rk}^{\beta }{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}\right)}^{\frac{1}{1-m}}\hfill \\ r=1,2,\dots ,c\hfill \end{array}.}$

Similarly, one can obtain

(3.12)${\displaystyle {\omega }_{ik}={\left({\sum }_{t=1}^l\frac{{\sum }_{j=1}^n{u}_{ij}^m\cdot {∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sum }_{j=1}^n{u}_{ij}^m\cdot {∥x_{jt}\left(m_{jt}\right)-v_{ik}∥}^2}\right)}^{\frac{1}{1-\beta }}.}$

Next, take the partial derivative of v_ik in Eq. 3.3 to get

(3.13)${\displaystyle \frac{\partial J_{OCS-WFCM}}{\partial v_{ik}}=-2{\sum }_{j=1}^n{u}_{ij}^m{\omega }_{ik}^{\beta }\cdot \left(x_{jk}\left(m_{jk}\right)-v_{ik}\right)=0.}$

Further obtained

(3.14)${\displaystyle v_{ik}=\frac{{\sum }_{j=1}^n{u}_{ij}^m{\omega }_{ik}^{\beta }{x}_{jk}\left(m_{jk}\right)}{{\sum }_{j=1}^n{u}_{ij}^m{\omega }_{ik}^{\beta }}.}$

It is observed by (Eq. 3.14) that when ${\omega }_{ik}^{\beta }=0$, there is v_ik = 0. The formula for v_ik is

(3.15)${\displaystyle v_{ik}=\left\{\begin{array}{c}0,if{\omega }_{ik}^{\beta }=0\hfill \\ \frac{{\sum }_{j=1}^n{u}_{ij}^m{x}_{jk}\left(m_{jk}\right)}{{\sum }_{j=1}^n{u}_{ij}^m},if{\omega }_{ik}^{\beta }\ne 0\hfill \end{array}\right.}$

The main steps of OCS-WFCM algorithm are:

(1) Initialization: Set the fuzzy parameter m, number of clusters $c\left(2\le c\le \sqrt{n}\right)$, utmost allowed iterations G, iterative convergence threshold ɛ, the missing attribute matrix ${\tilde{X}}_M^{\left(0\right)}$, and the membership matrix U⁽⁰⁾ combined with the constraint conditions.

(2) Updating the cluster center matrix: when the algorithm performs L (L = 1, 2, …) iterations, cluster center V^(l) is updated according to U^(l−1) and (Eq. 3.15).

(3) Calculate the membership matrix: according to V^(l), and (Eq. 3.11) solving membership matrix U^(l).

(4) Calculate the weight matrix: according to V^(l), and (Eq. 3.12) to solve the weight matrix.

(5) Update the missing value: calculate the missing value ${\tilde{X}}_M^{\left(l\right)}$ according to the membership partition matrix U^(l) and cluster center matrix V^(l) and (Eq. 3.4).

(6) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, OCS-WFCM algorithm iteration stops, and the program outputs membership U and cluster center V; or else L = L + 1, return (3) to continue.

Feature weighted FCM algorithm with NPS

In the interpolation of NPS-WFCM, the sample data x_jk is also divided into two parts, the complete attribute part x_jk (o_jk), and the missing attribute part x_jk (m_jk). Then, x_jk (o_jk) ∪ x_jk (m_jk) = x_jk, x_jk (o_jk) remain unchanged in the clustering process. The filling method of missing attributes in NPS-WFCM is as follows.

(3.16)${\displaystyle x_{jk}\left(o_{jk}\right)=v_{ik}=\left\{\begin{array}{c}0,if{\omega }_{ik}^{\beta }=0\hfill \\ \frac{{\sum }_{j=1}^n{u}_{ij}^m{x}_{jk}}{{\sum }_{j=1}^n{u}_{ij}^m},if{\omega }_{ik}^{\beta }\ne 0\hfill \end{array}\right.,D_{ij}=min\left\{D_{1j},D_{2j},\dots ,D_{cj}\right\}.}$

Similar to OCS-WFCM, only (Eq. 3.15) needs to be replaced with (Eq. 3.16) when updating the missing attributes.

The main steps of NPS-WFCM algorithm are:

(1) Initialization: Set the fuzzy parameter m, number of clusters $c\left(2\le c\le \sqrt{n}\right)$, utmost allowed iterations G, iterative convergence threshold ɛ, the missing attribute matrix ${\tilde{X}}_M^{\left(0\right)}$, and the membership matrix U⁽⁰⁾ combined with the constraint conditions.

(2) Updating the cluster center matrix: when the algorithm performs L (L = 1, 2, …) iterations, cluster center V^(l) is updated according to U^(l−1) and (Eq. 3.15).

(3) Calculate the membership matrix: according to V^(l), and (Eq. 3.11) solving membership matrix U^(l).

(4) Calculate the weight matrix: according to V^l, and (Eq. 3.12) to solve the weight matrix.

(5) Update the missing value attribute: calculate the missing value ${\tilde{X}}_M^{\left(l\right)}$ according to the membership partition matrix U^(l) and cluster center matrix V^(l) and (Eq. 3.16).

(6) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, NPS-WFCM algorithm iteration stops, and the program outputs membership U and cluster center V; or else L = L + 1, return (3) to continue.

Feature weighted kernel FCM clustering with OCS

This section introduces the kernel function into the OCS-WFCM of the previous section. Clustering is performed in the kernel space, and the observed data is mapped to a higher dimensional feature space in a nonlinear way to achieve nonlinear classification techniques. It is assumed that φ is a nonlinear mapping function, $\phi :x\to \phi \left(x\right)\in$ maps the high characteristic space, where $x\in X=\left\{x_1,x_2,\dots ,x_n\right\}.\phi \left(x_{jk}\right)$ is the mapping of the j th sample data point to the k th feature in the feature space. The optimization objective function of feature weighted kernel FCM (WKFCM) with OCS is as follows.

(3.17)${\displaystyle min{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }{∥\varphi \left(x_{jk}\left(m_{jk}\right)\right)-\varphi \left(v_{ik}\right)∥}^2}$

Expanding ${∥\varphi \left(x_{jk}\left(m_{jk}\right)\right)-\varphi \left(v_{ik}\right)∥}^2$ in (Eq. 3.17), we can get

(3.18)${\displaystyle \begin{array}{c}{∥\varphi \left(x_{jk}\left(m_{jk}\right)\right)-\varphi \left(v_{ik}\right)∥}^2\hfill \\ =\varphi \left(x_{jk}\left(m_{jk}\right)\right)\cdot \varphi \left(x_{jk}\left(m_{jk}\right)\right)-2\varphi \left(x_{jk}\left(m_{jk}\right)\right)\cdot \varphi \left(v_{ik}\right)+\varphi \left(v_{ik}\right)\cdot \varphi \left(v_{ik}\right)\hfill \\ =K\left(x_{jk}\left(m_{jk}\right),x_{jk}\left(m_{jk}\right)\right)-2K\left(x_{jk}\left(m_{jk}\right),v_{ik}\right)+K\left(v_{ik},v_{ik}\right)\hfill \end{array}}$

Where, $K\left(x,y\right)=\varphi \left(x\right)\cdot \varphi \left(y\right)$ represents the kernel function, which can be used to represent the dot product in the high-dimensional feature space. The kernel function used in this work is the Gaussian kernel function, that is, $K\left(x,y\right)=exp\left(\frac{-{∥x-y∥}^2}{{\sigma }^2}\right)$, then $K\left(x,x\right)=1$.

Simplifying (Eq. 3.17) relative to (Eq. 3.18) yields

(3.19)${\displaystyle \begin{array}{c}{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }{∥\varphi \left(x_{jk}\left(m_{jk}\right)\right)-\varphi \left(v_{ik}\right)∥}^2\hfill \\ =2{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }\left(1-K\left(x_{jk}\left(m_{jk}\right),v_{ik}\right)\right)\hfill \\ =2{\sum }_{i=1}^c{\sum }_{j=1}^n{\sum }_{k=1}^l{u}_{ij}^m{\omega }_{ik}^{\beta }\left(1-exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sigma }^2}\right)\right)\hfill \end{array}.}$

The optimal solution of (Eq. 3.19) can be further analyzed as

(3.20)${\displaystyle x_{jk}\left(m_{jk}\right)=\frac{{\sum }_{i=1}^c{u}_{ij}^m{\omega }_{ik}^{\beta }exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sigma }^2}\right)v_{ik}}{{\sum }_{i=1}^c{u}_{ij}^m{\omega }_{ik}^{\beta }exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sigma }^2}\right)}.}$

Through the Lagrange multiplier method, on the basic of on the objective function (Eq. 3.19), the updating formulas of membership degree, clustering center and weight matrix are as follows.

(3.21)${\displaystyle u_{ij}={\left({\sum }_{i=1}^c\frac{{\sum }_{k=1}^l{\omega }_{ik}^{\beta }\left(1-exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sigma }^2}\right)\right)}{{\sum }_{k=1}^l{\omega }_{rk}^{\beta }\left(1-exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{rk}∥}^2}{{\sigma }^2}\right)\right)}\right)}^{\frac{1}{1-m}},}$

(3.22)${\displaystyle {\omega }_{ik}^{\beta }={\left[{\sum }_{t=1}^l\frac{{\sum }_{j=1}^n{u}_{ij}^m\cdot \left(1-exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sigma }^2}\right)\right)}{{\sum }_{j=1}^n{u}_{ij}^m\cdot \left(1-exp\left(\frac{-{∥x_{jt}\left(m_{jt}\right)-v_{rt}∥}^2}{{\sigma }^2}\right)\right)}\right]}^{\frac{1}{1-\beta }},}$

(3.23)${\displaystyle v_{ik}=\left\{\begin{array}{c}0,if{\omega }_{ik}^{\beta }=0\hfill \\ \frac{{\sum }_{j=1}^n{u}_{ij}^{m}exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sigma }^2}\right)\cdot x_{jk}}{{\sum }_{j=1}^n{u}_{ij}^{m}exp\left(\frac{-{∥x_{jk}\left(m_{jk}\right)-v_{ik}∥}^2}{{\sigma }^2}\right)},if{\omega }_{ik}^{\beta }\ne 0\hfill \end{array}\right.}$

The main steps of OCS-WKFCM algorithm are:

(1) Initialization: Set the fuzzy parameter m, number of clusters $c\left(2\le c\le \sqrt{n}\right)$, utmost allowed iterations G, iterative convergence threshold ɛ, missing attribute matrix ${\tilde{X}}_M^{\left(0\right)}$, membership matrix U⁽⁰⁾ combined with the constraint conditions.

(2) Updating the cluster center matrix: when the algorithm performs L (L = 1, 2, …) iterations, cluster center V^(l) is updated according to U^(l−1) and (Eq. 3.23).

(3) Calculate the membership matrix: according to V^(l), and (Eq. 3.21) solving membership matrix U^(l).

(4) Calculate the weight matrix: according to V^(l), and (Eq. 3.22) to solve the weight matrix.

(5) Update the missing value: calculate the missing value ${\tilde{X}}_M^{\left(l\right)}$ according to the cluster center matrix V^(l) and membership partition matrix U^(l) and (Eq. 3.20).

(6) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, OCS-WKFCM algorithm iteration stops, and the program outputs membership U and cluster center V; or else L = L + 1, return (3) to continue.

Feature weighted kernel FCM clustering with NPS

NPS-WKFCM divides the sample data x_jk into two parts, the complete attribute part x_jk (o_jk) and the missing attribute part x_jk (m_jk), then x_jk (o_jk) ∪x_jk (m_jk) = x_jk and x_jk (o_jk) remain unchanged in the clustering process. The filling method of missing attributes in NPS-WKFCM is as follows.

(3.24)${\displaystyle x_{jk}\left(m_{jk}\right)=v_{ik}=\left\{\begin{array}{c}0,if{\omega }_{ik}^{\beta }=0\hfill \\ \frac{{\sum }_{j=1}^n{u}_{ij}^m{x}_{jk}}{{\sum }_{j=1}^n{u}_{ij}^m},if{\omega }_{ik}^{\beta }\ne 0\hfill \end{array}\right.,D_{ij}=min\left\{D_{1j},D_{2j},\dots ,D_{cj}\right\}.}$

Similar to OCS-WFCM, only (Eq. 3.20) needs to be replaced with (Eq. 3.24) when updating the missing attributes.

The main steps of NPS-WFCM algorithm are:

(1) Initialization: Set the fuzzy parameter m, number of clusters $c\left(2\le c\le \sqrt{n}\right)$, utmost allowed iterations G, iterative convergence threshold ɛ, missing attribute matrix ${\tilde{X}}_M^{\left(0\right)}$, membership matrix U⁽⁰⁾ combined with the constraint conditions.

(2) Updating the cluster center matrix: when the algorithm performs L (L = 1, 2, …) iterations, cluster center V^(l) is updated according to U^(l−1) and (Eq. 3.23).

(3) Calculate the membership matrix: according to V^(l), and (Eq. 3.21) solving membership matrix U^(l).

(4) Calculate the weight matrix: according to V^l, and (Eq. 3.22) to solve the weight matrix.

(5) Update the missing value: calculate the missing value ${\tilde{X}}_M^{\left(l\right)}$ according to cluster center matrix V^l andmembership partition matrix U^(l) and (Eq. 3.24).

(6) Iteration termination: when the iteration count approaches L = G, or ∀i, k, $max\left|u_{ik}^{\left(l\right)}-u_{ik}^{\left(l-1\right)}\right|<\varepsilon$, NPS-WKFCM algorithm iteration stops, and the program outputs membership U and cluster center V; or else L = L + 1, return (3) to continue.

The complexity of WKFCM

An algorithm requires analysis of time complexity and space complexity. The complexity of OCS-WKFCM and NPS-WKFCM is mainly generated by clustering. In the clustering process, the number of iterations t, the number of clusters c, the dimension of sample data l, the number of data samples n will affect the time complexity of the algorithm. Considering the worst case, the time complexity of FCM clustering algorithm is O (Tcnl). In the actual calculation process, a certain amount of storage space is needed to store data needed for clustering center matrix, weight matrix, the distance between sample data points, etc. Therefore, in order to store sample data, clustering center, weight matrix, and membership matrix, the space complexity is O(nc + nl + 2cl).

The storage space required for clustering centers is O(nc), where nc represents the number of clustering centers. The storage space required for sample data is O(nl), where nl represents the number of sample data points. The storage space required for the weight matrix and membership matrix is O(cl), where cl denotes the number of relationships between the clustering centers and sample data. By summing up these storage requirements, we obtain a total space complexity of O(nc + nl + 2cl).

This space complexity analysis helps us understand the storage space required during the execution of the algorithm. Through such analysis, we can evaluate the storage resource requirements of the algorithm on datasets of different scales, thus gaining a better understanding of the algorithm’s utilization of space.