moSCminer: a cell subtype classification framework based on the attention neural network integrating the single-cell multi-omics dataset on the cloud

Preprocessing and feature selection

To effectively integrate a multi-omics dataset, the conventional approach involves converting the sparse DNA methylation and DNA accessibility dataset matrices into gene-based matrices (Taguchi & Turki, 2021; Xu, Begoli & McCord, 2022). Here’s a breakdown of our preprocessing steps:

For gene expression data, we initially removed genes that lacked read counts for all samples. Subsequently, read counts were normalized according to library size and log-transformed using the ‘Scanpy’ Python package (Wolf, Angerer & Theis, 2018).

Concerning the DNA methylation dataset, we adopted a strategy where CpGs within 2 kb of the promoter regions of each gene were grouped to form a cluster (referred to as CpG cluster). We calculated the cluster’s average methylation values. This approach aligns with existing research demonstrating that the distribution of CpG density around promoter regions is closely linked to gene transcription and expression regulation (Deaton & Bird, 2011; Tian et al., 2022). To mitigate bias stemming from frequent missing values during model training, we performed mean imputation, eliminating CpG clusters with missing values for all samples.

In the case of DNA accessibility, we considered the summation of accessibility values for each gene body, which may be associated with transcription (Xu, Begoli & McCord, 2022). To achieve this, we summed all accessibility peaks within each transcription region to represent gene activity.

Subsequent to preprocessing, features lacking a common gene-based relationship across any omics were filtered out. This was essential to prevent overfitting resulting from high dimensionality, while simultaneously facilitating multi-omics integration grounded in biological connections. Min-max normalization was conducted on the gene expression and DNA accessibility datasets to align their value ranges with that of the DNA methylation dataset.

Cell subtype classification based on the omics-level attention

To empower neural networks for more effective cell subtype classification, we harnessed a self-attention mechanism. This mechanism trains the model to discern the relative importance of each feature (Lin et al., 2017). The self-attention approach was applied individually to each omics dataset. The model subsequently reconstructed features based on their learned importance weight for each omics. These features from all omics datasets were concatenated, creating a new feature representation for cell subtype classification.

Let $k$ represent the number of features, $x_i\in {\mathbf{R}}^k$ denote the $i$th sample, and $x=(x_1,\dots ,x_n)\in {\mathbf{R}}^{n\times k}$ represent a matrix containing all $x_i$. For each feature $j\in \{1:k\}$, we generated a $m$-dimensional embedding vector $e_j$ using random vectors to represent $x_i$ to $\widehat{x_i}$ through multiplication (Beykikhoshk et al., 2020).

(1) $${\hat{x}}_i^{(j)}=f_e(e_j,x_i^{(j)})=e_j{x}_i^{(j)},$$

The model assigns an attention score, ${\alpha }_j$, to each feature $j$ as follows:

(2) $${\overline{x}}_i^{(j)}=tanh(W_1{\hat{x}}_i^{(j)}+b_1)$$

(3) $$s_i^{(j)}=W_{3}tanh(W_2{\hat{x}}_i^{(j)}+b_2)$$

(4) $${\alpha }_i^{(j)}=\frac{\exp (s_i^{(j)})}{{\sum }_{l=1}^k\exp (s_i^{(l)})}$$

(5) $$c_i^{(j)}=\sum _{j=1}^k{\alpha }_i^{(j)}{\overline{x}}_i^{(j)},$$where $W_1$, $W_2$, $W_3$ are the weights and $b_1$ and $b_2$ are the bias terms for each layer. $s_i^{(j)}$ is the attention score representing the importance of each feature ${\hat{x}}_i^{(j)}$ for the $i$th sample, which is converted to ${\alpha }_i^{(j)}$ via normalization using the softmax function. Based on this, ${\hat{x}}_i^{(j)}$ is transformed into a new feature representation of $c_i^{(j)}$ by the weighted sum of the encoded feature vectors ${\overline{x}}_i^{(j)}$ and normalized attention scores ${\alpha }_i^{(j)}$. Each omics dataset underwent this self-attention mechanism. Transformed representations were then concatenated and forwarded to two fully connected layers, culminating in a softmax function for cell subtype classification.

To train the model, we utilized cross-entropy loss as the loss function:

(6) $$\mathcal{L}=-\sum _{i=1}^C{y}_i\log ({\hat{y}}_i),$$where C denotes the number of cell subtypes, and $y$ and $\hat{y}$ represent the true and model-predicted subtype probability distributions, respectively. To prevent overfitting, we incorporated dropout in the fully connected layers.

Implementation of web-based platform connected with the cloud system

We have developed moSCminer as a user-friendly web-based platform connected to a cloud system for enhanced user accessibility and convenience. The platform is designed to automate the analysis process and reduce user intervention. Here are the key features:

• User-friendly interface: The platform boasts a user-friendly interface accessible without requiring users to log in. Whether you have an account or prefer the guest mode, uploading your single-cell multi-omics datasets is a breeze.

• Automated cloud-based analysis: Our web-based platform handles the entire analysis pipeline, encompassing preprocessing, feature selection, and cell subtype classification. These computationally intensive tasks are executed on a cloud system, sparing users the need to manage complex computations.

• Email notifications: Users are promptly notified of their analysis results via email. These results encompass cell subtype annotations and lists of high-importance biomarker candidates identified during the prediction step.

• Interactive visualizations: We understand the significance of visualizing and interpreting results. Thus, our platform offers interactive visualizations of cell subtype classification results. Users can easily access and download various types of figures, including box plots, scatter plots, heatmaps, and correlation plots. These visual aids provide valuable insights into the data.

• Marker candidates: Our platform offers information on marker candidates, making it easier for users to distinguish cell subtypes effectively.

• Scalability: To address potential computational limitations, our platform is connected to a cloud system, ensuring scalability for analysis tasks of varying complexities.

The front-end of the website was meticulously crafted using HTML5, JavaScript, D3, JQuery, and CSS3. On the backend, a web server was developed using Node.js (Tilkov & Vinoski, 2010). The preprocessing, feature selection, and cell subtype classification modules were implemented using Python, utilizing libraries such as Tensorflow (Abadi et al., 2015) and Scikit-learn (Pedregosa et al., 2011). The Linux Bash Shell was also employed to facilitate these tasks.

In summary, moSCminer represents a powerful tool for single-cell multi-omics data analysis. Its user-friendly interface, automated cloud-based analysis, interactive visualizations, and scalability make it a valuable resource for researchers seeking to extract meaningful insights from complex datasets. Additionally, its ability to integrate multiple single-cell omics datasets offers improved cell subtype prediction, opening new avenues for understanding cellular heterogeneity and differentiation processes.

Experimental design

Performance evaluation with the baseline methods

To assess the effectiveness of our proposed method, we compared its performance to that of several machine learning-based classifiers, including Support Vector Machine (SVM), Random Forest (RF), Logistic Regression (LR), and Naive Bayes (NB), implemented using the ‘Scikit-learn’ package (Pedregosa et al., 2011). Similar to our model, we optimized each classifier for each dataset through grid search, selecting the hyperparameter combination yielding the highest average accuracy for the testing dataset. Those results are shown in Tables S2–S4. We performed five-fold cross-validation to evaluate accuracy, the weighted F1-score, the Matthews correlation coefficient (MCC), and the Area under the ROC Curve (AUC) as evaluation metrics. Additionally, we used the same multi-omics features selected during the feature selection step for the baseline methods. As illustrated in Fig. 2 and Table 2, moSCminer outperformed the baseline methods, achieving an average weighted F1-score of 0.954 and 0.986 for the GSE154762 and GSE136718 dataset, respectively, compared to the second-highest average F1-score of 0.916 and 0.970 achieved by RF. For the largest dataset of GSE140203, composed of 32,321 cells with 22 cell subtypes, our method achieved the highest classification performance with an average AUC of 0.983, where RF obtained 0.926. The cell subtype-wise performance results were reported in Table S5. For GSE136718 and GSE154762 datasets, relatively having small number of samples and subtypes, moSCminer generally demonstrated the best or similar prediction performance compared to other methods. But, when applied to more complex cell subtype prediction task using GSE140203 dataset, moSCminer outperformed all the baseline methods and its variant for all 22 cell subtypes, achieving the best performance.

Effectiveness of omics-level attention

Our proposed method leverages omics-level attention to transform features into new representations, capturing the relative importance weights for cell subtype classification. To assess the impact of omics-level attention on prediction performance, we implemented a variant of our model by removing the attention components, referred to as “moSCminer (no-attn),” and conducted 5-fold cross-validation to compare performance. Notably, without the attention module, classification performance decreased (Fig. 2 and Table 2). For the GSE136718 dataset, the average accuracy dropped from 0.986 to 0.957, while for relatively having larger datasets with more samples and complex cell subtypes, a significant performance drop was observed for average accuracy, from 0.956 to 0.816 (GSE154762), and from 0.983 to 0.856 (GSE140203). Similar performance change was also shown in the cell subtype-wise performance results (Table S5). These results underscore the effectiveness of omics-level attention in our proposed method and the importance of new feature representations derived from feature transformation in cell subtype classification using single-cell multi-omics data.

Cell subtype prediction improvement by multi-omics integration

To evaluate whether cell subtype classification based on multi-omics integration improved performance, we conducted five-fold cross-validation and compared our model’s performance using either single omics or a combination of two omics datasets. We denoted gene expression as ‘gene,’ DNA methylation as ‘methyl,’ and DNA accessibility as ‘acc.’ As shown in Table 3, utilizing single-omics dataset could provide the performance higher than 0.9, for example, the average accuracy of 0.962 in GSE136718 using gene expression profiles, or 0.950 using DNA accessibility data in GSE140203. However, the proposed model, when trained using all three multi-omics datasets, consistently achieved the highest average prediction performance across all three datasets. This indicates that integrating omics datasets from different biological layers enhances the cell subtype classification model’s accuracy compared to using single omics data.

Identification of marker candidates for cell subtype prediction

During the training phase of moSCminer, the omics-level attention scores were learned, providing relative importance scores for cell subtype classification. Features with the highest attention scores hold the potential to be cell marker candidates for cell subtype annotations. We analyzed the attention scores obtained from moSCminer for GSE136718 and GSE154762 datasets and identified the top 30 features with the highest scores from each omics for further examination (Table S6). To assess the relevance of these features to cell subtype classification within cells, we compared the normalized abundance differences between the cell subtypes. We conducted one-way analysis of variance (Lix, Keselman & Keselman, 1996) to test the statistical significance of the subtype differences. The results (Fig. 3) revealed that features with the highest attention scores exhibited significant differences among subtypes, with a p-value < 0.01, providing evidence that moSCminer can identify features highly relevant for distinguishing cell subtypes within cells.

Moreover, the overlap of the top 30 feature lists with the highest attention scores from each omics for the GSE136718 dataset is depicted in the Venn diagram (Fig. 4). This illustrates that moSCminer identifies a subset of features common to multiple omics, reinforcing their potential as robust marker candidates for cell subtype classification.

To further assess the biological relevance of the top 30 features from each omics for each cell type, we conducted a manual literature review. During the preprocessing step, as omics features were transformed into a gene-based matrix for multi-omics integration, we examined whether any features exhibited high attention scores in all three omics types. In the case of the GSE136718 dataset, we identified four gene-based features that overlapped among the top 30 features from each omics: H2-Q1, Ccdc64, C130026I21Rik, and 4933407C03Rik (Fig. 4). H2-Q1 gene was reported to be transcribed in the brains of the E14.5 embryos in mice and is also expressed in the adult brain, suggesting a potential functional role in both adult and embryonic brains (Ohtsuka et al., 2008). In case of Ccdc64, a synonym of Bicdr1 gene, high expression levels of Ccdc64 at early embryonic stages inhibit neuritogenesis and decrease during embryonic development, thereby controlling neuronal differentiation (Schlager et al., 2010).

We also found evidence that other features not in the overlap have the potential to be cell markers: Prdm10 is known to support cell growth and survival during early embryonic development (Han et al., 2020). Bptf regulates genes and signaling pathways essential for the development of key tissues in the early mouse embryo (Landry et al., 2008). TAF3 and Shb are reported to be involved in embryonic stem cell differentiation (Kriz et al., 2006; Liu et al., 2011).

For GSE154762 dataset, we did not find overlapping features in any of the three omics datasets. However, several features from each omics dataset demonstrated biological relationships with human oocytes and somatic cells: CAMK1D is reported as a potential follicular cell biomarker correlated with oocyte quality, embryo competence, and pregnancy outcome (Yerushalmi et al., 2014). TMOD3 plays a crucial role in oocyte maturation by controlling the density of the cytoplasmic actin mesh (Jo et al., 2016). MYO5B may play an important role in actin cytoskeleton remodeling during oocyte maturation (Jia et al., 2022). TMEFF2 was studied for its upregulation in early oocyte development in the primordial and primary follicle stages (Yu et al., 2020). These findings illustrate that moSCminer can provide cell marker candidates with the potential to serve as markers for cell subtype annotations.