iPro-CSAF: identification of promoters based on convolutional spiking neural networks and spiking attention mechanism

Experimental datasets

The first step in building a useful promoter predictor is to select reliable benchmark and independent datasets for method training and performance validation. Our datasets are from seven species including both prokaryotes and eukaryotes. Prokaryotes include E. coli, B. subtili, and cyanobacteria. The length of each sample is 81 bp. Eukaryotes include H. sapiens, D. melanogaster, M. musculus, A. thaliana. The length of each sample is 300 bp.

For E. coli promoter recognition, all the samples of our benchmark dataset are obtained from the dataset constructed by Xiao et al. (2019) (https://ars.els-cdn.com/content/image/1-s2.0-S088875431830613X-mmc1.pdf). The independent dataset for E. coli promoter recognition is obtained from the dataset constructed by Liu et al. (2018) (https://academic.oup.com/bioinformatics/article/34/1/33/4158035#supplementary-data). It contains 2,860 promoter sequences and 2,860 non-promoter sequences. For B. subtilis promoter recognition, all the samples are from the datasets constructed by (Umarov & Solovyev, 2017) (https://github.com/solovictor/CNNPromoterData.git). For cyanobacteria promoter recognition, all the samples are obtained from the DB1 datasets constructed by (Yang et al., 2024) (https://github.com/Passion4ever/SiamProm).

For promoter recognition of A. thaliana, M. musculus, D. melanogaster, H. sapiens, all their samples are obtained from the datasets constructed by Zhu et al. (2021) (https://github.com/zhuyaner/Depicter/). The four eukaryotic promoter datasets mentioned above are randomly divided into benchmark datasets and independent datasets at a ratio of 9:1. For each of the eukaryotic promoter datasets, we perform three recognition tasks including TATA promoter and non-promoter recognition, non-TATA promoter and non-promoter recognition, and also TATA&non-TATA promoter and non-promoter recognition. For the benchmark datasets of all these seven species, we validate our method using five-fold cross-validation. After that, in order to prevent experimental over-fitting and further evaluate prediction performance of our iPro-CSAF, independent datasets are used to test the method. The samples of benchmark datasets are summarized in Table 1.

The overall architecture of iPro-CSAF

iPro-CSAF consists of one-hot coding layer, a spiking feature extraction module and a spiking classification module, as shown in Fig. 1. Spiking feature extraction module consists of a 1-d convolutional spiking layer, a spiking feature fusion module which contains two parallel branches and one concatenation module.

In Fig. 1, the first convolutional spiking layer receives one-hot encoding matrices of promoter and non-promoter samples. And after the spiking neuron layer and batch normalization (BN) layer, samples are converted into discrete binary spikes (Xiong et al., 2021), thus realizing the transmission of spikes between different layers. Therefore, the first convolutional spiking layer not only extracts features but also encodes one-hot vectors into spikes. In feature fusion module, each branch further extracts features from the output of first convolutional layer. The output of these two parallel branches is concatenated to achieve the fusion of local and global features of promoter sequences. The spiking features after fusion are fed to a spiking fully connected layer for making decisions. Surrogate gradient (Neftci, Mostafa & Zenke, 2019) is used to enable end-to-end training of iPro-CSAF. Dropout operation is adopted in both the feature extractor and the spiking classifier, which can randomly remove some neurons to avoid overfitting (Dahl, Sainath & Hinton, 2013).

One-hot encoding of input promoter sequences

There are four types of nucleotides in a DNA sequence: A (adenine), C (cytosine), G (guanine), and T (thymine). DNA sequences must be converted into numerical vectors to perform feature extraction and classification. The one-hot encoding approach is widely used in the recognition of promoter sequences in deep learning. By one-hot encoding, each nucleotide is encoded as a four-dimensional binary vector whose elements represent four types of nucleotides, respectively. One position of the one-hot vector is 1 and the elements of the remaining positions are all 0. The detailed form of each nucleotide encoded by one-hot encoding is shown below: $A\left(1,0,0,0\right)$, $C\left(0,1,0,0\right)$, $G\left(0,0,1,0\right)$, $T\left(0,0,0,1\right)$. After one-hot encoding, a promoter sequence of length L is encoded into an L × 4 two-dimensional matrix.

Spiking neuron model

Spiking neuron models simulate the firing of biological neurons and are the basic processing units of SNNs. Frequently used spiking neuron models are the IF model, leaky integration and firing (LIF) model, Hodgkin-Huxley (H-H) model and Izhikevich model. A spiking neuron receives input which changes its neuronal membrane potential, and emits a spike when its membrane potential increases to a prespecified threshold potential. The process of charging, discharging, and resetting of a spiking neuron can be represented by the following three equations:

(1) $$H\left(t\right)=f\left(V\left(t-1\right),X\left(t\right)\right)$$ (2) $$S(t)=\theta (H(t)-V_{thr})=\{\begin{array}{c}1,H(t)-V_{thr}\ge 0\\ 0,H(t)-V_{thr}<0\end{array}$$ (3) $$V\left(t\right)=H\left(t\right)\ast \left(1-S\left(t\right)\right)+V_{reset}\ast S\left(t\right)$$where Eq. (1) describes the charging process of a neuron. $t$ represents simulation time step. $V\left(t-1\right)$ is the membrane potential at time step $t-1$. $X\left(t\right)$ is the external input. $f\left(V\left(t-1\right),X\left(t\right)\right)$ is a function describing the updating of neuron membrane potential. For different spiking neuron models, this function is different. $H\left(t\right)$ is the hidden state of a neuron, which is instantaneous voltage before the neuron fires a spike at time step $t$. Eq. (2) describes the discharging process of a neuron. $V_{thr}$ refers to the threshold membrane potential of a neuron. If $H\left(t\right)$ exceeds the threshold membrane potential, the neuron is activated and emits a spike. Eq. (3) describes the resetting of a neuron’s membrane potential. $V_{reset}$ is the reset potential after the neuron sends out a spike. If the neuron does not send out a spike, then $V\left(t\right)=H\left(t\right)$. If the neuron sends out a spike, then $V\left(t\right)=V_{reset}$.

In our convolutional spiking layers and fully connected spiking layer, we use IF neuron model because it has a simple structure and is suitable for large-scale simulation. The membrane potential $V\left(t\right)$ remains constant when there is no external input, and the update process of $V\left(t\right)$ of IF neuron is shown as follows:

(4) $$dV\left(t\right)/dt=R\cdot I\left(t\right)$$where $R$ is the leakage resistance of an IF neuron model. $I\left(t\right)$ is the external input current.

In our spiking self-attention module, LIF neuron model is adopted to generate spikes. The update process of $V\left(t\right)$ is shown as follows:

(5) $$\tau \cdot dV\left(t\right)/dt=-\left(\mathrm{V}\left(\mathrm{t}\right)-{\mathrm{V}}_{\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{t}}\right)+R\cdot I\left(t\right)$$where $\tau$ is time constant of membrane potential.

The convolutional spiking layers

In our method we use one-dimensional convolutional layers which are suitable for processing sequence data. The first convolutional spiking layer receives one-hot encoding binary matrices, and the remaining convolutional spiking layer receives spikes which are output from the IF neuron layer. An example illustration of the spiking convolutional operation based on IF neurons is shown in Fig. 2. Here, the input is [0, 0, 1, 0, 0, 1, 0]. The size of the one-dimensional convolution kernel is 3 and the stride size is 1. The convolution kernel covers the corresponding positions of the input spiking sequence in turn and performs the convolution operation. After that the feature response map [0.5, 0.2, 1.3, 0.2] is obtained and it represents the membrane potentials of IF spiking neurons in spiking neuron layer. The membrane potential threshold is set to be 1. If the feature value exceeds the membrane potential threshold, the neuron will discharge and release a spike and the membrane potential will be reset. If the feature value is less than the membrane potential threshold, the membrane potential will remain constant. Finally, the output spike train is [0, 0, 1, 0].

Multi-head spiking self-attention

The input of multi-head spiking self-attention is spike feature maps $\mathrm{X}$ from the first convolutional spiking layer, which have the data format $\mathrm{T}\times \mathrm{N}\times \mathrm{D}$. Here T is the time step, N is the number of spiking sequences, and D is the dimension of spiking sequences.

The SSA module consists of linear layer, batch normalization layer and spiking neuron layer.

The input is mapped to Query (Q), Key (K), Value (V) by linear layer: $\mathrm{Q}=\mathrm{X}\cdot {\mathrm{W}}_{\mathrm{Q}}$, $\mathrm{K}=\mathrm{X}\cdot {\mathrm{W}}_{\mathrm{k}}$, $\mathrm{V}=\mathrm{X}\cdot {\mathrm{W}}_{\mathrm{v}}$. The weights of linear layer are $W_Q,W_K,W_V$. Then, Q, K and V are encoded into spike matrices $Q,K,V$by being input to batch normalization layer and spiking neuron layer. The calculation equation is as follows:

(6) $$\mathrm{Q}=\mathrm{L}\mathrm{I}\mathrm{F}\left(\mathrm{B}\mathrm{N}\left(\mathrm{X}\cdot {\mathrm{W}}_{\mathrm{Q}}\right)\right),\mathrm{K}=\mathrm{L}\mathrm{I}\mathrm{F}\left(\mathrm{B}\mathrm{N}\left(\mathrm{X}\cdot {\mathrm{W}}_{\mathrm{K}}\right)\right),\mathrm{V}=\mathrm{L}\mathrm{I}\mathrm{F}(\mathrm{B}\mathrm{N}\left(\mathrm{X}\cdot {\mathrm{W}}_{\mathrm{V}}\right).$$

Then the attention maps $\mathrm{S}\mathrm{S}\mathrm{A}\left(\mathrm{Q},\mathrm{K},\mathrm{A}\right)$ are calculated and encoded into spikes by linear layer and spiking neuron layer. The specific process is shown as follows:

(7) $$\mathrm{S}\mathrm{S}\mathrm{A}\left(\mathrm{Q},\mathrm{K},\mathrm{A}\right)=\mathrm{L}\mathrm{I}\mathrm{F}\left(\mathrm{B}\mathrm{N}\left(\mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}\left(\mathrm{L}\mathrm{I}\mathrm{F}\left(\mathrm{Q}{\mathrm{K}}^{\mathrm{T}}\mathrm{V}\cdot \mathrm{s}\right)\right)\right)\right)$$where $\mathrm{Q},\mathrm{K},\mathrm{V}\mathrm{ϵ}{\mathrm{R}}^{\mathrm{T}\times \mathrm{N}\times \mathrm{D}}$, and $\mathrm{s}$ is a scaling factor to control the large value of an attention map. Multi-head spiking self-attention mechanism (MSSA) is multiple parallel SSA operations. $\mathrm{Q},\mathrm{K},\mathrm{V}$ are split into $\mathrm{H}$ parts. We run SSA operations for H times. The calculation process of multi-head attention maps are as follows:

(8) $$\mathrm{M}\mathrm{S}\mathrm{S}\mathrm{A}\left(\mathrm{Q},\mathrm{K},\mathrm{V}\right)=[\mathrm{S}\mathrm{S}{\mathrm{A}}_1({\mathrm{Q}}_1,{\mathrm{K}}_1,{\mathrm{V}}_1),\mathrm{S}\mathrm{S}{\mathrm{A}}_2({\mathrm{Q}}_2,{\mathrm{K}}_2,{\mathrm{V}}_2),..........,\mathrm{S}\mathrm{S}{\mathrm{A}}_{\mathrm{H}}\left({\mathrm{Q}}_{\mathrm{H}},{\mathrm{K}}_{\mathrm{H}},{\mathrm{V}}_{\mathrm{H}}\right)]$$

The attention maps of SSA are obtained by logical AND operation and addition of spike-form Q, K, V. It avoids float-form multiplications.

Spiking feature fusion module

We propose a spiking feature fusion module which contains two parallel branches and one concatenation module. The two parallel branches are a convolutional spiking layer and a multi-head spiking self-attention module, respectively. Each branch further extracts features from the output of the first convolutional layer. And the concatenation module fuses local features output from the convolutional spiking layer and global features output from the spiking attention mechanism. All features are fused and further transmitted in the form of spikes. They are input to the fully connected spiking layer to make decisions. The feature maps from the convolutional spiking layer and spiking attention mechanism are $E_{conv}\mathrm{ϵ}{R}^{T\times N\times D}$ and $E_{MSSA}\mathrm{ϵ}{R}^{T\times N\times D}$, respectively, where T is the time step, N is the number of channels of the feature maps, and D is the dimension of spiking sequences. They are concatenated after being flattened. The feature fusion maps are $E_{concatenation}\mathrm{ϵ}{R}^{T\times \left(N\times D+N\times D\right)}$.

The surrogate gradient algorithm

Because in the spiking neuron model, the step function $\theta \left(x\right)$ in Eq. (2) is non-differentiable (Guo & Wang, 2023), it is not possible to train a spiking neural network directly using backpropagation algorithm. We use surrogate gradient learning to train iPro-CSAF. $\theta \left(x\right)$is approximated by a differentiable gating function, known as surrogate gradient. In the forward propagation process, the network still uses the step function $\theta \left(x\right)$. In the backpropagation process, the surrogate gradient is calculated by surrogate function $g\left(x\right)$. In convolutional layers, we use arctan function $g\left(x\right)=(1/\pi )\cdot \arctan \left((\pi /2)\cdot \alpha \cdot x\right)+1/2$. In spiking self-attention, we use sigmoid function $g\left(x\right)=sigmoid\left(\alpha x\right)=1/\left(1+e^{-\alpha x}\right)$. Here the parameter $\alpha$ is proportional to the degree of similarity between $g\left(x\right)$ and $\theta \left(x\right)$. It can control the degree of smoothing of the surrogate function. The larger $\alpha$ is, the more prone to vanishing gradient and exploding gradient, which makes the network more difficult to be trained. In arctan function, we set $\alpha =2$. In sigmoid function, we set $\alpha =4$.

Experimental setup and parameter selection

Our network is implemented by utilizing the spiking neural network framework SpikingJelly based on Pytorch and python3.9. The mean-square error (MSE) is used as loss function and the Adam optimization algorithm is adopted.

The choice of parameters has great influence on promoter recognition performance of computational methods. We perform hyperparameter tuning for the convolutional layers, time steps, learning rate, the number of heads in MSSA and dropout ratio. The range of tuning parameters of our method is shown in Table 2. The values of hyperparameters in Table 2 are slightly different for specific promoter recognition tasks in this study.

The spatial complexity of the model can be represented by the number of model parameters which is often used to measure the performance of deep learning models in terms of complexity. Table 3 gives the specific network structure and parameter numbers of each layer in our method. In the Results & discussion section, we will compare the number of parameters of iPro-CSAF with those of compared methods.

Method evaluation metrics

Four commonly used metrics are used to assess the performance of the method, including sensitivity (Sn), specificity (Sp), accuracy (Acc), Matthews correlation coefficient (MCC), and F1-score. The metrics are described as follows:

(9) $$recall,Sn=TP/\left(TP+FN\right)$$ (10) $$Sp=TN/\left(TN+FP\right)$$ (11) $$Pre=TP/\left(TP+FP\right)$$ (12) $$Acc=(TP+TN)/\left(TP+TN+FP+FN\right)$$ (13) $$MCC=\left(TP\times TN-FP\times FN\right)/\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}$$ (14) $$F1score=2\times \left(Pre\times Sn\right)/\left(Pre+Sn\right)$$where TP, TN, FP and FN are the number of samples that the method predicts as true positive, true negative, false positive and false negative, respectively. Sp and Sn denote the ability of the method to correctly identify negative and positive samples, respectively. Pre denotes the ratio of true promoters that are recognized as promoters. Acc denotes the percentage of samples correctly categorized by the method. MCC shows the recognition results of methods on unbalanced datasets. F1-score (F1) is a comprehensively evaluation metric which considers precision and recall. The values of sensitivity, specificity, F1 score and accuracy are all between 0 and 1, and the value of the MCC is between −1 and 1. Larger values of these metrics represent better performance of the method. In addition, receiver operating characteristic (ROC) curves and the area under ROC curve (AUC) are also applied to evaluate the overall performance of recognition methods. The AUC value closer to 1 indicates better performance. Area under the precision recall curve (AUPRC) is used to evaluate the recognition performance of methods on imbalanced datasets. AUPRC calculates the area under Precision-Recall curve. The value of AUPRC indicates Pre performance at different Recall levels.

K-fold cross-validation is widely used to estimate generalization of computational methods. We validate the promoter recognition performance of our method by performing five-fold cross-validation on benchmark datasets from the seven species. Each dataset is divided into five subsets of equal size, where four subsets are used as the training datasets and the remaining subset is used as a validation dataset, alternately. The final recognition results are the average evaluation metric values of five-fold cross-validation.

Experiments design and compared methods

In this section, we introduce our experiments design and some state-of-art deep learning methods which are used to make performance comparison with our method.

In our experiments, we first make ablation study to verify the effectiveness of our proposed feature fusion module. Then we take three-layer CSNNs and two-layer CSNNs cascaded with a multi-head SSA module as baseline methods, and compare the promoter recognition performance of our iPro-CSAF with baseline methods on A. thaliana benchmark datasets.

After that, we compare our method with some advanced deep learning methods on both benchmark datasets and independent datasets. Most of these compared methods are designed only for a certain species or a certain class of species, while in our work our datasets are from seven species including both prokaryotes and eukaryotes, therefore in our comparison experiments, for different species we compare our method with different deep learning methods. For E. coli datasets, we compare our method with FastText N-grams (Le et al., 2019), iPSW(PseDNC-DL) (Tayara, Tahir & Chong, 2020), iPromoter-BnCNN (Amin et al., 2020), iPro2L-CLA (Zhang et al., 2022) and iProL (Peng, Sun & Fan, 2024). For B. subtilis dataset, we compare our method with CapsProm (Moraes et al., 2022). For the cyanobacteria dataset, the compared methods are DeePromoter (Oubounyt et al., 2019), CyaPromBERT (Mai, Nguyen & Lee, 2022) and SiamProm (Yang et al., 2024). For A. thaliana datasets, we compare our method with Depicter. For H. sapiens datasets, we compare our method with Depicter and DeePromClass (Kari et al., 2023). For D. melanogaster and M. Musculus datasets, we compare our method with Depicter, DeeProPre (Ma et al., 2022) and DeePromClass (Kari et al., 2023).

Finally, we calculate the power consumption of iPro-CSAF and compare it with that of traditional neural network with the same structure.

Ablation study

We conduct ablation experiments to evaluate the effectiveness of feature fusion module which has a parallel structure including a convolutional spiking layer and a multi-head spiking self-attention module.

In this experiment we use H. sapiens non-TATA promoter datasets which are eukaryotic promoters and E. coli promoter datasets which are prokaryotic promoters. These two datasets are relatively more difficult to classify among the seven species. iPro-CSAF consists of two convolutional spiking layers and a spiking attention mechanism. We name the first convolution layer Conv_1 and the second convolution layer Conv_2 as shown in Fig. 1. In our feature fusion module, it contains two parallel branches and one concatenation module. The two parallel branches are Conv_2 and MSSA, respectively.

Table 4 shows the results of ablation experiments. For H. sapiens non-TATA promoter recognition, iPro-CSAF outperforms two-layer CSNNs (Conv_1 + Conv_2), one-layer CSNN cascaded with a MSSA (Conv_1 + MSSA) and one-layer CSNN (Conv_1) by 1.69%, 2.40% and 4.51%, respectively. For E. coli promoter recognition, iPro-CSAF outperforms two-layer CSNNs (Conv_1 + Conv_2), one-layer CSNN cascaded with a MSSA (Conv_1 + MSSA) and one-layer CSNN (Conv_1) by 1.95%, 2.12% and 3.11%, respectively. The results of ablation experiments show that after extracting feature with Conv_1, our feature fusion module with parallel structure can significantly improve the performance of the method and obtain higher recognition accuracy than using convolutional spiking layer or spiking attention mechanism alone to further extract features.

Comparison of iPro-CSAF with baseline methods

As shown in Fig. 1, our method consists of two convolutional spiking layers and a multi-head SSA module. The extracted features from the two parallel branches are fused to make decisions. In order to further evaluate the effectiveness of our method, we take three-layer CSNNs and two-layer CSNNs cascaded with a multi-head SSA module as baseline methods. We compare promoter recognition performance of iPro-CSAF with baseline methods on A. thaliana benchmark datasets. Table 5 shows the recognition performances of iPro-CSAF and baseline methods. We use a Mann-Whitney’s U test to evaluate statistically significant difference between Acc results for A. thaliana TATA&non-TATA promoter recognition of different methods. Statistical difference is significant at p-value < 0.05. The p-values between Acc results of iPro-CSAF and three-layer CSNNs and two-layer CSNNs +MSSA both are 0.027. It is shown that the Acc results of iPro-CSAF and baseline methods are significantly different.

As shown in Table 5, iPro-CSAF can obtain better promoter recognition accuracy than baseline methods. For A. thaliana non-TATA promoter recognition, iPro-CSAF outperforms three-layer CSNNs by 0.44% Two-layer CSNNs cascaded with a multi-head SSA module outperforms two-layer CSNNs cascaded with a convolutional layer. This is due to that SSA can extract long-distance dependence features and focus on the more important features in sequences. By using parallel branches of a convolutional layer and a multi-head spiking self-attention module, our iPro-CSAF outperforms two-layer CSNNs cascaded with a multi-head SSA module by 0.15%

Comparison of iPro-CSAF with state-of-the-art methods on benchmark datasets

In this section, we compare the recognition performance of our iPro-CSAF with some state-of-the-art deep learning methods. The recognition performance of iPro-CSAF and compared methods on benchmark datasets is shown in Table 6.

First we compare our iPro-CSAF with other methods on prokaryotic promoter recognition. For the E. coli benchmark dataset, we compare the recognition performance of our iPro-CSAF with FastText N-grams (Le et al., 2019), iPSW(PseDNC-DL) (Tayara, Tahir & Chong, 2020), iPro2L-CLA (Zhang et al., 2022) and iProL (Peng, Sun & Fan, 2024). iPSW(PseDNC-DL) and FastText N-grams both used multiple CNN layers to make further feature extraction. Differently, iPSW(PseDNC-DL) extracted priori biological features by PseDNC, while FastText N-grams extracted priori text features by FastText natural language processing method. iPro2L-CLA used one-hot coding scheme, then adopted multiple deep learning algorithms including CNNs, LSTM, capsule networks and attention mechanism to extract promoter sequence features. iProL used longformer (Beltagy, Peters & Cohan, 2020), CNNs and bidirectional LSTM (BiLSTM) to extract features directly from input promoter sequences to identify E. coli promoters. As shown in Table 6, iPro-CSAF obviously outperforms all these compared methods in terms of Sp, Acc and MCC. The results show that our method which uses parallel combination of convolution and MSSA outperforms promoter recognition methods which used combination of CNNs, LSTM, capsule networks and attention mechanism (iPro2L-CLA, iProL), and it also outperforms CNNs which needed priori biological feature extraction (iPSW(PseDNC-DL)) or priori text feature extraction (FastText N-grams).

For B. subtilis promoter recognition, we compare the recognition performance of our iPro-CSAF with CapsProm. CapsProm used one-hot coding scheme and combined CNNs with capsule networks to extract features. Our iPro-CSAF outperforms CapsProm in terms of Sn, Sp, Acc, F1 and MCC. The Acc value of iPro-CSAF is higher by 2.46% than CapsProm. Moreover, the number of parameters of CapsProm is 1271127, much more than 77248 of iPro-CSAF. Therefore, our iPro-CSAF can achieve much better recognition performance than CapsProm with much lower network complexity.

For cyanobacteria promoter recognition, we compare our iPro-CSAF with DeePromoter (Oubounyt et al., 2019), CyaPromBERT (Mai, Nguyen & Lee, 2022) and SiamProm (Yang et al., 2024). DeePromoter was a CNNs-based promoter recognition method combined with BiLSTM. CyaPromBERT utilized Bidirectional Encoder Representations from Transformers (BERT) to perform pre-training on genomes for promoter recognition. SiamProm was a cyanobacterial promoter recognition method based on the Siamese network which contained two same subnetworks. Each subnetwork comprised four parallel modules including an embedding initializer, a k-mer attention module, a bi-directional context catcher and a nearest-neighbor aggregator. It was a complex deep learning method. Our iPro-CSAF outperforms DeePromoter and CyaPromBERT in terms of Sn, Sp, Acc and MCC. The Acc value of iPro-CSAF is 96.63%, slightly lower than 96.80% of SiamProm. But parameter numbers of SiamProm is 7,253,376, much more than 77,248 of iPro-CSAF. This shows that our method can achieve comparable accuracy with SiamProm, while has much lower network complexity than SiamProm.

We then compare our iPro-CSAF with other methods on eukaryotic promoter recognition. For the A. thaliana promoter recognition, we compare the recognition performance of iPro-CSAF with Depicter (Zhu et al., 2021). Depicter first used one-hot coding, then adopted CNNs and capsule networks to extract promoter features. For all the three recognition tasks of A. thaliana, our iPro-CSAF outperforms Depicter in terms of Sp, Acc, F1 and MCC. The Acc value of iPro-CSAF is higher by 1.63% on A. thaliana TATA promoter recognition. It shows our parallel combination of convolution and MSSA is superior to serial combination of convolution and capsule networks (Depicter).

For M. musculus promoter recognition, we compare iPro-CSAF with DeePromClass (Kari et al., 2023), DeeProPre (Ma et al., 2022) and Depicter. DeePromClass first used one-hot coding, then adopted CNNs and LSTM networks to extract promoter features. DeeProPre first encoded promoter sequences by a word embedding layer, then extracted further features by combing BiLSTM with CNNs. For all the three recognition tasks of this species, iPro-CSAF outperforms all the three compared methods in terms of Sp, Acc, F1, MCC and AUPR, and the Acc values of iPro-CSAF are all above 99% on M. musculus promoter recognition. It shows our parallel combination of convolution and MSSA is also superior to serial combination of convolution and LSTM networks.

For H. sapiens promoter recognition, we compare iPro-CSAF with DeePromClass (Kari et al., 2023) and Depicter. For all the three recognition tasks of H. sapiens, our iPro-CSAF outperforms Depicter in terms of Sn, Sp, Acc, F1, MCC and AUPR. For the TATA&non-TATA promoter recognition, the Acc of iPro-CSAF is 1.25% and 2.14% higher than Depicter and DeePromClass, respectively.

For D. melanogaster promoter recognition, the compared methods are DeePromClass (Kari et al., 2023), DeeProPre (Ma et al., 2022) and Depicter, the same as M. musculus promoter recognition. iPro-CSAF outperforms all the three methods in terms of Acc, F1, MCC and AUPR. The Acc values of iPro-CSAF are higher by 2.29%, 2.42% and 3.37% than Depicter on the three types of D. melanogaster promoter recognition, respectively. We also compare the network complexity of our method with Depicter, DeeProPre and DeePromClass. And the parameter numbers of Depicter, DeeProPre and DeePromClass are 4,177,744, 606,337 and 38,329,410 respectively, which are much more than 77,248 of iPro-CSAF. It shows that our iPro-CSAF can achieve better recognition performance than these methods with much lower network complexity.

From our results on prokaryotic promoter recognition, it can be seen that our method which uses parallel combination of convolution and MSSA outperforms promoter recognition methods which used combination of CNNs, LSTM, capsule networks and attention mechanism (iPro2L-CLA, iProL, CapsProm, DeePromoter), and it also outperforms CNNs which needed priori biological feature extraction (iPSW(PseDNC-DL) or priori text feature extraction (FastText N-grams). Our iPro-CSAF can achieve comparable accuracy with promoter recognition method which used combination of text features and biological sequence features (SiamProm), while has much lower network complexity. For B. subtilis promoter recognition, the Acc value of iPro-CSAF is higher than CapsProm. It indicates good performance of our method on unbalanced datasets. From our results on eukaryotic promoter recognition, it can be seen that our method outperforms promoter recognition methods which used combination of CNNs and LSTM (DeePromClass and DeeProPre), and it also outperforms method which used combination of CNNs and capsule networks (Depicter).

As a whole, the Acc values of iPro-CSAF on eukaryotic promoter recognition are higher than that on prokaryotic promoter recognition. The results show that our iPro-CSAF with simple structure has good ability and generalization to extract promoter features.

Comparison of iPro-CSAF with state-of-the-art methods on independent datasets

The independent datasets constructed in ‘Materials and Methods’ are used to validate the promoter recognition performance of our proposed method. The recognition performance of iPro-CSAF and compared methods on independent datasets is shown in Table 7. First we compare our iPro-CSAF with other methods on prokaryotic promoter recognition. For E. coli promoter recognition, our iPro-CSAF is compared with iProL (Peng, Sun & Fan, 2024), iPro2L-CLA (Zhang et al., 2022), iPSW(PseDNC-DL) (Tayara, Tahir & Chong, 2020) and iPromoter-BnCNN (Amin et al., 2020). The first three methods are also used as compared methods for E. coli promoter recognition on benchmark dataset. iPromoter-BnCNN used four parallel three-layer CNNs to extract sequence features and each of the four CNNs used different encoding methods. As shown in Table 7, iPro-CSAF outperforms all the compared methods in terms of Sn, Sp, Acc and MCC. The Acc value of iPro-CSAF is significantly higher than iProL, iPro2L-CLA, iPromoter-BnCNN and iPSW(PseDNC-DL) by 1.12%, 8.39%, 5.72% and 8.36%, respectively. The results again show that our method outperforms promoter recognition methods which used combination of CNNs, LSTM, capsule networks and attention mechanism (iPro2L-CLA), CNNs which needed priori biological feature extraction (iPSW(PseDNC-DL)). And it also outperforms method which used parallel three-layer CNNs (iPromoter-BnCNN).

Then we compare our iPro-CSAF with other methods on eukaryotic promoter recognition. For the A. thaliana promoter recognition, we compare iPro-CSAF with Depicter (Zhu et al., 2021) which is also the compared method on benchmark datasets of A. thaliana. For all the three types of promoter recognition, iPro-CSAF outperforms Depicter in terms of F1. And for the non-TATA promoter recognition and TATA&non-TATA promoter recognition, the Acc values of iPro-CSAF are slightly higher than that of Depicter. For the TATA promoter recognition, the Acc value of iPro-CSAF is 99.29%, very close to that of 99.37% of Depicter. But the parameter numbers of Depicter is 4,177,744, much more than 77,248 of iPro-CSAF.

For the M. musculus promoter recognition, we compare iPro-CSAF with DeePromClass (Kari et al., 2023), DeeProPre (Ma et al., 2022) and Depicter, which are also the compared methods on benchmark datasets. For all the three types of promoter recognition, iPro-CSAF outperforms all the compared methods in terms of F1, Acc, MCC and AUPR.

For H. sapiens promoter recognition, we compare iPro-CSAF with DeePromClass (Kari et al., 2023) and Depicter, which are also the compared methods on benchmark datasets. For all the three types of promoter recognition, iPro-CSAF outperforms all the compared methods in terms of Sp, F1, Acc, MCC and AUPR. The AUC value of iPro-CSAF is higher than Depicter by 5.24% on TATA&non-TATA promoter recognition.

For D. melanogaster promoter recognition, we compare iPro-CSAF with DeePromClass (Kari et al., 2023), DeeProPre (Ma et al., 2022) and Depicter, the same as benchmark datasets. iPro-CSAF outperforms all the three compared methods in terms of Acc, F1, MCC and AUPR. The Acc values of iPro-CSAF is higher than Depicter by 2.89%, 4.3% and 3.13%, respectively, on the three types of D. melanogaster promoter recognition.

The results on prokaryotic promoter recognition in Table 7 again show that our method outperforms promoter recognition methods which used combination of CNNs, LSTM, capsule networks and attention mechanism (iPro2L-CLA), CNNs which needed priori biological feature extraction (iPSW(PseDNC-DL)). And it also outperforms methods which used parallel three-layer CNNs (iPromoter-BnCNN). The results on eukaryotic promoter recognition in Table 7 again show our method is superior to serial combination of convolution and LSTM networks (DeePromClass and DeeProPre), and also superior to combination of convolution and capsule networks (Depicter). Our method shows good ability to identify promoters from multiple species and good generalization for promoter recognition.

Moreover, we plot ROC curves of our model for promoter recognition on independent datasets as shown in Fig. 3. All the curves cover almost entire area, and the values of AUC are close to 1, which indicates the good ability of predicting promoters.

The power consumption of iPro-CSAF

To evaluate the power consumption of our method, we calculate the power consumption using Zhou’s method (Zhou et al., 2022). We calculate the power consumption of iPro-CSAF for A. thaliana TATA&non-TATA promoter recognition and compare it with that of traditional neural network with the same structure.

To calculate the power consumption of spiking neural networks, we first need to calculate the number of spike-based accumulate (AC) operations SOPs (Merolla et al., 2014) which is represented as follows:

(15) $$\mathrm{S}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{l}\right)={\mathrm{f}}_{\mathrm{r}}\times \mathrm{T}\times \mathrm{F}\mathrm{L}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{l}\right)$$where $\mathrm{l}$ is a block or layer in SNNs. ${\mathrm{f}}_{\mathrm{r}}$ refers to firing rate of spiking neurons of this block or layer in SNNs. $\mathrm{T}$ is the simulation time steps. $\mathrm{F}\mathrm{L}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{l}\right)$ refers to the number of multiply-and-accumulate (MAC) operations of $\mathrm{l}$ in traditional neural networks. It is assumed that the MAC and AC operations are implemented on the 45 nm hardware, where the power consumption of a MAC operation is ${\mathrm{E}}_{\mathrm{M}\mathrm{A}\mathrm{C}}=4.6\mathrm{p}\mathrm{J}$ and the power consumption of an AC operation is ${\mathrm{E}}_{\mathrm{A}\mathrm{C}}=0.9\mathrm{p}\mathrm{J}$. The power consumption calculation equation of iPro-CSAF is as follows:

(16) $${\mathrm{P}}_{\mathrm{i}\mathrm{P}\mathrm{r}\mathrm{o}-\mathrm{C}\mathrm{S}\mathrm{A}\mathrm{F}}={\mathrm{E}}_{\mathrm{M}\mathrm{A}\mathrm{C}}\cdot \mathrm{F}\mathrm{L}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{\_}1\right)+{\mathrm{E}}_{\mathrm{A}\mathrm{C}}\cdot \left(\mathrm{S}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{\_}2+\mathrm{M}\mathrm{S}\mathrm{S}\mathrm{A}\right)+\mathrm{S}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{F}\mathrm{C}\right)\right)$$where ${\mathrm{E}}_{\mathrm{M}\mathrm{A}\mathrm{C}}\cdot \mathrm{F}\mathrm{L}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{\_}1\right)$ is the power consumption calculation of Conv_1. It performs traditional convolutional operation and there is no spike in this layer. ${\mathrm{E}}_{\mathrm{A}\mathrm{C}}\cdot \left(\mathrm{S}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{\_}2+\mathrm{M}\mathrm{S}\mathrm{S}\mathrm{A}\right)+\mathrm{S}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{F}\mathrm{C}\right)\right)$ is power consumption calculation of Conv_2, MSSA and the fully connected layer FC. The feature maps generated by Conv_1 are encoded into spikes train. Therefore, they perform spike-based operations. And we calculate the power consumption of traditional neural network with the same structure as iPro-CSAF for comparison. Its power consumption equation is as follows:

(17) $${\mathrm{P}}_{\mathrm{A}\mathrm{N}\mathrm{N}}={\mathrm{E}}_{\mathrm{M}\mathrm{A}\mathrm{C}}\cdot \left(\mathrm{F}\mathrm{L}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{\_}1\right)+\mathrm{F}\mathrm{L}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{\_}2+\mathrm{M}\mathrm{S}\mathrm{S}\mathrm{A}\right)+\mathrm{F}\mathrm{L}\mathrm{O}\mathrm{P}\mathrm{s}\left(\mathrm{F}\mathrm{C}\right)\right)$$

Table 8 shows the power consumption of iPro-CSAF and that of traditional neural network with the same structure of iPro-CSAF on A. thaliana TATA&non-TATA benchmark dataset. As shown in Table 8, the power consumption of spiking neural network with spiking neurons is significantly lower than traditional neural network with the same structure. It indicates that our SNN method can achieve good promoter recognition results with low power consumption.