Hardware implementation of FPGA-based spiking attention neural network accelerator

Introduction

Spiking neural networks (SNNs) represent the third generation of neural networks that fundamentally differ from traditional artificial neural networks (ANNs) in their information processing mechanisms. Unlike ANNs that use continuous activation functions, SNNs communicate through discrete, binary spike events that more closely mimic the temporal dynamics of biological neural systems.

SNNs are built upon spiking neuron models that maintain an internal membrane potential V(t). The most widely adopted model is the leaky integrate-and-fire (LIF) neuron, where the membrane potential evolves according to the differential equation:

(1) $$\tau \frac{dV}{dt}=-V(t)+I(t)$$where $\tau$ represents the membrane time constant and $I(t)$ denotes the input current. When $V(t)$ exceeds a predefined threshold $V_{th}$, the neuron generates a spike and resets its potential to a resting state $V_{reset}$.

Information in SNNs is encoded in the precise timing and frequency of spikes rather than continuous numerical values. This temporal coding scheme naturally leads to sparse, event-driven computation where processing occurs only when spikes are present, contrasting sharply with the dense matrix operations characteristic of traditional neural networks.

SNNs inherently process both spatial and temporal information through their membrane dynamics and spike timing dependencies. This capability enables SNNs to capture time-dependent patterns and sequential relationships without requiring explicit recurrent connections, making them particularly advantageous for processing dynamic, time-varying data streams.

The energy efficiency of SNNs stems from several fundamental characteristics: (1) sparse, event-driven computation that consumes energy only when spikes occur; (2) elimination of expensive multiply-accumulate operations in favor of simple accumulate-and-compare operations; (3) natural compatibility with neuromorphic hardware architectures that exploit asynchronous processing; and (4) binary spike communication that reduces data movement and storage requirements.

In recent years, SNNs have gradually become a research hotspot in the field of artificial intelligence (Yamazaki et al., 2022), regarded as the ‘third-generation neural networks’ (Holmes, Sacchi & Bellazzi, 2004) that can better reflect the information processing mechanism of the human brain (Frackowiak et al., 2004) by simulating the spikes between biological neurons. SNNs exhibit unique neurobiological plausibility through their intrinsic capacity for spatiotemporal integration during both computational processing and learning phases (Cai et al., 2023). These neuromorphic architectures process discrete binary spike inputs that demonstrate dynamic spatiotemporal sparsity, a characteristic enabling self-regulated neuronal activation patterns.

This biological fidelity facilitates the substitution of conventional multiply-accumulate operations with energy-efficient additive mechanisms, thereby substantially reducing computational complexity while maintaining temporal pattern recognition capabilities. The resultant energy-optimized computational paradigms have demonstrated particular advantages in embedded neural processing applications, especially in low-power scenarios such as Internet of Things (IoT) devices (Huh, Cho & Kim, 2017), embedded systems (Kaelbling, 1993), and edge computing environments (Tran et al., 2017). The sparsity and event-driven nature of SNNs reduces memory accesses by decreasing the number of operations (Stuijt et al., 2021), in contrast to traditional neural networks (Razi & Athappilly, 2005) where a large percentage of energy consumption comes from algorithm-related memory accesses.

Field programmable gate arrays (FPGAs) represent an ideal platform for implementing SNN hardware accelerators (Han et al., 2020) due to their flexible hardware reconfigurability and parallel computing capabilities (Brown et al., 2012; Kulkarni et al., 2019; Bhattacharjee et al., 2024). FPGAs offer numerous advantages in neural network acceleration, including low latency, high efficiency, and effective power consumption control. However, FPGAs have limited resources, and balancing inference speed, power consumption, and model accuracy with these constraints remains a key challenge.

While GPU architectures demonstrate limitations in harnessing the intrinsic sparsity of SNNs during training phases (Cong et al., 2018), their fundamental mismatch with temporal accumulation-dominant computations further restricts efficient implementation. Recent advances in neuromorphic computing have yielded specialized ASIC solutions (Isik, 2023), including Intel’s Loihi (Davies et al., 2018), the SpiNNaker system from Manchester (Manchester & Loeb, 2017), and IBM’s True North (Akopyan et al., 2015). Despite achieving benchmark performance, these fixed-function designs face dual challenges of architectural rigidity against evolving SNN topologies and prohibitive fabrication costs. In contrast, modern FPGA platforms leverage reconfigurable logic fabrics and adaptive memory hierarchies to dynamically reconcile computational sparsity with temporal dependencies.

Recent advances in neuromorphic circuit design have yielded systematic improvements in spatiotemporal signal processing. Researchers have enhanced the biological fidelity of LIF neurons through adaptive membrane threshold modulation and dynamic encoding mechanisms (Lu & Xu, 2022). Parallel innovations in hardware-software co-design include sparsity-driven acceleration architectures (Lien & Chang, 2022) that implement bitmask-compressed event streams to exploit activation sparsity at synaptic granularity. Complementary approaches have introduced convolution-pooling co-design paradigms (Liu et al., 2023) that minimize layer-wise latency through fused spatial-temporal operations, achieving 23% higher resource utilization efficiency than conventional pipelined implementations. Scaling these principles, researchers have developed dual-path spiking convolutional neural network (CNN) frameworks (Parashar et al., 2017) featuring hierarchical complexity allocation, where compact micro-networks handle basic pattern recognition while macro-structures process high-dimensional temporal features. Additionally, channel balanced workload prediction techniques (Chen et al., 2022) have been proposed to optimize computational resource allocation.

However, significant challenges remain in SNN training methodologies. While spike timing dependent plasticity (STDP) (Caporale & Dan, 2008) provides biological plausibility, it suffers from poor accuracy and limited scalability to multi-layer networks (Putra, Hanif & Shafique, 2022; Datta et al., 2022; Lu & Xu, 2022; Lien & Chang, 2022). Supervised backpropagation algorithms (LeCun et al., 1988) and stochastic gradient descent (Bottou, 2012), though achieving high accuracy for large models, face theoretical limitations when applied to discrete, non-differentiable spike functions in SNNs.

While neuromorphic hardware demonstrates ultra-low power profiles during SNN inference phases, existing implementations often improve inference speed through network simplification, parameter reduction, and computational complexity decrease. Although this approach can enhance inference speed, it frequently leads to decreased recognition accuracy and fails to fully exploit SNN potential. Therefore, improving inference speed while guaranteeing network accuracy and optimizing power consumption under FPGA resource limitations represents an urgent research direction.

To address these challenges, this article proposes SeaSNN, an FPGA-based spiking neural network accelerator that introduces a novel spiking efficient channel attention (SECA) mechanism. This mechanism enhances the network’s ability to capture key information by weighting spiking signals, thereby improving recognition accuracy while effectively reducing redundant information propagation and maintaining low power consumption. In addition, to optimize hardware parallelism and inference speed, this article implements various hardware architecture optimizations in the FPGA implementation. Firstly, for loop unrolling techniques are employed to unroll operations requiring multiple iterations into parallel computations to reduce loop control overhead; secondly, for loop pipelining is performed to improve throughput and execution efficiency by executing tasks in different iterations simultaneously (Luo et al., 2023). For further optimization, this article also employs loop flattening and array partitioning techniques to spread nested loops and partition data and parameters to enhance parallelism and data processing efficiency.

The combined use of these optimizations enables SeaSNN to achieve significant performance improvements on FPGAs. In terms of experimental validation, this article uses the MNIST handwritten digit recognition dataset, and test results show that SeaSNN achieves an inference accuracy of 94.28% on FPGA, which is a significant improvement compared to 93.73% before optimization. The inference speed is also significantly improved, with a single inference time of only 0.000401 s, while power consumption is controlled within 0.42 TOPS/W. Compared with traditional SNN-based accelerator designs, SeaSNN achieves a better balance of accuracy, speed and energy consumption, indicating that the proposed spiking neural network accelerator has excellent performance in handwritten digit recognition tasks and demonstrates wide potential in practical application scenarios.

In summary, this work provides an efficient implementation scheme for SNN hardware accelerator design with the following main contributions:

• The accuracy of the model and hardware execution efficiency are successfully improved by introducing the spiking efficient channel attention mechanism (SECA).

• Various optimizations are performed on the FPGA hardware architecture. Firstly, for loop unrolling techniques are adopted to unroll operations requiring multiple iterations into parallel computation to reduce loop control overhead; secondly, for loop pipelining is performed to improve hardware throughput and execution efficiency by executing tasks in different iterations simultaneously. For further optimization, this article also employs loop flattening and array partitioning techniques, which spread nested loops and split data and parameters to enhance parallelism and data processing efficiency.

• Experimental results show that the FPGA-based SNN accelerator proposed in this article can achieve high-precision inference tasks with low power consumption, which is especially suitable for applications in resource-limited embedded systems and edge computing scenarios.

The manuscript’s organizational framework proceeds systematically through three principal components. We commence our technical exposition in “Methods” with comprehensive analysis of the accelerator’s architectural paradigm, including its operational data pipeline, core component implementations, and neural network infrastructure design. “Results and Discussion” subsequently validates these architectural decisions through empirical verification, demonstrating measurement outcomes and performance benchmarks obtained from our FPGA-based SNN implementation. The culminating “Conclusion” synthesizes key findings while projecting potential trajectories for neuromorphic computing development.

Related work

With the development of deep learning (Schmidhuber, 2015), SNNs have received much attention in many fields due to their bio-inspired and low-power characteristics, especially in applications such as the Internet of Things (IoT) (Huh, Cho & Kim, 2017), embedded systems (Kaelbling, 1993), and edge computing (Shi et al., 2016). SNNs achieve more energy-efficient information processing mechanisms by mimicking the impulsive communication of biological neurons, and thus have an advantage over traditional CNNs (O’Shea, 2015). However, hardware implementations of SNNs face a number of challenges, particularly how to optimize accelerator performance while balancing inference speed, accuracy and hardware power consumption (Maguire et al., 2007). This section reviews important recent work on FPGA-based SNN accelerators, attention mechanisms in SNNs, and hardware optimization strategies, specifically analyzing their limitations and how our work addresses these challenges.

Several research teams have made significant progress in implementing SNN hardware accelerators on FPGAs, but existing approaches face critical limitations that our work specifically addresses. Khodamoradi, Denolf & Kastner (2021) proposed an FPGA-based accelerator design for SNNs, which employs a spiking accumulation mechanism to reduce redundant transmission of spiking data and optimizes inference speed through resource reuse and hardware pipelining techniques. While this approach achieves high inference speed in low-power environments, it suffers from limited network accuracy improvement and lacks sophisticated feature selection mechanisms. Our SeaSNN addresses this limitation by introducing the SECA mechanism, which enhances accuracy while maintaining the energy efficiency advantages of spike accumulation.

Similarly, Liu, Yenamachintala & Li (2019) designed a sparsified storage and transmission mechanism based on spiking data characteristics, significantly improving inference speed and energy efficiency through efficient storage architectures and parallel computation models in FPGAs. However, this work primarily focuses on data movement optimization without addressing the challenge of intelligent feature selection within the sparse spike patterns. Building upon their sparsification insights, our SECA mechanism adds a crucial layer of intelligent channel weighting that selectively processes the most informative sparse spike patterns, thereby achieving both computational efficiency and enhanced recognition capability.

The resource-constrained problem represents a critical challenge when implementing SNNs in FPGAs, particularly in optimizing network structure and parameters for efficient inference with limited hardware resources. Existing optimization strategies have made valuable contributions but leave important gaps that our approach fills.

To address resource constraints, Hoefler et al. (2021) proposed pruning and sparsification methods for deep neural networks, removing redundant connections and neurons to reduce hardware resource consumption and accelerate inference. While effective for general network compression, this approach does not specifically exploit the unique temporal sparsity characteristics of SNNs. Our work leverages this pruning foundation but extends it through SECA’s dynamic channel attention, which adaptively identifies and emphasizes the most informative spike channels rather than applying static pruning criteria.

In quantization research, Wu et al. (2023) proposed low-bit quantization techniques to represent pulse signals and weights in SNNs as low-bit integers, significantly reducing storage requirements and computational complexity. This quantization approach successfully reduces hardware resource consumption but does not address the challenge of maintaining high accuracy under severe resource constraints. Our SeaSNN incorporates these quantization benefits while introducing SECA to compensate for potential accuracy loss through intelligent feature selection, achieving better accuracy-efficiency trade-offs than quantization alone.

Based on these insights, our work implements efficient parallel computation on FPGA through hardware optimization strategies such as loop flattening and array partitioning, extending beyond existing approaches by combining these techniques with attention-driven computation scheduling.

The attention mechanism has become crucial for improving neural network performance (Niu, Zhong & Yu, 2021) by dynamically weighting different network components to focus on key feature information. However, the integration of attention mechanisms with SNNs remains significantly underdeveloped compared to their widespread adoption in traditional CNNs, presenting both challenges and opportunities that our work addresses.

Recent pioneering efforts have begun exploring attention in SNNs. Zhu et al. (2024) proposed temporal attention-based SNNs that introduce weighting mechanisms in the temporal domain, enabling dynamic adjustment of network connection weights according to time-dimensional spiking information. While this approach demonstrates performance improvements in temporal data processing, it suffers from high implementation complexity and substantial hardware resource demands, making it impractical for resource-constrained FPGA implementations. Our SECA mechanism addresses this limitation by focusing on channel-wise attention rather than complex temporal dependencies, achieving comparable performance benefits with significantly lower computational overhead.

Building on this foundation, Cai et al. (2023) proposed a lightweight channel attention mechanism specifically for weighting spiking data in SNNs, demonstrating improved computational efficiency and accuracy through optimized channel distribution of spiking data. However, this work lacks comprehensive hardware implementation validation and does not provide detailed analysis of the attention mechanism’s integration with FPGA-specific optimizations. Our SECA design extends their channel attention concept by providing a complete hardware-software co-design solution, demonstrating concrete FPGA implementation results and achieving measurable accuracy improvements (from 93.73% to 94.28% on MNIST).

The reviewed literature reveals three critical gaps that our SeaSNN architecture specifically addresses: (1) existing FPGA-based SNN accelerators achieve speed and efficiency but lack sophisticated accuracy enhancement mechanisms; (2) current resource optimization strategies do not fully exploit SNN-specific sparsity characteristics for intelligent feature selection; and (3) attention mechanisms for SNNs remain either too complex for practical hardware implementation or lack comprehensive validation in resource-constrained environments.

Our work synthesizes the strengths of existing approaches while addressing their limitations through: leveraging proven sparsification and quantization techniques from Liu, Yenamachintala & Li (2019), Wu et al. (2023) while adding SECA’s intelligent channel selection; building upon the hardware optimization foundations of Khodamoradi, Denolf & Kastner (2021), Hoefler et al. (2021) while introducing attention-driven computation scheduling; and extending the channel attention concepts from Cai et al. (2023) with complete FPGA implementation and validation.

This comprehensive approach enables SeaSNN to achieve superior accuracy-efficiency trade-offs compared to existing solutions, demonstrating the practical viability of attention-enhanced SNNs in resource-constrained hardware environments.

Methods

The system architecture for handwritten digit recognition is shown in Fig. 1. It consists of two main components: a programmable logic module (PL) and a processing system (PS), based on the ZYNQ architecture. The PL includes the SECA module, multi-layer fully connected (FC) layers, and spiking neuron layers (LIF) for feature extraction and spiking signal processing. The FC and LIF layers are alternately arranged and communicate with the PS via the AXI interconnection interface (AXI Master).

The PS integrates dual ARM Cortex-A9 CPUs for loading model parameters, displaying results, and controlling system processes. DDR DRAM stores large datasets and model parameters, while high-speed data transmission between the PS and PL is achieved via the AXI bus, ensuring efficient computation and data management.

Figure 2 shows the workflow of the handwritten digit recognition system. After initialization, weight parameters are stored on an SD card, read through an interface function, and cached in DDR memory on the PS side. Image data is defined in a RAM array on the PS side. The PL performs computations, transferring image data to DRAM via DMA. The ARM processor then sends parameters and image data to the FPGA-based accelerator on the PL for inference, with results returned to the PS.

LIF neurons

The leaky integrate-and-fire (LIF) neuron model bridges the gap between the Hodgkin-Huxley and artificial neuron models. Similar to an artificial neuron, it computes a weighted sum of inputs but integrates the input over time with a leakage mechanism, resembling an RC circuit. When the integrated value surpasses a threshold, the LIF neuron generates a voltage spike.

LIF neurons simplify output spikes by treating them as discrete events, encoding information in spike timing or frequency rather than shape. This abstraction makes them suitable for studying neural codes, memory, network dynamics, and deep learning. LIF neurons balance biological plausibility and computational efficiency, providing both realism and practicality, as shown in Fig. 3. A neuron fires and resets its membrane potential when V exceeds $V_{t}hr$, unless in the refractory period $T_R$, where it stays inactive despite surpassing $V_T$. Due to leakage, the membrane potential also decreases without input spikes.

At time step $a$, the variables $U_a^b(i)$ and $S_a^b(i)$ denote the membrane potential and spike of neuron $i$ in layer $b$, respectively. The temporal dynamics are determined by the membrane potential and spike from the previous time step, along with the leakage constant $\alpha$. The spatial dynamics, on the other hand, are shaped by the weighted spikes received from the preceding layer. The firing function, as defined in Eq. (2), is triggered when the membrane potential exceeds the threshold $t{h}_f$, causing the neuron to fire a spike and reset its potential to zero.

(2) $$\{\begin{array}{c}{U}_a^b(i)=\underset{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}}{\underbrace{\alpha U_{a-1}^b(i)\left(1-S_{a-1}^b(i)\right)}}+\underset{\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l}}{\underbrace{{\sum }_j{S}_a^{b-1}(j)w^{b-1}(j,i)}},\\ S_a^b(i)=fire\left(U_a^b(i)-t{h}_f\right).\end{array}$$

(3) $$fire(x)=\{\begin{array}{cc}x,& x\ge 0\\ 0,& x<0\end{array}$$

Equation (3) illustrates the output sparsity of a neuron during the backward pass. When the membrane potential lies within the threshold range $t{h}_b-t{h}_h$, the value of $\sim S_a^b(i)$ equals $1$. Conversely, if the membrane potential falls outside this range, $\sim S_a^b(i)$ is set to $0$. When $\sim S_a^b(i)$ is $0$, the term $\mathrm{\Delta }{S}_a^b(i)$ is excluded from the update of $\mathrm{\Delta }{U}_a^b(i)$.

(4) $$\sim S_a^b(i)\approx \{\begin{array}{cc}0,& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\\ 1,& t{h}_l<u_t^l[i]<t{h}_h\end{array}$$

(5) $$\{\begin{array}{c}\mathrm{\Delta }{S}_a^b(i)=\underset{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}}{\underbrace{\mathrm{\Delta }{U}_{a+1}^b(i)(-\alpha U_a^b(i))}}+\underset{\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l}}{\underbrace{{\sum }_j\mathrm{\Delta }{U}_{a+1}^b(j)w^b(i,j)}},\\ \mathrm{\Delta }{U}_a^b(i)=\mathrm{\Delta }{U}_{a+1}^b(i)\alpha \left(1\text{-}{S}_a^b(i)\right)+\mathrm{\Delta }{S}_a^b(i)\sim S_a^b(i).\end{array}$$

The computation of $\mathrm{\Delta }{S}_a^b(i)$, as described in Eq. (4), involves two components. The temporal component is influenced by the gradient of the potential at the subsequent time step, the membrane potential at that step, and the leakage constant $\alpha$. The spatial component is determined by the weighted sum of the potential gradients from the next layer at that time step. The weight update, given by Eq. (5), is obtained by accumulating the product of the potential gradient from the previous layer and the input spike over time.

(6) $$\mathrm{\Delta }{w}^b(i,j)=\sum _a\mathrm{\Delta }{U}_a^{b+1}(j)S_a^b(i).$$

SeaSNN model for digital recognition

The network model is implemented using snnTorch (Eshraghian et al., 2023), a Python library for gradient-based learning of SNNs, and realized with digital circuits. To achieve high recognition accuracy and inference speed, a seven-layer spiking neural network was designed, as shown in Fig. 4. Training combines spike-timing-dependent plasticity (STDP) with gradient-based learning for handwritten digit recognition on the MNIST dataset.

To deploy SNNs on mobile devices for handwritten digit recognition, two key criteria are essential: high recognition accuracy and low complexity, ensuring compatibility with general FPGA implementations. Based on these criteria, we designed a lightweight network, “SeaSNN,” as shown in Fig. 4. The network comprises seven layers, including a SECA layer, three fully connected layers, and LIF neural layers.

SECA attention mechanism model

The SECA mechanism captures inter-channel dependencies through local interactions, eliminating the need for additional parameters or complex global pooling operations. A central innovation of SECA is the adoption of a parameter-free one-dimensional local convolution, which facilitates efficient channel-wise interaction while reducing computational overhead and enhancing salient feature identification. The design of the SECA module incorporates insights derived from the analysis of channel dimensionality reduction and cross-channel interactions, as depicted in Fig. 5.

An aggregated feature $\mathbf{y}\in {\mathbb{R}}^C$, where C denotes the channel dimension, carries channel information through the network.

(7) $$\omega =\sigma (\mathbf{W}\mathbf{y})$$

Here, $\mathbf{W}$ is a $C\times C$ parameter matrix. A band matrix, ${\mathbf{W}}_k$, is used to model channel attention and capture local cross-channel interactions.

(8) $$\left[\begin{array}{cccccccc}{w}^{1,1}& \cdots & w^{1,k}& 0& 0& \cdots & \cdots & 0\\ 0& w^{2,2}& \cdots & w^{2,k+1}& 0& \cdots & \cdots & 0\\ ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& ⋮& ⋮\\ 0& \cdots & 0& 0& \cdots & w^{C,C\text{-}k+1}& \cdots & w^{C,C}\end{array}\right]$$

The matrix contains $k\times C$ parameters, as shown in Eq. (7), and is generally small. Equation (8) ensures some interaction between groups, while Eq. (6) captures local dependencies by focusing on interactions between $y_i$ and its $k$ neighbors. This method strikes a balance between preserving local context and computational efficiency.

(9) $${\omega }_i=\sigma \left(\sum _{j=1}^k{w}_i^j{y}_i^j\right),y_i^j\in {\mathrm{\Omega }}_i^k$$

Here, ${\mathrm{\Omega }}_i^k$ denotes the set of $k$ neighboring channels of $y_i$, capturing nearby channels that contribute to local cross-channel interactions.

A more efficient strategy involves having all channels utilize the same learning parameters, i.e.:

(10) $${\omega }_i=\sigma \left(\sum _{j=1}^k{w}^j{y}_i^j\right),y_i^j\in {\mathrm{\Omega }}_i^k$$

This strategy can be efficiently implemented using a fast 1D convolution with kernel size $k$, i.e.:

(11) $$\omega =\sigma (\mathrm{C}1{\mathrm{D}}_k(\mathbf{y}))$$

Here, $\mathrm{C}1\mathrm{D}$ denotes a 1D convolution operation. The method in Eq. (5) is implemented via the SECA module, using only $k$ parameters. With $k=3$, it balances model complexity and local cross-channel interaction capture (Kulkarni et al., 2019), enhancing efficiency and effectiveness.

Group convolutions have advanced CNN architectures by applying convolutions to channels of varying dimensions, depending on the number of groups. Inspired by this, we propose a correlation between the interaction extent, represented by the kernel size $k$ in 1D convolution, and the channel dimension C. This relationship is formalized as a functional mapping, $\phi$, linking $k$ and C:

(12) $$C=\varphi (k).$$

The function $\phi$ defines how the optimal kernel size $k$ adapts to changing channel dimensions, offering a framework to tailor interaction ranges across architectures. This reduces reliance on manual tuning and resource-intensive cross-validation.

The simplest mapping, $\phi (k)=\gamma k-b$, is inherently limited. Since channel dimensions C are often powers of two, introducing nonlinearity to $\phi (k)$ improves flexibility and better exploits these characteristics.

(13) $$C=\varphi (k)=2^{(\gamma \ast k-b)}.$$

Then, given the channel dimension C, the suitable kernel size $k$ can be dynamically determined through an adaptive process by:

(14) $$k=\psi (C)={\left|\frac{lo{g}_2(C)}{\gamma }+\frac{b}{\gamma }\right|}_{odd}.$$

Here, $|t{|}_{odd}$ denotes the nearest odd number to $t$. In all experiments, $\gamma$ and $b$ are set to 2 and 1, respectively. This mapping ensures that high-dimensional channels interact over larger distances, while low-dimensional channels interact over shorter ones. The nonlinear mapping dynamically adjusts the kernel size according to channel dimensions, improving cross-channel interaction efficiency.

Circuit optimization method

SeaSNN’s acceleration circuitry is implemented using high-level synthesis (HLS), allowing developers to focus on algorithm design rather than circuit implementation, thus improving development efficiency. The modular design maps each module to a SeaSNN layer, storing intermediate feature data in DDR3 DRAM instead of the FPGA. This reduces FPGA storage demands and enhances network scalability and adaptability. Each layer’s circuit code is encapsulated as an IP core, which is interconnected in Vivado to build the SeaSNN circuit. The ZYNQ7 processing system connects these IP cores via four AXI high-performance (HP) buses, ensuring high-throughput data exchange between the processing system (PS) and programmable logic (PL).

Circuits synthesized with HLS generally require more FPGA resources and longer execution times than those using hardware description languages (HDLs). However, with proper optimizations, HLS-synthesized circuits can achieve comparable performance. To minimize delays in the SeaSNN circuit and enable handwritten digit recognition, techniques like loop unrolling, loop pipelining, and array partitioning are employed.

Unrolling the for_loop

The schematic of unrolling the for loop (Fig. 6) illustrates a loop with three iterations, each performing a multiplication followed by an addition. Without unrolling, the loop uses a single circuit, executing iterations sequentially. With the UNROLL optimization, three circuits are synthesized to execute iterations in parallel, tripling FPGA resource usage but reducing execution time by 66%. This trade-off significantly accelerates convolution calculations, enhancing inference efficiency.

Pipelining the for_loop

When a for loop transfers data between the PS and PL, pipelined optimization (PIPELINE) improves execution efficiency. As shown in Fig. 7, the operation “input[i] /= pix;” involves four sequential steps: address calculation, data read, division, and result write-back (Fig. 7B). PIPELINE optimization converts these steps into a 4-stage pipeline, enabling parallel execution (Fig. 7C) and reducing execution time by approximately 75%.

Array partitioning

Low data access bandwidth is the primary bottleneck in SeaSNN’s parallel computing. Limited RAM access ports in FPGA’s on-chip RAM lead to conflicts when multiple circuits access the same RAM, restricting parallelism. ARRAY_PARTITION optimization addresses this by splitting data into smaller RAMs, reducing conflicts and enhancing efficiency and throughput. As shown in Fig. 8, methods include block, round-robin, and complete partitioning. Complete partitioning, when resources permit, maximizes throughput and circuit efficiency.

Results and discussion

In this section, we evaluate the proposed SeaSNN model on the MNIST benchmark dataset. All experiments were conducted on a computer equipped with a 3.6 GHz Intel (R) CoreTM i7-12700KF processor and an NVIDIA GeForce GTX 3090 graphics card (24G RAM). The deep learning framework used is PyTorch, and the optimizer is Adam, with a learning rate of 2e-3.

Resource utilization

Figure 9 illustrates the FPGA resource utilization of the SeaSNN accelerator. LUT and FF resources dominate, indicating heavy use of logic units and flip-flops, while BRAM and DSP usage is comparatively lower, reflecting minimal storage and computational resource demands.

Conclusion

This article presents a low-power, high-precision FPGA-based system for accelerating spiking neural networks (SNNs) in handwritten digit recognition tasks. To mitigate accuracy degradation due to hardware constraints in existing SNN accelerators, a simple and efficient four-layer network, “SeaSNN,” is proposed, incorporating a pulse channel attention mechanism (SECA) to boost performance. Experiments on the MNIST dataset demonstrate that SECA improves accuracy from 93.73% to 94.28%, validating its effectiveness.

To boost inference speed, FPGA circuit implementation is optimized using loop unrolling, pipelining, and array partitioning, greatly enhancing parallelism. These optimizations achieve an inference speed of 0.000401 seconds per frame at 200 MHz with a power efficiency of 0.42 TOPS/W. The results demonstrate high accuracy and efficiency in handwritten digit recognition, with low power consumption, making the system ideal for resource-constrained edge computing.

This work offers a practical solution for deploying SNNs on hardware and provides insights into network and hardware optimization. Future research will focus on more complex SNN architectures and advanced hardware techniques to further advance SNN adoption in real-world applications.

Supplemental Information