A unified Big-O complexity analysis of convolutional and transformer architectures in deep learning
Abstract
Deep learning has emerged as a central paradigm in modern computing, where both convolutional neural networks (CNNs) and Transformer architectures dominate vision, language, and multimodal applications. While accuracy remains a critical benchmark, the mathematical characterization of computational efficiency is equally essential for sustainable deployment. This study develops a unified Big-O framework that formalizes the asymptotic time and memory complexity of CNN and Transformer components. The analysis rigorously distinguishes between forward and backward propagation, parameter versus activation storage, and the scaling behavior induced by convolutional operations compared with self-attention mechanisms. Empirical measurements are then aligned with the theoretical derivations through regression-based scaling laws, confirming that CNNs exhibit near-linear growth with input resolution, whereas self-attention layers follow quadratic dependence on sequence length. Efficient variants such as depthwise separable convolutions reduce constant factors without altering asymptotic order, while hierarchical and approximate attention mechanisms mitigate but do not eliminate quadratic growth. By integrating mathematical derivation with computational validation, this work provides a principled reference for comparing architectures, clarifying the interplay between asymptotic theory and observed scaling. The unified perspective contributes to both computer science and applied mathematics by situating complexity analysis at the core of model evaluation, offering guidance for the design of efficient deep learning systems.