Predicting adaptability of students in online entrepreneurship education using deep learning

Data collection

In this study, we employed the dataset introduced in the work of Hasan Suzan et al. (2021). The data were gathered through a combination of online and offline surveys to ensure diversity among participants. The survey covered learners across different educational levels, including high school students, college students, and university students. The collection process was carried out between December 10, 2020 and February 5, 2021, resulting in a total of 1,205 valid records after removing incomplete or duplicate responses. This dataset is particularly valuable because it captures the factors influencing students’ adaptability in the context of online learning, which plays a critical role in shaping their engagement and performance in online entrepreneurship education. The dataset consists of 14 input features along with1 target variable (adaptability level in online learning). These attributes reflect demographic, socio-economic, infrastructural, and academic aspects of the learners. Specifically, they include:

• Age and Gender: basic demographic information.

• Education Level: distinguishes between school, college, and university students.

• Type of Institution: differentiates public and non-public institutions.

• Geographic Location: captures regional variations in access to online education.

• IT-related background: whether the participant is an IT student or not.

• Prior Educational Background: academic preparedness of the learner.

• Load Shedding Level and Internet Quality: infrastructural conditions affecting access.

• Class Time: reflects flexibility or constraints in scheduling online classes.

• Family Economic Condition: socio-economic factor influencing access to resources.

• Device Type: such as computer, smartphone, or tablet used for attending classes.

• Institutional learning management system (LMS) Availability: indicates whether the institution provides its own learning management system.

To provide a clearer understanding of the dataset, we illustrate the distribution of key influencing factors. Specifically, Fig. 1 presents the analysis of environmental factors, such as geographic location, economic condition, and load shedding level, which directly affect students’ access to online learning platforms. Figure 2 highlights the technological infrastructure aspects, including device type, internet type, and network type, showcasing how technological readiness influences online learning adaptability. Finally, Fig. 3 (Adaptability Level Analysis) provides a comprehensive view of learners’ adaptability scores, derived from gender, education level, and financial condition, serving as the target variable in our study. These attributes are especially relevant when analyzing adaptability in online entrepreneurship education, where factors such as reliable digital infrastructure, economic background, and prior academic preparedness strongly influence students’ ability to engage with entrepreneurial content, participate in collaborative tasks, and apply innovative ideas in a virtual learning environment.

Data preprocessing

As the survey was originally conducted in the Bengali language, all responses were manually translated into English to facilitate subsequent processing. String-type attributes were standardized and then converted into numerical values for model training. The target variable (adaptability level) was encoded with values 0, 1, and 2, corresponding to low, moderate, and high adaptability, respectively. Additional preprocessing steps such as data cleaning, removal of invalid responses, and handling of missing values were performed to ensure dataset quality. In summary, the final dataset consists of 1,205 samples, each represented by 14 features and one target label. By encompassing demographic, socio-economic, infrastructural, and academic factors, the dataset provides a strong foundation for examining and predicting students’ adaptability in online education. The original class distribution was moderately imbalanced, with the majority of learners falling into the moderate adaptability category, followed by low, and a relatively small proportion in the high group. To ensure robust model evaluation, the dataset was split into training, validation, and test sets in proportions of approximately 80%, 10%, and 10%, respectively. Table 1 summarizes the class distribution across the original dataset and each subset. This splitting helps reduce the risk of overfitting while enabling the model to be trained, tuned, and validated in an objective manner. The training set is used to learn and capture the underlying patterns, the validation set supports hyperparameter tuning and model selection, and the test set is reserved for final performance evaluation, ensuring that the reported results accurately reflect the model’s generalization ability on unseen data.

Support vector machine

Support Vector Machine is a powerful classification technique that aims to identify an optimal separating hyperplane with the maximum margin between different classes. For a binary classification problem, the decision function can be formulated as:

(1) $$f(x)=\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}(\mathbf{w}\cdot \mathbf{x}+b),$$where $\mathbf{w}$ denotes the weight vector and $b$ is the bias term. The margin is maximized by solving the following optimization problem:

(2) $$\underset{\mathbf{w},b}{min}\frac{1}{2}||\mathbf{w}|{|}^2\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}{y}_i(\mathbf{w}\cdot {\mathbf{x}}_i+b)\ge 1,\mathrm{\forall }i.$$Since our dataset involves three adaptability classes, we adopted the one-vs.-one strategy, where multiple binary classifiers are trained and combined to handle the multi-class scenario effectively.

Decision tree

Decision Tree classifiers construct a tree-like structure in which internal nodes represent tests on features, and leaf nodes indicate class labels. The splitting criterion is generally based on entropy and information gain. Entropy of a dataset S is defined as:

(3) $$H(S)=-\sum _{i=1}^n{p}_i{\log }_2{p}_i,$$where $p_i$ is the probability of an instance belonging to class $i$. The information gain for splitting on attribute A is:

(4) $$Gain(S,A)=H(S)-\sum _{v\in Values(A)}\frac{|S_v|}{|S|}H(S_v).$$This recursive partitioning continues until a stopping condition is reached, yielding a human-interpretable tree structure that is highly suitable for categorical and tabular data.

Random forest

Random Forest is an ensemble learning method that aggregates predictions from multiple decision trees to improve accuracy and reduce overfitting. Each tree is trained on a bootstrap sample of the data, and random subsets of features are considered at each split. The final prediction is obtained by majority voting across all trees. The margin function of RF can be expressed as:

(5) $$\overline{r}(X,D_n)=E_{\mathrm{\Theta }}[r_n(X,\mathrm{\Theta },D_n)],$$where $\mathrm{\Theta }$ represents the randomization in tree construction and $E_{\mathrm{\Theta }}$ is the expectation over different trees. By averaging across diverse trees, Random Forest enhances robustness and generalization compared to individual decision trees.

K-nearest neighbor

KNN is a non-parametric and instance-based learning algorithm. Classification is performed by finding the $k$ closest data points to a query instance according to a distance metric, typically Euclidean distance. The predicted class corresponds to the majority class among the nearest neighbors. The Euclidean distance between two data points $p$ and $q$ in $n$-dimensional space is given by:

(6) $$d(p,q)=\sqrt{\sum _{i=1}^n{(p_i-q_i)}^2}.$$KNN is computationally simple and interpretable, making it effective for small to medium-sized datasets where local relationships among samples strongly influence classification outcomes.

Light gradient boosting machine (LightGBM)

Light gradient boosting machine (LightGBM) is a gradient boosting framework that constructs an ensemble of weak learners (decision trees) in a sequential manner. Unlike traditional boosting methods, LightGBM employs a leaf-wise growth strategy with depth constraints, enabling it to handle large-scale data efficiently. The objective function combines a differentiable loss function L and a regularization term $\mathrm{\Omega }$ to prevent overfitting:

(7) $$Obj=\sum _{i=1}^{n}L(y_i,{\hat{y}}_i)+\sum _{k=1}^K\mathrm{\Omega }(f_k),$$where $y_i$ and ${\hat{y}}_i$ are the true and predicted labels, respectively, and $f_k$ denotes the $k$-th decision tree in the ensemble. LightGBM’s efficiency in handling high-dimensional categorical features and its ability to capture complex non-linear patterns make it a strong candidate for predicting student adaptability levels.

Approach to model selection

Although traditional ML algorithms such as SVM, DT, RF, KNN, and LightGBM have shown effectiveness in tabular classification tasks, they exhibit limitations when applied to predicting students’ adaptability in online learning. Linear models and tree-based methods often struggle to capture complex non-linear relationships, while ensemble and boosting approaches, despite improving accuracy, remain insufficient in modeling long-range dependencies and higher-order feature interactions across diverse attributes such as demographic, socio-economic, infrastructural, and academic factors. Moreover, most existing approaches treat features independently or rely on simplistic aggregation strategies, ignoring the intricate interplay between diverse data modalities that is critical for understanding adaptability in online entrepreneurship education. This gap indicates a clear need for models capable of jointly capturing feature heterogeneity, long-range dependencies, and non-linear interactions in tabular data.

To overcome these limitations, we propose TabNSA (Eslamian & Cheng, 2025), a deep learning framework specifically designed for tabular data. TabNSA integrates Native Sparse Attention to efficiently capture long-range dependencies among features, and TabMixer to model intricate non-linear interactions between heterogeneous attributes. This hybrid design ensures both scalability and enhanced representational power, enabling more accurate and comprehensive prediction of adaptability levels in the context of online entrepreneurship education.

Model description

To accurately predict students’ adaptability in the context of online entrepreneurship education, we propose a hybrid deep learning framework named TabNSA (Eslamian & Cheng, 2025). This model is tailored for tabular data and is designed to efficiently capture both long-range dependencies among features and complex non-linear feature interactions. The architecture integrates Native Sparse Attention (NSA) (Yuan et al., 2025) and TabMixer modules, thus balancing global contextual modeling with effective feature mixing. These modules will be further explained in the following subsections.

Given an input batch of tabular data represented by matrix $X\in {\mathbb{R}}^{B\times N}$, where B is the batch size and N denotes the total number of features extracted from student records, the TabNSA model undergoes a series of transformations to produce the final prediction of adaptability. The overall design is illustrated in Fig. 5.

Feature embedding: The input features consist of both numerical and categorical types, each requiring tailored processing to produce a unified embedding representation. Numerical features are first normalized and then projected into a high-dimensional embedding space of dimension D using a shared linear transformation:

(8) $$X_{\mathrm{n}\mathrm{u}\mathrm{m}}\in {\mathbb{R}}^{B\times N_{\mathrm{n}\mathrm{u}\mathrm{m}}}\stackrel{Linear(1\to D)}{⟶}{E}_{\mathrm{n}\mathrm{u}\mathrm{m}}\in {\mathbb{R}}^{B\times N_{\mathrm{n}\mathrm{u}\mathrm{m}}\times D}.$$For categorical features, one-hot encoding are applied to convert each categorical value into a fixed 1-dimensional scalar representation. Subsequently, each scalar categorical feature is linearly projected into the same D-dimensional embedding space as numerical features through a shared linear layer:

(9) $$X_{\mathrm{c}\mathrm{a}\mathrm{t}}\in {\mathbb{R}}^{B\times N_{\mathrm{c}\mathrm{a}\mathrm{t}}}\stackrel{Encoding\to Linear(1\to D)}{⟶}{E}_{\mathrm{c}\mathrm{a}\mathrm{t}}\in {\mathbb{R}}^{B\times N_{\mathrm{c}\mathrm{a}\mathrm{t}}\times D}.$$This approach ensures a consistent embedding dimension D for both numerical and coded categorical features without using a dedicated embedding lookup table. The embedded numerical and categorical features are then concatenated along the feature dimension to form the complete embedding tensor:

(10) $$E=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}(E_{\mathrm{n}\mathrm{u}\mathrm{m}},E_{\mathrm{c}\mathrm{a}\mathrm{t}})\in {\mathbb{R}}^{B\times N\times D},\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}N=N_{\mathrm{n}\mathrm{u}\mathrm{m}}+N_{\mathrm{c}\mathrm{a}\mathrm{t}}.$$

Native Sparse Attention: The embedded features E are processed via the NSA module, which addresses the computational challenges of standard self-attention’s quadratic complexity in feature dimension. NSA employs three core mechanisms:

• Sliding-window local attention: Focuses on capturing dependencies among neighboring features to model local context effectively.

• Block-wise compression: Aggregates non-overlapping blocks of features to reduce dimensionality and computational expense.

• Block selection: Selects only the most informative blocks for further processing, thus maintaining efficiency while preserving key information.

Through these components, NSA models long-range inter-feature relationships with linear complexity, making it scalable for high-dimensional tabular datasets.

TabMixer module: Parallel to the attention mechanism, the embedded features are fed into TabMixer, an architecture inspired by recent MLP-Mixer frameworks. TabMixer facilitates both intra-feature and inter-feature interactions via:

• Channel-wise MLP: Processes each feature embedding vector independently to learn complex patterns within feature dimensions.

• Token-wise MLP: Operates across the feature tokens to capture interactions between different features.

Non-linear activation functions such as ReLU are employed to enhance the representational capacity of these MLP layers, enabling the model to effectively capture heterogeneous dependencies in educational data.

Fusion and aggregation: Outputs of the NSA and TabMixer modules, denoted $Y_{attn}$ and $Z_{mixer}$ respectively, are combined through element-wise summation:

(11) $$Y=Y_{attn}+Z_{mixer},Y\in {\mathbb{R}}^{B\times N\times D}.$$

To obtain a fixed-length vector representation summarizing all feature information, mean pooling is applied across the feature tokens:

(12) $$X=\frac{1}{N}\sum _{i=1}^N{Y}_{:,i,:}\in {\mathbb{R}}^{B\times D}.$$

Classification head: The aggregated embedding X is fed into a two-layer multilayer perceptron (MLP) with GeLU activations to compute the final adaptability prediction score or classification logits:

(13) $$\hat{y}=\mathrm{M}\mathrm{L}\mathrm{P}(X)\in {\mathbb{R}}^{B\times C},$$where C denotes the number of target classes or regression outputs. This classification head reduces the embedding dimension and produces the model’s final output corresponding to students’ adaptability levels.

Model configuration

Table 2 summarizes the key configuration details of the proposed TabNSA model, including embedding dimensions, NSA and TabMixer settings, and classification head parameters. These default values are chosen to balance model capacity and computational efficiency for tabular data prediction tasks.

Ablation study

To assess the impact of each architectural component in the proposed TabNSA model, we performed an ablation study focusing on the contributions of the NSA and TabMixer modules. Four model configurations were evaluated:

1. Baseline MLP: a simple two-layer feedforward neural network without attention or mixing mechanisms.

2. TabNSA (w/o NSA): the proposed architecture excluding the attention mechanism, retaining only the TabMixer module to evaluate the role of non-linear feature mixing.

3. TabNSA (w/o TabMixer): the model incorporating only the NSA module to isolate the effect of long-range dependency modeling.

4. Full TabNSA: the complete hybrid design combining both NSA and TabMixer modules.

All variants were trained using identical hyperparameter settings and optimization strategies. The ablation study aims to clarify how each module contributes to overall performance improvements in adaptability prediction, particularly in modeling complex dependencies and heterogeneous educational attributes.

Fusion strategy comparison in the fusion and aggregation module

To further evaluate the effectiveness of the proposed fusion mechanism in TabNSA, we conducted an additional experiment comparing three commonly used feature fusion strategies: element-wise summation, concatenation, and attention-based fusion. While element-wise summation offers computational simplicity and concatenation increases representational capacity, neither explicitly models the relative importance of different feature channels. In contrast, attention-based fusion adaptively reweights features, allowing the network to highlight more informative dimensions before aggregation.

Comparative analysis of attention mechanisms

To further investigate the effectiveness of the NSA module in modeling long-range dependencies, we conducted additional experiments comparing NSA with other widely used attention mechanisms: dot-product attention (Vaswani et al., 2017) and additive attention (Wu et al., 2021). The objective is to examine how each mechanism captures global contextual information and contributes to adaptability prediction across different student groups. For a fair comparison, we replaced the NSA module in the TabNSA architecture with either:

• Dot-Product Attention: standard scaled dot-product attention commonly used in transformers.

• Additive Attention: a learnable attention mechanism that computes a weighted sum of input features using trainable attention scores.

• NSA (baseline): the original sparse attention mechanism proposed in TabNSA.

All models retained the TabMixer module to maintain local feature interaction modeling. Hyperparameters, training schedules, and dataset splits were kept identical across all variants to ensure consistency.

Comparison with existing models

To establish a baseline reference, we compared TabNSA with a set of conventional machine learning algorithms widely applied to tabular data, including SVM, DT, RF, KNN, and LightGBM. These models were optimized using grid search to ensure competitive configurations. Table 3 summarizes the searched parameters and the final configurations selected for each algorithm. The comparison highlights the limitations of shallow or tree-based methods in capturing higher-order relationships and global contextual dependencies across diverse student features.

Beyond traditional baselines, we further benchmarked TabNSA against several state-of-the-art (SOTA) deep learning architectures for tabular data, namely TabNet (Arik & Pfister, 2019), NODE (Popov, Morozov & Babenko, 2019), FT-Transformer (Gorishniy et al., 2021), SAINT (Somepalli et al., 2021), DANets (Chen et al., 2021), DeepGBM (Ke et al., 2019), and TabTransformer (Huang et al., 2020). These models represent distinct paradigms of neural processing for tabular inputs, including attentive feature selection, differentiable decision trees, and transformer-based contextual encoding. For fairness, all models were trained on the same dataset split, using identical training epochs, learning rates, and batch sizes.