Temporal and modal contributions to smartphone-based multimodal driving behavior classification: a comparative study of classical, deep learning, and patch-based time series transformer models

Literature review on driving behavior classification

In recent years, research on the classification of driving behaviors has expanded rapidly, with various approaches utilizing different data sources and methodologies. Studies in the literature frequently employ OBD-II data, motion sensors such as accelerometers and gyroscopes, global positioning system (GPS)-based location data, and machine learning techniques to analyze driving behavior.

Andria et al. (2016) developed a low-cost data collection platform for automotive telemetry applications, including driving style analysis, fleet management, and fault detection. Lattanzi & Freschi (2021) applied support vector machine (SVM) and neural networks to classify safe and unsafe driving using in-vehicle sensor data. Malik & Nandal (2023) analyzed OBD-II signals to distinguish between safe and reckless driving, while Singh & Singh (2022) combined ECU and accelerometer data to classify driving behaviors into categories such as bad, normal, and aggressive. Kumar & Jain (2023) modeled ten driving styles based on fuel consumption, steering, speed, and braking patterns, utilizing OBD-II data without requiring additional sensors. Ameen et al. (2021) continuously recorded speed, RPM, throttle, and load to categorize driving behaviors into safe, normal, aggressive, and dangerous classes. Zou et al. (2022) predicted vehicle acceleration using features such as distance, speed, and acceleration, while Azadani & Boukerche (2022) reviewed driving behavior studies by data type, objectives, and modeling techniques. Liu, Wang & Qiu (2020) introduced a motion-capture-based system employing multiple miniature inertial measurement units (IMUs) for real-time driver motion tracking. Cendales, Llamazares & Useche (2023) examined links between driving stress and risky behavior. Martinelli et al. (2020) used controller area network (CAN) bus and OBD-II data to classify driving styles, achieving 99% accuracy in identifying drivers.

Recent studies have also focused on fuel efficiency and driver categorization. Rastegar et al. (2024) analyzed driving patterns based on fuel consumption and powertrain variables such as acceleration and deceleration. Canal, Riffel & Gracioli (2024) applied machine learning to ECU data for fuel efficiency classification and consumption prediction. Mohammed et al. (2023) designed an electronic card system for remote vehicle monitoring. Yen et al. (2021) used deep learning with a universal OBD-II module supporting various CAN standards to analyze fuel use under different conditions. Fafoutellis et al. (2023) investigated the impact of the COVID-19 on driving behavior, identifying three main profiles: aggressive, eco, and typical.

Several studies have focused on electric and hybrid vehicles. Lee & Yang (2023) analyzed OBD-II data from electric and hybrid vehicles using deep learning. Pirayre et al. (2022) studied behavior across road types (e.g., urban vs. highway) with GPS-based Markov chain modelling, while Lin, Zhang & Chang (2023) combined geographic data with OBD-II and CAN bus recordings to extract image-based driving patterns. Zhang & Lin (2021) integrated video, road signs, and GPS data to identify critical visual indicators of driving behavior.

Advancements in deep learning and artificial intelligence have further improved classification accuracy. Merenda et al. (2022) trained convolutional neural network (CNN) models, while Yarlagadda & Pawar (2022) analyzed real-time performance features, and Li, Lin & Chou (2022) evaluated risky driving using fuzzy inference systems and long short-term memory (LSTM) networks. Al-Rakhami et al. (2021) proposed an edge-fog-cloud deep learning frameworks that improved efficiency and accuracy 1.84%. Tripicchio & D’Avella (2022) combined Bayesian network with long short-term memory (LSTM) and SVM, achieving 92% accuracy for advanced driver assistance systems (ADAS) application. Fadzil et al. (2026) analyzed online OBD-II datasets from Kaggle to identify driver groups based on driving styles. Lastly, Fattahi, Golroo & Ghatee (2023) developed a smartphone-based system for detecting aggressive maneuvers using speed, RPM, and accelerometer data, demonstrating the feasibility of cost-effective driver monitoring.

Research gaps in driving behavior analysis

A holistic examination of recent studies on driving behavior analysis through engine operations reveals several gaps in the literature. Some studies (Andria et al., 2016; Ameen et al., 2021; Mohammed et al., 2023; Pirayre et al., 2022; Lin, Zhang & Chang, 2023; Zhang & Lin, 2021) incorporate GPS sensors; however, their usage is often limited to acquiring only speed and location data, overlooking other potential insights that could be derived from GPS-based analysis. Additionally, several studies utilize intermediate hardware components such as Raspberry Pi (Andria et al., 2016), Arduino (Singh & Singh, 2022), peripheral interface controller (PIC) microcontrollers, and ESP32 modules (Ameen et al., 2021; Mohammed et al., 2023) to record, store, and process ECU data. While these external devices facilitate data acquisition, their integration introduces higher costs and installation complexities compared to smartphones. Given that smartphones are widely used, equipped with multiple built-in sensors, and offer seamless data connectivity, leveraging them for driving behavior analysis presents a cost-effective and practical alternative to dedicated hardware solutions.

To address the limitations identified in existing research, this study expands the use of GPS data beyond speed and location by incorporating additional contextual features such as road type, speed limit, number of lanes, and road surface conditions via the Overpass application programming interface (API). This enhancement allows for a more comprehensive evaluation of driving behaviors by considering not only vehicle dynamics but also environmental factors. Furthermore, unlike previous studies that rely on external hardware for ECU data acquisition, this research collects data directly through a smartphone and an OBD-II connection. This approach eliminates the need for additional recording devices, reducing both cost and installation complexity. Additionally, by leveraging built-in smartphone sensors (such as the accelerometer, gyroscope, and magnetometer) as well as environmental audio signals, driving behavior analysis becomes more robust and multidimensional. Ultimately, this study introduces a cost-effective and scalable driving behavior analysis model that integrates multiple data sources, providing a more comprehensive and accessible alternative to existing systems.

Moreover, while numerous studies have explored traditional machine learning algorithms and basic deep learning architectures such as CNNs and recurrent neural networks (RNNs) for driving behavior classification, there remains a noticeable gap in the application of more recent and advanced sequence modeling techniques. In particular, transformer-based models that leverage attention mechanisms (such as vision transformers (ViT), temporal fusion transformers (TFT), and PatchTST) have not been extensively adopted in this domain. These models offer significant advantages in capturing long-range temporal dependencies and complex multimodal interactions, which are critical for accurately characterizing dynamic driving behaviors. The lack of utilization of such architectures in prior research highlights a methodological gap that this study aims to address through the integration and comprehensive evaluation of attention-based models in driving behavior analysis.

Limitations and optimization strategies

One of the primary limitations of this study is the high computational demand required to process data from motion sensors and audio signals. In particular, the extraction of frequency-based features from audio signals and the continuous processing of motion data imposes significant computational costs on mobile devices. Additionally, the reliance on the Overpass API, which plays a crucial role in enriching GPS data, necessitates an active internet connection, potentially limiting the system’s usability in offline environments. Moreover, transformer-based models such as PatchTST, while offering improved temporal modeling capabilities, typically require large amounts of data and computational resources to achieve optimal performance. This can pose further challenges for real-time deployment on resource-constrained platforms such as mobile devices.

Another important limitation concerns the taxonomy employed in this study, which was restricted to three categories. While real-world driving behavior encompasses additional conditions such as distraction, drowsiness, reckless maneuvers, or eco-driving, these could not be reliably incorporated without additional modalities (e.g., driver-facing cameras, eye-tracking) and standardized labeling procedures. The present taxonomy should therefore be regarded as a baseline that can be practically deployed with smartphone-only sensing, while also remaining extendable to finer-grained states in future investigations.

A further limitation is that the experimental design was restricted to short-window benchmarks conducted on data collected from a single vehicle and driver. Cross-driver, cross-vehicle, and cross-environment validation (essential for assessing model generalizability) were not performed due to dataset constraints. Similarly, aspects such as robustness to sensor noise, computational energy efficiency on mobile platforms, and interpretability of deep and transformer-based models were not addressed in the current study. These omissions limit immediate deployment potential but provide a clear direction for subsequent investigations.

To mitigate these challenges, several optimization strategies can be implemented. Adjusting data refresh frequencies, employing longer window intervals in classification processes, and performing less frequent but more comprehensive analyses over extended time periods can help reduce computational load. These approaches not only enhance the system’s efficiency on mobile devices but also contribute to the sustainability of real-time analysis by balancing performance and resource utilization.

Proposed approach and contributions

This study introduces a novel, cost-effective, and multimodal framework for driving behavior classification by leveraging the synergy between vehicle ECU data, smartphone-based motion sensors, GPS-derived road metadata, and environmental audio signals. Unlike many prior works that rely on external microcontrollers, limited sensor inputs, or handcrafted features, the proposed system emphasizes accessibility, scalability, and real-time analysis through a unified mobile application platform.

The core innovation of this study lies in its seamless integration of four distinct sensor domains: (1) engine operational data via the OBD-II interface, (2) inertial motion data from accelerometer, gyroscope, gravity, and magnetometer sensors, (3) contextual road information retrieved dynamically from the Overpass API, and (4) spectral and statistical features extracted from environmental sound recordings. This diverse sensor fusion enables the system to capture not only vehicle and driver dynamics but also environmental and contextual cues, leading to a more holistic assessment of driving behaviors.

The present study was designed to address style-level driving behavior rather than cognitive driver states. A coarse-grained taxonomy was selected to align with the modalities available from a smartphone-centric sensing pipeline and to ensure reliable labeling. Categories such as distracted or drowsy usually require driver-facing measurements (e.g., gaze or eyelid closure) and standardized annotation protocols; incorporating such states without the appropriate modalities would risk label noise and construct conflation. Consequently, the three-class taxonomy is positioned as an extendable foundation for telematics applications, compatible with future refinements once additional sensing and validated ground-truth resources become available.

This study is driven by the following research questions: (1) To what extent can smartphone-centric multimodal sensing provide reliable classification of driving behaviors? (2) How do classical machine learning, deep learning, and transformer-based models compare in terms of accuracy and robustness across different temporal window lengths and feature subsets? (3) What is the incremental contribution of each sensing modality to overall classification performance? These questions structure the experimental design and guide the interpretation of findings.

To represent these multimodal signals, the study proposes two complementary data preparation pipelines: one for classical machine learning and another for deep learning. In the classical pipeline, 66 handcrafted features were engineered from motion, engine, road, and audio signals using statistical and domain-specific descriptors. In contrast, the deep learning and PatchTST pipeline preserved the raw temporal structure of sensor streams, forming T × 37 matrices from time-windowed segments of raw data (with T varying according to the selected window duration). This dual approach enabled a comparative analysis of performance across traditional and modern classification paradigms.

Crucially, the study systematically evaluated the effects of time window length (3, 5, 7, and 9 s) and feature subset composition (ranging from ECU-only data to full sensor fusion) on classification performance. By structuring the dataset into 16 experimental configurations (spanning all combinations of window lengths and feature subsets) fine-grained insights were obtained regarding the temporal resolution required for accurate behavior modeling and the incremental benefit of adding each sensor modality. The impact of these variables was not only evaluated through performance metrics (accuracy, precision, recall, F1-score, area under the curve (AUC)), but also substantiated with detailed statistical comparisons between configurations, providing robust evidence of the performance gains resulting from longer window durations and multimodal integration. This study advances the field by introducing a smartphone-centric multimodal framework that unifies OBD-II data, motion sensing, enriched road metadata, and environmental audio into a single dataset. Unlike prior work relying on limited modalities or external devices, the present design enables cost-effective, scalable data acquisition. Furthermore, by systematically comparing classical machine learning (ML), deep learning, and attention-based transformer models, the study provides one of the first evaluations of transformer architectures for multimodal driving behavior analysis. This combination of modality integration, smartphone-based deployment, and advanced modeling establishes a new benchmark for future research in intelligent transportation systems. The major contributions of this study are summarized as follows:

1. Standardized multimodal dataset construction: This study introduces one of the first smartphone-centric multimodal datasets that combines ECU, motion, road metadata, and audio in a standardized collection framework.

2. Dynamic road context integration via Overpass API: By leveraging the Overpass API, the study dynamically retrieves contextual road metadata (such as speed limits, road type, lane count, one-way status, surface condition, and toll information) for each GPS trace.

3. Systematic evaluation of temporal granularity: Driving sequences were segmented into sliding windows of 3, 5, 7, and 9 s to explore the effect of temporal resolution on classification performance.

4. Comprehensive benchmarking of classical and deep learning models: The study compares engineered-feature-based classical models (SVM, LR, NB, ANN) with raw-sequence-based deep learning architectures (CNN, RNN, LSTM, GRU).

5. Modal contribution analysis across sensor domains: The individual and joint effects of motion sensors, road metadata (from Overpass), and environmental audio signals on classification accuracy are systematically assessed.

6. Integration of time and frequency domain audio features: Thirteen handcrafted audio features (spanning both time and spectral domains) were extracted from environmental sound recordings to capture subtle contextual cues (e.g., road noise, acceleration intensity).

7. Cost-effective and scalable mobile-based acquisition pipeline: The entire system runs on a consumer-grade Android smartphone connected to a low-cost OBD-II Bluetooth adapter, eliminating the need for microcontrollers like Raspberry Pi or Arduino.

8. Integration and evaluation of advanced Transformer-based architectures (PatchTST): This study pioneers the integration of PatchTST, a state-of-the-art transformer-based time-series classification model utilizing patch-level attention mechanisms, into the domain of driving behavior analysis.

Through these contributions, the proposed approach advances the field of intelligent transportation systems by demonstrating that low-cost, mobile-sensor-driven architectures can achieve high accuracy in driving behavior classification when supported by multimodal fusion, window-aware modeling, and statistically validated experimental designs.

Sensor framework

Sensor axes enable the data obtained from different sensors to be represented within a static reference frame, relative to the physical location of the device. The Android sensor API defines this coordinate system based solely on the natural orientation of the device’s screen. Notably, the sensor axes remain fixed even if the screen orientation of the device changes (Android, 2025). Figure 1 illustrates the coordinate system utilized by the Android sensor API.

Accelerometer

The accelerometer measures acceleration along the X, Y, and Z axes in units of G (9.81 m/s²), capturing both linear acceleration and gravitational force (Sağbaş, Korukoglu & Balli, 2020). Figure 2 illustrates variations in Z-axis acceleration across calm, normal, and aggressive driving, demonstrating its utility for behavior discrimination.

The linear acceleration sensor isolates motion by subtracting gravity from the accelerometer signal. When the device is stationary, its values should approach zero. Depending on hardware, it either fuses data from the gyroscope and accelerometer or, in the absence of a gyroscope, relies on accelerometer–magnetometer combinations (Android, 2025). The gravity sensor outputs the direction and magnitude of Earth’s gravity. Like the linear acceleration sensor, its calculation depends on device hardware, using either gyroscope–accelerometer fusion or accelerometer–magnetometer data (Android, 2025).

Gyroscope

The gyroscope sensor measures the angular velocity of the smartphone along the three axes, as shown in Fig. 1. The raw data obtained from the gyroscope sensor represents the rotational motion of the device around the X, Y, and Z axes, expressed in radians per second (rad/s) (Ballı, Sağbaş & Peker, 2019a). Example gyroscope data for the X axis, corresponding to the three driving behavior classes considered in this study, is presented in Fig. 3.

Magnetometer

The magnetometer measures the strength of the magnetic field surrounding the device and provides data in microtesla (µT) units across the X, Y, and Z axes. These values are normalized within the range of −128 to +128. The total magnetic field detected by the device is the vector sum of the Earth’s geomagnetic field and the local magnetic fields surrounding the device. The magnetometer can function as a digital compass, and when integrated with accelerometer and gyroscope data, it allows for the detection of the device’s real-time movement and directional deviations (Sağbaş & Ballı, 2015).

Global positioning system (GPS)

The Global Positioning System (GPS), developed by the US Department of Defense in the 1970s, is a passive satellite-based navigation system that provides location and time information worldwide under all weather conditions (El-Rabbany, 2002). Modern smart devices determine location primarily via the GPS sensor, but can also use wireless networks and base stations with lower accuracy when GPS is unavailable. GPS signals include parameters such as latitude (positive north, negative south), longitude (positive east, negative west), speed (instantaneous velocity, invalid if negative or unavailable), and altitude (height above sea level in meters) (Ballı, Sağbaş & Peker, 2019b).

Overpass API

The Overpass API is a database system developed to efficiently query and manage data from the OpenStreetMap (OSM) project (Ramm, 2008). It updates global OSM data with minimal delay (typically only a few minutes) allowing users to access this information via the web. One of the primary goals of OSM is to foster new and innovative uses of geographic data. However, certain user queries may be too complex or specialized for traditional APIs that are primarily designed for map creators and geographic information editors. These challenges led to the development of the Overpass API (Olbricht, 2012). The Overpass API is optimized for users who need to perform specific and often complex queries against OSM data. It allows users to efficiently access data for specialized analysis and data processing projects that extend beyond the capabilities of typical map creation. The API’s design ensures that users can handle unique data retrieval needs, making it a valuable tool for research, development, and other specialized applications involving geographic information (Sağbaş, 2024).

OBD-II and ELM327

On-board diagnostics (OBD) has been mandatory in all light-duty vehicles since 1996, providing standardized monitoring of engine performance and emission control systems (Aris et al., 2007; Süzen & Kayaalp, 2018). Communication protocols are defined under ISO 15031 and SAE J1962, enabling external tools to access ECU data for diagnostics. OBD-II supports multiple protocols, including the CAN bus, allowing reliable collection of engine performance data essential for vehicle health monitoring (Kumar & Jain, 2022).

The ELM327 microcontroller, developed by ELM Electronics, facilitates communication between the OBD-II port and external devices by abstracting protocol complexity into a simple universal asynchronous receiver–transmitter (UART) interface. It supports universal serial bus (USB), Bluetooth, and wireless fidelity (Wi-Fi) connections, with wired USB offering greater reliability (Carignani et al., 2015). The multi-protocol device accommodates a wide range of OBD-II standards, such as SAE J1939, J1850 PWM, ISO 14230-4 KWP, ISO 15765-4 CAN, and ISO 9141-2 (Kumar & Jain, 2023). Through these protocols, parameters such as engine load, RPM, fuel system status, speed, coolant temperature, manifold pressure, airflow rate, and intake air temperature can be retrieved. In this study, the ELM327 device (Fig. 4) was employed for real-time data visualization and recording.

Android application and data logging setup

To classify driving behaviors, data regarding vehicle engine operations and various sensor data from the smartphone were collected. A specialized Android-based mobile application was developed to continuously and synchronously record this data. The primary goal of the application was to promote safe and economical driving practices. During the experiment, the smartphone was kept in a fixed position to ensure consistent data collection (Fig. 5).

The developed application utilizes the ELM327 adapter via the OBD-II port to obtain engine operation data from the vehicle. The data retrieved from the ECU is updated once per second. Table 1 presents the engine parameters, related parameter ID (PID) codes, and units used in this study.

In addition, raw data obtained from the three-axis accelerometer, gyroscope, linear acceleration sensor, gravity sensor, and magnetometer were processed to analyze the dynamic movements and posture of the vehicle. The data sampling rate of the motion sensors was configured to provide 50 data points per second, ensuring high temporal resolution for capturing detailed motion and orientation changes during the driving process.

In addition to the data obtained from the ECU and motion sensors, latitude, longitude, altitude, and speed data were recorded from the GPS sensor to determine the location information of the vehicle and provide detailed information about the road conditions. The GPS data was integrated with the Overpass API and enriched with essential information such as the type of road the vehicle is on, legal speed limits, number of lanes, whether it is one-way, road surface type, and whether it requires tolls. This enriched data was used to create a warning system within the developed mobile application, offering real-time feedback to the driver and promoting safe driving practices. Screenshots of the developed application are shown in Fig. 6.

Finally, as an additional factor to enhance the accuracy of the model, environmental audio data was collected. To prevent potential inconsistencies in the synchronization of textual and audio data, audio signals were not recorded in their raw form but rather by extracting specific features. These extracted features will be discussed in detail under the relevant analysis section. During the processing of the audio signals, a sampling frequency of 44.1 kHz was chosen, and the buffer size was set to 2,048.

The application provides the functionality to save the collected data in comma-separated values (CSV) format, with the file name being specified by the user (as shown in the last line of Fig. 6). These recorded data serve as a rich resource for analyzing the vehicle’s usage patterns and evaluating driving behaviors. The general flowchart outlining the data acquisition process for driving behavior classification is presented in Fig. 7.

The developed mobile application can be utilized by individual drivers to receive real-time feedback on their driving habits, as well as by fleet managers to monitor driver performance and ensure compliance with safety standards. Additionally, the system’s ability to classify driving behaviors enables its integration into telematics-based insurance models, allowing insurance providers to assess driving risks more accurately and offer customized pricing based on real-time behavior.

To ensure transparency and reproducibility, the complete source code of the mobile application used for data acquisition is publicly available. DrivingHelper Application: Android-based application that collects real-time sensor data from vehicle ECU and smartphone sensors. Repository link: https://github.com/arifsagbas/DrivingHelper (https://doi.org/10.5281/zenodo.17286884).

Classical feature extraction

Raw input configuration for deep learning

A total of 37 raw sensor columns were selected for deep learning models, without manual feature extraction, to provide a comprehensive view of vehicle dynamics, environmental context, and auditory feedback. These were grouped into four categories:

Motion sensors (15 columns): Tri-axial data from accelerometer, linear acceleration, gravity, gyroscope, and magnetometer, capturing fine-grained motion dynamics.

ECU data (five columns): Speed, engine load, RPM, mass air flow (MAF), and manifold absolute pressure (MAP), reflecting engine performance and driver interaction.

Road metadata (four columns): Attributes from the Overpass API (maxspeed, highway type, oneway status, and surface condition) providing environmental and infrastructural context.

Audio features (13 columns): Low-level descriptors (RMS, zero-crossing rate, spectral centroid, entropy, dominant frequency, etc.) derived from raw recordings, adding complementary auditory cues (e.g., tire or engine noise).

These 37 columns formed the unified raw input matrix for all deep learning models, allowing automatic learning of nonlinear temporal and contextual patterns across motion, mechanical, environmental, and auditory modalities.

Data normalization and encoding strategies

Prior to training classical machine learning models, a set of preprocessing steps was applied to the extracted feature dataset to ensure that all variables were represented in a format suitable for numerical algorithms. These operations included data type conversions, normalization of continuous features, and encoding of categorical variables. Firstly, all continuous attributes were normalized using z-score standardization to bring them to a common scale, thereby eliminating potential biases caused by varying units and magnitudes. This step was essential for ensuring the effectiveness and convergence of distance-based and gradient-based learning algorithms. For categorical data preprocessing, the speed trend, one-way, and lanes attributes (originally in categorical form) were transformed using label encoding, which maps each category to an integer value. The lanes feature was further transformed using one-hot encoding due to its limited and nominal nature, allowing the model to interpret each lane configuration as an independent binary feature. The highway type feature was encoded via frequency encoding, which replaces each category with its relative frequency in the dataset. This approach was chosen to preserve the global distributional information of road types while maintaining numerical compatibility. As a result of these transformations, the number of engineered features derived from Overpass API metadata increased from five to seven. This enriched representation aimed to enhance the descriptive power of the road environment context in the classification task. All transformations were applied globally across the full dataset to ensure consistency and were performed prior to model training in a reproducible preprocessing pipeline.

Unlike traditional machine learning methods that rely on handcrafted feature extraction, the deep learning models developed in this study utilize raw sensor data as direct input. However, in order to ensure model convergence and training stability, basic preprocessing steps were applied to the 37-column raw input matrix described in ‘Raw input configuration for deep learning’. These steps involved global normalization of continuous variables and label encoding of categorical features, preserving the raw temporal structure of each sample while standardizing input scales across the dataset. To this end, all CSV files corresponding to 3, 5, 7, and 9-s time windows were loaded from three behavior-specific folders (calm, normal, aggressive) and concatenated into a unified dataset. The dataset includes four categorical features, corresponding to road-related metadata extracted from the Overpass API: road type, oneway status, surface condition, and lane count. These categorical features were label-encoded using the LabelEncoder method, transforming each category into a unique integer value to ensure compatibility with embedding layers in deep learning architectures. The remaining 33 continuous features (originating from motion sensors, ECU parameters, and audio signals) were normalized using StandardScaler to yield zero-mean and unit-variance values across the entire dataset, a step critical for mitigating internal covariate shift and accelerating the training process. After encoding and normalization, the unified dataset was reshaped back into individual windowed samples and saved into behavior-specific folders, preserving the original time structure of each segment. No aggregation or manual feature engineering was applied at this stage, thereby allowing the deep learning models to autonomously learn hierarchical and temporal patterns directly from the preprocessed raw input. This standardized preprocessing approach was applied consistently across all time window configurations and driving behavior classes, ensuring methodological uniformity and reproducibility throughout the deep learning pipeline.

Results from machine learning models

For classification, a set of commonly used and literature-backed classical machine learning models were employed: ANN, SVM, LR, and NB. All experiments were conducted in a Python-based environment using deterministic seeds and 10-fold cross-validation to ensure statistically reliable and reproducible performance estimates. Table 4 presents the classification results (accuracy, precision, recall, and F1-score) for each combination of dataset, feature subset, and classifier, illustrating temporal granularity and sensor modality affect performance.

As shown in Table 4, richer feature subsets and longer time windows generally improve performance across most models. Moving from SET1 (ECU-only) to SET4 (including motion, Overpass, and audio data), a steady increase is observed in ANN, SVM, and LR metrics, whereas NB shows a slight decline due to its sensitivity to correlated and high-dimensional features. Longer windows (DS7, DS9) yield better results than shorter ones (DS3, DS5), as they capture more complete driving patterns and contextual transitions. ANN and SVM achieve the highest overall accuracies, benefiting from their ability to model non-linear relationships and adapt to higher feature dimensionality. In contrast, NB underperforms with complex inputs, while LR provides balanced but moderate accuracy with higher interpretability. The ranking across configurations remains consistent: ANN ≈ SVM > LR > NB. These findings confirm that multimodal sensor fusion and temporal context enhance classification performance, particularly for models capable of learning non-linear relationships. To provide a clearer understanding of how the models perform at the class level, confusion matrices corresponding to the best-performing configurations of each classifier are presented in Fig. 9.

Figure 9 illustrates the confusion matrices for the best configurations of each classifier, showing class-level prediction distributions. ANN (DS7–SET3) demonstrates the most balanced performance, effectively distinguishing calm and aggressive classes. SVM (DS9–SET4) also performs strongly but exhibits slightly higher confusion between normal and adjacent classes. LR achieves good accuracy but struggles in the normal class, while NB (DS9–SET1), limited to ECU data, fails to fully separate overlapping behaviors. Collectively, these results confirm that ANN offers the most reliable performance, followed by SVM; NB remains limited by feature simplicity. The receiver operating characteristic (ROC) curves corresponding to the best-performing configurations of these models are presented in Fig. 10, further illustrating their discriminative capabilities across the behavior classes.

Figure 10 shows the ROC curves of the best-performing configurations, illustrating class-wise discrimination. ANN (DS7–SET3) and SVM (DS9–SET4) achieve near-perfect separability (AUC ≈ 0.99 for aggressive and calm), while LR and NB perform slightly lower, particularly for the normal class. These results reaffirm that ANN and SVM provide the best overall discrimination, LR offers interpretable and moderate performance, and NB remains limited under simplified assumptions. The class-wise precision, recall, F1-score, and support values obtained from the best-performing configurations of each machine learning method are presented in Table 5.

As noted, Table 5 presents only the best-performing configuration for each model. While this highlights the maximum potential of individual algorithms, direct cross-model comparisons are limited since the underlying dataset/subset combinations differ. ANN achieved the highest accuracy (93.88%) on DS7–SET3, showing balanced performance across all classes. SVM (DS9–SET4) followed closely with 92.69%, maintaining strong generalization. LR achieved 91.32%, while NB reached 87.96%, performing best in aggressive and calm classes but struggling with the normal class. Overall, ANN achieved the best class balance, SVM offered stable generalization, and LR provided interpretable reliability, whereas NB was most sensitive to class overlap.

Results from deep learning models

In addition to classical models, this study also explored the effectiveness of deep learning architectures on raw multivariate time-series data. Instead of relying on handcrafted features, the models were trained directly on segmented sensor sequences with varying window sizes (DS3–DS9), each represented as a T × 37 matrix containing motion, ECU, road, and audio features. Based on their widespread popularity and proven success in time-series classification tasks, four deep learning models were selected for this study: CNN, RNN, LSTM, GRU. All models were implemented using the PyTorch with CUDA (RTX 4060) acceleration. Performance was evaluated using 10-fold cross-validation, with the primary metrics being accuracy, precision, recall, F1-score, and AUC. Training and validation accuracy/loss curves for the best-performing configurations of each model are presented in Fig. 11.

As shown in Fig. 11, all models converge effectively within 100 epochs under the full feature set (SET4). CNN and GRU exhibit the most stable and smooth convergence, with minimal overfitting and steady validation improvements. LSTM also performs strongly but shows mild overfitting after later epochs, while RNN converges more slowly and displays slightly noisier training curves, indicating limited robustness to temporal variations. Overall, CNN and GRU demonstrate the highest stability and generalization capacity under multimodal input. Table 6 presents the classification results for each combination of dataset, feature subset, and deep learning classifier.

As shown in Table 6, each of the four deep learning models was systematically evaluated under all dataset–subset combinations, yielding 64 results in total. This design ensures comparability across models under consistent experimental conditions. Among all models and configurations, CNN consistently achieves the highest performance, particularly in subsets SET2–SET4, confirming the advantage of multimodal fusion. For instance, CNN reaches 96.5% accuracy (F1 = 95.8%) on DS5–SET4, highlighting its strong pattern-learning ability. LSTM follows with slightly lower accuracy (91.1%), performing best on longer windows but showing underfitting in ECU-only configurations (SET1). GRU demonstrates performance comparable to LSTM and sometimes higher (e.g., 92.2% accuracy (F1 = 90.8%) on DS7–SET4) benefiting from efficient gating and generalization even with shorter input lengths. RNN achieves the lowest performance, particularly under limited features (e.g., DS9–SET1 F1 = 63.4%), but improves with richer inputs. Across all models, performance increases from SET1 to SET4 and with longer windows, confirming the benefits of multimodal integration and temporal richness. To provide a clearer understanding of how the models perform at the class level, confusion matrices corresponding to the best-performing configurations of each classifier are presented in Fig. 12.

As illustrated in Fig. 12, confusion matrices from the best-performing configurations of each deep learning model reveal distinct patterns in class-specific prediction behavior. CNN achieves the most balanced classification with minimal confusion across classes, followed closely by GRU. Both models effectively separate calm and aggressive behaviors, though normal class misclassifications persist across all networks. LSTM also performs well but shows moderate confusion in the normal class, while RNN exhibits the highest overall confusion, consistent with its difficulty in capturing long-term dependencies. Collectively, these results confirm that CNN and GRU deliver the most accurate and robust predictions, with CNN being the most consistent across all classes. The ROC curves corresponding to the best-performing configurations of these models are presented in Fig. 13, further illustrating their discriminative capabilities across the behavior classes.

As depicted in Fig. 13, the ROC curves and corresponding AUC scores validate these results. CNN achieves AUC = 1.00 for aggressive and calm, and 0.99 for normal, confirming near-perfect separability. GRU and LSTM follow closely (AUC ≈ 0.99–0.95), while RNN lags slightly, particularly for the normal class (AUC = 0.94). These findings reinforce that CNN and GRU provide the best class separability, while LSTM performs competitively and RNN remains the least effective under complex multimodal conditions. The class-wise precision, recall, F1-score, and support values obtained from the best-performing configurations of each method are presented in Table 7.

Table 7 summarizes the best configurations of each model. CNN (DS5–SET4) achieved the highest accuracy (96.5%), with outstanding F1-scores across all classes (0.97–0.98). GRU (DS3–SET4) followed with 92.8%, showing excellent efficiency even with shorter windows. LSTM reached 91.1%, performing strongly on calm and aggressive classes but weaker in normal. RNN achieved 88.4%, limited by its difficulty in learning long-term dependencies. Overall, CNN remains the top performer, while GRU offers a strong balance between performance and computational cost. LSTM maintains competitive accuracy, and RNN is the least suitable for complex temporal modeling.

Results from PatchTST models

To further investigate the performance of transformer-based models in driving behavior classification, a series of comprehensive experiments were conducted using the PatchTST architecture. Unlike traditional recurrent or convolutional models, PatchTST employs a patch-wise tokenization mechanism that enables efficient modeling of long-term temporal dependencies. Two latent dimensions (d_model = 64, 128) and four encoder depths (2–5 layers) were tested, producing eight model variants. Each variant was evaluated across four sensor configurations (SET1–SET4) and four window lengths (3, 5, 7, 9 s), resulting in 128 experiments. The results are summarized in Table 8, which outlines the performance trends across these configurations.

Table 8 shows that models with lower dimensionality (d_model = 64) and fewer encoder layers (2–3) consistently outperform deeper, higher-capacity variants. Deeper models with d_model = 128 often show reduced accuracy due to overfitting or instability. Performance improves with richer sensor inputs (particularly SET4, which yields the best results across all datasets) and with longer time windows (DS7, DS9), which provide broader temporal context. The best accuracy (96.29%) was achieved on DS9 using d_model = 64, 2 layers, and SET4, indicating that compact transformer architectures can deliver state-of-the-art accuracy with efficient computation. These results highlight the importance of architectural simplicity and multimodal richness in optimizing transformer-based time-series classification. In continuation of the analysis, Table 9 presents the detailed class-wise performance metrics (precision, recall, F1-score and support) corresponding to the highest-performing PatchTST configuration identified for each dataset, as reported in Table 8.

Table 9 provides class-wise metrics for the top-performing configuration in each dataset. Across all cases, PatchTST achieves high and balanced F1-scores, particularly for aggressive and calm behaviors. For example, on DS9, the best model (d_model = 64, 2 layers, SET4) achieves 96.29% accuracy with F1-scores above 0.96 in both classes. The normal class performs slightly lower due to its transitional nature, but overall class balance remains strong. These findings confirm that a properly tuned PatchTST model can generalize robustly across diverse driving behaviors. To support and further illustrate the quantitative results presented above, Fig. 14 provides the training and validation accuracy and loss curves for each dataset based on the corresponding best-performing PatchTST configuration.

Figure 14 shows training and validation curves of the best configurations. All models converge rapidly within the first few epochs, maintaining close alignment between training and validation losses. Minor fluctuations in DS7 and DS5 suggest slight instability in deeper models, but no evidence of severe overfitting is observed. In DS9 (SET4, d_model = 64, 2 layers), both training and validation accuracies exceed 95%, indicating excellent convergence and generalization. Overall, compact PatchTST models exhibit stable optimization and strong learning dynamics across datasets. To further analyze the model’s class-wise predictive performance, the confusion matrices of the best-performing configurations for each dataset are presented in Fig. 15.

The confusion matrices in Fig. 15 confirm the model’s consistent and balanced performance. Aggressive and calm behaviors are almost perfectly classified, with minor overlap in the normal class across datasets. DS3 and DS5 maintain strong diagonal dominance, while DS7 shows slightly more confusion due to transitional patterns. These results reinforce the PatchTST’s capacity to distinguish between distinct behaviors, particularly at the extremes. Figure 16 presents the ROC curves corresponding to the best-performing configurations for each dataset.

In Fig. 16, all AUC values exceed 0.98, indicating excellent separability across the three classes. In particular, DS9 and DS3 achieve near-perfect AUC scores of 1.00 for the aggressive class and 0.99 for calm and normal classes. The slightly lower but still impressive AUC values observed in DS5 and DS7 further validate the model’s robustness, even in configurations with higher dimensional complexity. These results align with the precision, recall, F1 metrics and confusion matrix analyses, reinforcing the model’s generalization capacity and consistent high performance across diverse sensor and temporal settings.

Performance evaluation of the proposed method

The experimental results reveal several important insights regarding the classification performance of both classical machine learning and deep learning models in the context of driving behavior recognition. Among classical models, ANN demonstrated the most consistent and superior performance across different datasets and feature subsets, achieving its highest accuracy of 93.88% on DS7 with SET3. SVM closely followed, with particularly stable results and a peak accuracy of 92.69% on DS9 with SET4. LR also performed competitively, especially when rich multimodal inputs were provided, reaching 91.32% accuracy on DS5 with SET4. In contrast, NB showed diminished performance with more complex feature sets, likely due to its assumption of conditional independence among features, but still achieved a strong result of 87.96% on DS9 with SET1.

When PatchTST experiments are evaluated in this context, further statistical analyses were conducted to investigate how architectural design choices and data configurations influence classification performance. A multifactorial analysis of variance (ANOVA) was applied to four categorical factors (dModel, layer, dataset, and subset) to determine their individual contributions to the observed accuracy scores. As shown in Table 10, all four variables yielded statistically significant p-values (p < 0.0001), confirming that both architectural depth and input diversity considerably impact model performance.

According to the ANOVA results, layer count was identified as the most influential factor, followed by dataset, subset, and dModel. To gain deeper insight into pairwise differences among group levels, Tukey’s Honest Significant Difference (HSD) test was applied. The post-hoc analysis of the subset factor, as presented in Table 11, revealed statistically significant improvements in accuracy between SET1 and both SET2 and SET4, demonstrating that incorporating richer sensor modalities notably enhances performance.

Regarding the dModel parameter, Table 12 shows that models using a lower latent dimension (dModel = 64) significantly outperformed those with dModel = 128, suggesting better generalization capacity and reduced risk of overfitting.

For the layer factor, Table 13 highlights a consistent drop in performance as model depth increased, with two-layer architectures achieving significantly higher accuracy than deeper counterparts. This underscores the importance of architectural simplicity in temporal modeling tasks.

Finally, in terms of dataset, Table 14 shows that DS9 consistently outperformed other datasets, indicating that longer time windows provide more informative temporal context and contribute positively to classification accuracy.

These findings collectively emphasize that careful tuning of architectural complexity (particularly limiting layer depth and dimensionality), combined with the strategic selection of feature subsets and temporal granularity, is critical to optimizing transformer-based models such as PatchTST for driving behavior recognition.

On the deep learning side, CNN outperformed all other models, achieving the highest overall classification accuracy of 96.50% on DS5 with SET4. This demonstrates the effectiveness of CNNs in learning discriminative spatial-temporal features from multivariate sensor sequences. Recurrent architectures, particularly GRU and LSTM, also yielded competitive results, with GRU achieving 92.84% accuracy on DS3 with SET4 and LSTM reaching 91.07% on DS5 with SET4. RNNs, while trailing slightly in overall accuracy, performed reasonably well in scenarios with shorter time windows or less complex input features. Overall, the results show that increasing the temporal resolution (i.e., using longer time windows such as in DS5 or DS9) and integrating multimodal sensor data (e.g., motion, overpass metadata, and sound in SET4) substantially improve classification performance. The combination of deep learning methods and rich input representations leads to more robust models capable of capturing the dynamic patterns underlying aggressive, normal, and calm driving behaviors.

Normality tests (Shapiro–Wilk) indicated that both machine learning (ML) and deep learning (DL) accuracy scores violated the assumption of normality (p < 0.0001). Therefore, the Mann–Whitney U test was used to compare the two approaches. The results showed no statistically significant difference between ML and DL models in terms of classification accuracy (p = 0.4832), suggesting that both paradigms offer comparable performance in the context of driving behavior classification.

Since the normality assumption was violated for multiple feature subsets (e.g., SET4: p = 0.0107, SET3: p = 0.0005), a non-parametric Kruskal–Wallis H test was conducted to assess the effect of different feature sets on model performance. The test revealed statistically significant differences (H(4) = …, p < 0.0001). To identify the specific differences between groups, a Dunn’s post-hoc test with Bonferroni correction was performed. The results (see Table 15) indicate that the ECU-only configuration significantly underperformed compared to all other subsets (e.g., SET1 vs. SET4: p ≈ 1.6 × 10⁻³⁶), whereas no significant difference was observed between the SET4 and SET3 subsets (p = 1.0000).

To further examine the influence of both model architecture and feature subset on classification accuracy, a two-way ANOVA was conducted. As shown in Table 16 model type had a statistically significant main effect (F(19, 1,248) = 4.35, p = 0.0017), while the main effect of the feature subset and the interaction between model and subset were not significant (p = 1.0000 and p = 0.9576, respectively). This suggests that accuracy was primarily determined by the model architecture, regardless of the input feature set used.

Comparison with studies in the literature

A direct comparison between the present study and other studies in the literature is not feasible due to significant differences in data sources and the devices employed. Generally, research on driver behavior classification categorizes driving styles into safe (Fadzil et al., 2026), unsafe (Lattanzi & Freschi, 2021; Mohammed et al., 2023), and aggressive (Singh & Singh, 2022; Ameen et al., 2021; Lin, Zhang & Chang, 2023; Merenda et al., 2022; Yarlagadda & Pawar, 2022; Zhang & Lin, 2021; Al-Rakhami et al., 2021) driving behaviors. Additionally, certain studies focus on enhancing driving economy (Azadani & Boukerche, 2022; Rastegar et al., 2024; Canal, Riffel & Gracioli, 2024; Yen et al., 2021; Pirayre et al., 2022) or vehicle ownership control (Martinelli et al., 2020).

The most commonly utilized data source for driver behavior classification is OBD-II data, which is typically collected using the ELM327 device (Kumar & Jain, 2023; Ameen et al., 2021; Fattahi, Golroo & Ghatee, 2023). However, alternative devices such as iSaddle (Singh & Singh, 2022), HH OBD Advanced (Merenda et al., 2022), and CarbigsP (Martinelli et al., 2020) have also been employed in various studies. In addition to engine operation data, motion sensors (Singh & Singh, 2022; Ameen et al., 2021; Azadani & Boukerche, 2022; Liu, Wang & Qiu, 2020; Pirayre et al., 2022; Merenda et al., 2022; Fattahi, Golroo & Ghatee, 2023; Al-Rakhami et al., 2021) are frequently integrated into driving behavior analysis. GPS data is not only utilized for location and speed estimation but also to retrieve OpenStreetMap-based road information (Lin, Zhang & Chang, 2023; Zhang & Lin, 2021). Furthermore, vision-based systems incorporating cameras have been developed for analyzing driver behaviors (Lin, Zhang & Chang, 2023; Yarlagadda & Pawar, 2022; Zhang & Lin, 2021).

To support data collection, a variety of microcontrollers and embedded systems have been utilized, including Shimmer v3 (Al-Rakhami et al., 2021) wearable sensors, ESP32 Wi-Fi modules (Mohammed et al., 2023; Yen et al., 2021), Arduino boards (Singh & Singh, 2022; Ameen et al., 2021), Raspberry Pi (Al-Rakhami et al., 2021), and other microcontrollers such as MCP2515 (Singh & Singh, 2022), nRF24L01+PA/LNA (Ameen et al., 2021), PIC18f4580 (Mohammed et al., 2023), PIC18F46K80 (Yen et al., 2021), and B-L475E-IOT01A (Merenda et al., 2022). Additionally, smartphones (Singh & Singh, 2022; Azadani & Boukerche, 2022; Pirayre et al., 2022; Merenda et al., 2022; Li, Lin & Chou, 2022; Fattahi, Golroo & Ghatee, 2023) have been widely employed for data transmission, collection, and real-time warning systems, with mobile applications such as ScanMaster (Fattahi, Golroo & Ghatee, 2023) and CarScanner (Merenda et al., 2022) facilitating these processes. Moreover, liquid crystal display (LCD) screens and light-emitting diode (LED)-based warning systems (Ameen et al., 2021) have been incorporated as part of the warning mechanisms.

In terms of classification methodologies, deep learning models such as GRU (Zou et al., 2022; Lee & Yang, 2023), CNN (Merenda et al., 2022), LSTM (Tripicchio & D’Avella, 2022), and deep neural networks (DNN) (Al-Rakhami et al., 2021) are among the most commonly used approaches for driver behavior classification. Additionally, ensemble learning techniques such as random forest (Singh & Singh, 2022; Fattahi, Golroo & Ghatee, 2023), Adaptive Boosting (AdaBoost) (Azadani & Boukerche, 2022; Fadzil et al., 2026), and extreme gradient boosting (XGBoost) (Canal, Riffel & Gracioli, 2024) have demonstrated strong performance in related studies. Moreover, supervised learning methods, including ANN (Lattanzi & Freschi, 2021; Yen et al., 2021), as well as unsupervised learning techniques such as k-means clustering (Lin, Zhang & Chang, 2023; Zhang & Lin, 2021) and fuzzy logic-based approaches (Li, Lin & Chou, 2022), have been explored in various studies.

Unlike prior studies that primarily rely on either OBD-II data or motion sensors in isolation, the present study adopts a multi-modal approach by integrating engine diagnostics, inertial motion data, road context information, and environmental sound features. While many works in the literature focus on classifying driver behavior into binary categories such as safe/unsafe or normal/aggressive (e.g., Singh & Singh, 2022; Ameen et al., 2021), this study formulates a three-class classification framework, allowing for a more nuanced understanding of behavioral variations. Additionally, rather than depending on off-the-shelf mobile applications or fixed sensor platforms, a custom data acquisition pipeline was developed to ensure synchronized and windowed data segments, enabling consistent deep learning-based analysis. Compared to previous studies utilizing individual devices such as Arduino, Raspberry Pi, or ESP32 for partial data collection (Merenda et al., 2022; Mohammed et al., 2023), our system leverages the synergistic potential of multiple sensor streams, thus enriching the temporal and contextual representational capacity of the dataset. Furthermore, while deep models such as CNN, RNN, LSTM, and GRU have been frequently applied in isolation, this study performs a systematic comparison across multiple deep architectures and traditional ML baselines, supported by rigorous statistical validation. This comprehensive setup not only contributes a rich and hybrid feature space but also enables a statistically grounded benchmarking framework, which remains limited in many prior works. Table 17 presents a comparative overview of recent studies on driver behavior classification, summarizing the devices and sensors utilized, the employed machine learning approaches, and the reported classification performances.