MAPE-ViT: multimodal scene understanding with novel wavelet-augmented Vision Transformer

Introduction

In the field of computer vision, scene understanding involves analyzing images or videos to interpret the visual environment. To achieve this, various techniques and algorithms have been developed to recognize and categorize entities within a scene and understand the spatial relationships between those entities. Scene understanding plays a pivotal role in diverse applications, including autonomous vehicles (Chen & Huang, 2017), context-aware augmented reality (Tahara, Ushiku & Harada, 2020), surveillance and security (Calavia et al., 2012), and agriculture (Tsouros, Bibi & Sarigiannidis, 2019). However, despite the advancements in this area, numerous challenges persist, such as variations in scale and viewpoints, high computational demands, and issues related to generalization and overfitting. Specifically, using RGB-D data introduces additional complexities, including limitations of depth sensors, occlusions, data gaps, and misalignments between depth and color images. These challenges make accurate scene representation and analysis a difficult task.

RGB-D scene classification has advanced significantly in recent years, yet most existing methods continue to rely heavily on convolutional neural networks (CNNs) and other conventional paradigms. As demonstrated by Eitel et al. (2015) and Song, Lichtenberg & Xiao (2017), Ahmed et al. (2024) CNN approaches are effective at extracting spatial features from RGB-D data. However, such methods often struggle with the complexities of multi-modality and intricate scene structures. One of the primary issues is that CNNs frequently fail to properly align depth and RGB channels, especially when affected by background noise, occlusions, or misalignment caused by sensor shifts. Additionally, CNNs prioritize local or global features without effectively balancing the two. This can be problematic in tasks like scene classification, where understanding the relationships between objects in space and preserving object boundaries are crucial (Jia et al., 2021). Research has further indicated that current methods inadequately address challenges related to sensor misalignment, noise distortion, and multi-modal data fusion, especially when preserving the structural integrity of objects within the scene (He et al., 2022). Despite their success in capturing long-range dependencies, attention-based models face challenges in efficiently processing high-resolution RGB-D data. When handling dense spatial information, these models often struggle with computational scalability and memory constraints. Sequence models, while effective for temporal data, have shown limitations in capturing spatial hierarchies and maintaining local feature consistency in RGB-D fusion tasks (Silberman et al., 2012a, 2012b; Zhang, Li & Zhang, 2020; Ahmed & Jalal, 2024a).

Current transformer-based approaches for RGB-D scene understanding face several key challenges. First, their self-attention mechanisms, while powerful for modeling global relationships, often struggle to effectively balance local and global feature representation (Meena, Kumar & Yadav, 2024). Second, the fixed-size patch embedding commonly used in Vision Transformers can lead to information loss at object boundaries and struggle with varying scales of objects in scenes (Li et al., 2023). Third, existing attention-based models often fail to effectively handle the unique characteristics of depth information, particularly in cases of sensor noise and missing data (Park et al., 2023).

To address these limitations, we propose a novel method that builds upon recent advancements in scene understanding. Our key contributions can be summarized as follows:

• We propose an enhanced Vision Transformer architecture incorporating masked pre-training techniques from MAPE-ViT and adaptive patch integration, enabling better feature learning and scene understanding across different scales.

• We introduce a novel multi-modal feature extraction approach that combines wavelet coefficients with maximally stable extremal regions (MSER), significantly improving both local feature detection and global context understanding in scene analysis.

• We develop an efficient classification framework that integrates an extreme learning machine (ELM) with a Grey Wolf Optimizer (GWO) for optimal feature selection, enhancing the model’s accuracy and computational efficiency.

• We demonstrate state-of-the-art performance on both SUN RGB-D and NYU v2 datasets through extensive experimentation, validating the effectiveness of our proposed architecture in indoor scene understanding tasks.

• We provide comprehensive ablation studies and analyses that validate the effectiveness of each component in our proposed framework, offering insights into the contribution of each architectural decision.

The rest of the article is structured as follows: The “Related Work” section represents previous work in this field. “Materials and Methods” covers the proposed methodology, including the stages of pre-processing, RGB and depth image fusion, segmentation, feature extraction, feature-level fusion, and classification. “Results” compares the experimental results with conventional object segmentation, detection, and classification methods. In “Conclusions” we conclude with some key insights and future directions.

Related work

In recent years, there’s been a lot of progress in detecting and classifying objects in RGB images. Various methods are being used by moving from traditional machine learning to advanced deep learning models. For example, convolutional neural networks (CNN), You Only Look Once (YOLO), Faster region-based convolutional neural network (R-CNN), and solid state drive (SSD) are the most widely used and are well-known for object recognition performance. Meanwhile architectures such as ResNet, Inception, and EfficientNet have proven quite effective for classification. Lately, attention-based models like Vision Transformers have become quite popular, offering new ways to understand images. Researchers have also explored ensemble methods and integrated semantic segmentation with object detection, as seen in Mask R-CNN, to enhance scene comprehension (Gupta et al., 2014; Al Mudawi et al., 2024; Zhou et al., 2022a, 2022b).

From Table 1, it is clear that multi-object detection and scene classification is still a challenging task for RGB-D images. The comprehensive literature review presented above reveals several critical research gaps in existing RGB-D scene understanding approaches. Current methods like CDNet and BCINet struggle with effective RGB-D fusion, particularly in handling sensor misalignment and noise. While models like Multi-task attention network (MTANet) and multi-modal feature fusion network (MFFNet) attempt to address this through attention mechanisms, they still face challenges in preserving structural integrity during fusion, especially at object boundaries. Current methods predominantly use fixed-size processing units (convolutional kernels or transformer patches), limiting their ability to effectively handle varying object scales. While models like saliency and edge reverse attention (SERA) and differentiable local feature selection (DLFS) attempt to address this through multi-scale processing, they still struggle with dynamic scene compositions. Existing architectures often fail to strike an optimal balance between local and global feature representation. While attention-based models excel at capturing global relationships, they struggle with local feature consistency. These gaps highlight the need for a more comprehensive approach that can effectively balance feature quality, and robustness while maintaining strong performance across varying scales and conditions. Our proposed MAPE-ViT framework aims to address these limitations through its novel integration of adaptive patch embedding, wavelet coefficients, and MSER regions.

Materials and Methods

Figure 1 shows the structural diagram of our proposed model for scene understanding by combining RGB and depth data. Initially, RGB and depth images undergo preprocessing to enhance quality and remove noise. The preprocessed images are fused using our Fusion Model, combining complementary information from both modalities for efficient processing. The fused images are segmented using the proposed Multi-dimensional Gradient-Aware Segmentation method. We propose a novel feature extraction method, Multimodal Adaptive Patch Embedding with Vision Transformer (MAPE-ViT), which extracts relevant and discriminative features from the segmented images. For multi-object classification, we employ the extreme learning machine (ELM).The ELM classifier takes the optimized features as input and accurately classifies objects in the scene. Lastly, we utilize conditional random fields (CRFs), a powerful probabilistic graphical model for scene classification. The CRF integrates information from multi-object classification results, contextual information extracted using a MAPE-ViT, and depth data from the original images.

RGBDFusionNet

The fusion of RGB and depth information for indoor scene understanding is challenging due to the variations between these images. The RGB and depth data exhibit different characteristics, making it difficult to fuse them efficiently to prepare them for segmentation tasks (Farahnakian & Heikkonen, 2022; Radford et al., 2021). To effectively integrate RGB and depth information for indoor scene understanding, we propose the FuseNet architecture, which employs an attention-based fusion strategy. The network architecture is split into two parallel channels in which the first stream handles RGB inputs, while on the other hand the second stream handles the depth data. The RGB and the depth streams are processed in parallel and consist of two residual block and two convolution layers. Each layers has a 3 × 3 kernel which is followed by a batch normalization and ReLU activation function (Kazakos, Nikou & Kakadiaris, 2018).

The outputs of these streams are then combined along the channel dimension to create a fused feature map. The important thing to note here is instead of simply element wise adding the feature maps of the two modalities, we incorporate an attention mechanism which helps to weight the features accordingly. This attention module has two convolutional layers. We used 1 × 1 kernel and it will reduce the dimension of the channel to half before passing it to ReLU activation. The second layer which is also 1 × 1 kernels is used to provide a single-channel attention map. A point wise multiplication is performed between attention map and fused feature map in order to emphasize on salient features.

These fused representation from the attention weights are then passed to two more convolutional layers. The first layer consist of 3 × 3 kernel and a ReLU activation function while the second layer consist of 3 × 3 kernel. With the help of this attention-based fusion approach, the proposed FuseNet architecture tries to address issues which arise from the large differences between the RGB and depth modalities, and the nature of depth maps uncertainty. The implementation of selective feature weighting makes it possible for the network to harness complementary information from both modalities; thus, it could enhance performance in indoor scene understanding tasks. The proposed multimodal fusion process is illustrated in Fig. 2 which integrate RGB and depth modalities. The RGB input undergoes three convolutional layers to extract spatial features, while the depth input is processed through a similar pathway to capture depth-related details. These feature maps are concatenated, forming a unified representation that incorporates both modalities. This fusion enables the model to leverage complementary information from RGB and depth inputs, resulting in more robust image fusion.

Segmentation

Multi-dimensional gradient aware segmentation

This method is derived from density based clustering that include DBSCAN (Brahmana, Mohammed & Chairuang, 2020) with the addition of a distance function tailored for fused image analysis. MGAS uses spatial information, color values together to form local gradients from the fused image and it offers a similarity measure for each pixel.

The prosped distance metric between two pixels $pandq$ is defined as

(1) $$d\left(p,q\right)=\sqrt{{\omega }_s.d_{spatial}{\left(p,q\right)}^2+{\omega }_c.d_{color}{\left(p,q\right)}^2+{\omega }_g.d_{gradient}{\left(p,q\right)}^2}$$here, $d_{spatial}\left(p,q\right)$ is the euclidean distance in pixel coordinates, $d_{color}\left(p,q\right)$ is the difference in color values, $d_{gradient}\left(p,q\right)$ is the combined difference in local color gradients, ${\omega }_s,{\omega }_{c}and{\omega }_g$ are weights for the spatial, color and gradient components, respectively (Ahmed & Jalal, 2024b). Each component of the distance metric is defined as follows:

(2) $$d_{spatial}\left(p,q\right)=\sqrt{(P_x-q_x{)}^2+{\left(p_y-q_y\right)}^2}$$where, $(P_x-p_x)$ and $(q_x-q_y)$ are the cordiantes of pixel p and q.

(3) $$d_{color}\left(p,q\right)=\sqrt{{\left(R\left(p\right)-R\left(q\right)\right)}^2+{\left((G\left(p\right)-G\left(q\right)\right)}^2+{\left(B\left(p\right)-B\left(q\right)\right)}^2}$$where, $R(p),G(p),B(p)$ and $R(q),G(q),B(q)$ are the red, green and blue color values at pixels p and q, respectively.

(4) $$d_{gradient}\left(p,q\right)=\sqrt{{\left(G_{xc}\left(p\right)-G_{xc}\left(q\right)\right)}^2+{\left(G_{yc}\left(p\right)-G_{yc}\left(q\right)\right)}^2}.$$

This approach preserves object boundaries by emphasizing local gradients and spatial coherence, addressing the challenge of boundary degradation in cluttered scenes. By integrating depth gradients into the distance metric, it enhances segmentation accuracy in regions with ambiguous color/texture but distinct geometric profiles, directly improving scene parsing for complex indoor environments. Figure 3 illustrates the stages of data processing using our novel fused and segmentation approach; column 1 shows RGB images, which provide the color and texture information of the scenes. Column 2 shows depth images, column 3 depicts fused images that combine the complementary features of RGB and depth modalities, enhancing the richness of extracted features, and column 4 displays the segmented images showcase the final semantic segmentation results, where different objects in the scene are accurately identified and labeled with distinct colors.

Extreme learning machine for multi-object recognition

The extracted features sets obtained from MAPE-ViT are fed into an ELM for multi-object recognition. The ELM architecture is chosen for its rapid training speed and strong generalization capabilities which makes it suitable for complex multi-object classification tasks with varying numbers of objects per image (Wang & Wang, 2021).

ELM architecture

The ELM consist of three layers, the first layer is the input layer, then a single hidden layer, and third layer is the output layer. The input layer contains 128 nodes which corresponds to the dimension of the optimized feature vectors. The hidden layer has 512 neurons, which are determined through extensive experimentation in order to balance performance and computational efficiency. The last layer which is the output layer contains m nodes, where m represents the total number of possible object classes in the dataset. Figure 4 represents an ELM with three layers: an input layer (1,024 neurons) for receiving feature vectors, a hidden layer (1,536 neurons) with a sigmoid activation function for nonlinear transformations, and an output layer for generating predictions. The input and hidden layers are fully connected with fixed random weights, while the hidden-to-output connections are trained. The output layer applies a threshold of 0.4 for binary classification. This structure ensures fast training and efficient computation.

The ELM output is interpreted as a set of probability scores for multi-object recognition for each object class. The final output is computed as:

(14) $$output=H\beta .$$

A threshold of 0.4 is applied to the output probabilities to determine the presence of each object class. This thresholding approach allows for recognizing a variable number of objects in each image.

Scene classification with conditional random field

For scene classification, we used a conditional random field (CRF) which will receive three inputs: multi-object classification probabilities, contextual information from the Vision Transformer output, and depth gradient statistics of the images. Depth gradient statistics describe spatial and geometric properties of depth images which are essential for describing scene structure. These gradients are then analyzed to extract various statistical measures that describe the distribution and behavior of the depth gradients across the image. Specifically, we compute mean, variance, skewness, and kurtosis.

CRF formulation for scene classification

The CRF (Cao et al., 2020) is employed to refine scene classification by integrating multi-object classification probabilities, contextual information from the Vision Transformer, and depth gradient statistics. The CRF models the conditional probability of scene labels $Y$ given the input features $X$ as follows:

(15) $$P(Y|X)={\displaystyle \frac{1}{z\left(X\right)}exp\left(\sum _i{\psi }_u\left(Y_i,X\right)+\sum _{i,j}{\psi }_p\left(Y_i,Y_j,X\right)\right)}$$where $Z\left(X\right)$ is the partition function ensuring normalization, $\psi u$ is the near potential, which captures the relationship between each scene label and the input features. It is computed using:

(16) $${\psi }_u\left(Y_i,X\right)={\omega }_{MO}.P_{MO}\left(Y_i\right)+{\omega }_{DGS}.DGS\left(Y_i\right)$$where, ${\omega }_{MO}and{\omega }_{DGS}$ are the weights learned during the training. $P_{MO}\left(Y_i\right)$ is multi object classification probabilities, and DGS(Yi) are the depth gradient statistics (Li et al., 2020). Similarly, the pairwise potential ${\psi }_p$ models the dependencies between neighborhood scene labels as shown in Eq. (17)

(17) $${\psi }_p\left(Y_i,Y_j,X\right)={\omega }_{pair}.exp\left(-{\left|\left|X_i-X_j\right|\right|}^2\right)$$where ${\omega }_{pair}$ is a weight that controls the strength of the pairwise relationship, and $X_i$ and $X_j$ are the input features at positions i and j, respectively. Figure 5 illustrates the classification of the scene using CRF that combines probability of multiple objects predication by ELM, contextual information of Vision Transformer outputs and DGS that quantifies the geometrical features using statistical measure as mean, skewness and Kurtosis. The CRF leverages these features to refine scene labels and ensure spatial coherence.

Computing infrastructure

We conducted the experiments using a PC which is equipped with an x64-based Windows 10 operating system, an Intel Core i3-4010U 1.70.GHz CPU, 4GB RAM. The system’s performance was evaluated using two benchmark datasets: SUN RGB-D and NYU v2. Table 2 represents the hyper parameter of MAPE-ViT framework.

Datasets description

The NUY-Dv2 (Silberman et al., 2012a) dataset contains labeled and unlabeled images from indoor scenes. The dataset consists of different scenes which includes bathroom, bedroom, bookstore, cafe, kitchen, living room, and office. These scenes encompass a diverse range of objects, including beds, bookshelves, books, cabinets, ceilings, floors, pictures, sofas, tables, TVs, walls, windows, and various background elements. We also used Sun-RGBD (Silberman et al., 2012a) dataset for the experiment purposes which also include scenes from indoor environment. From both the datasets we chose 10 common classes for our experimentation.

Results

The experimental results presented in the article demonstrate the effectiveness of the proposed method in scene classification tasks using RGB-D data. The combination of MSER, wavelet transforms, and Vision Transformers for feature extraction has proven to be a robust and discriminative approach, overcoming challenges related to sensor misalignment, depth noise, and object boundary preservation. One of the key strengths of our method is its ability to accurately classify scenes with distinct visual characteristics and depth patterns. It was highly accurate with offices, bedrooms, and living rooms, primarily due to clearer structural patterns and furniture arrangements. This can be attributed to the introduction of MSER-guided image patches, and Vision Transformers whereby these methodologies are capable of independent scene analysis due to their inherent capability of capturing both local and global features, object boundaries, as well as spatial relationships. The classification results of both datasets are shown in Tables 3 and 4 using confusion matrix which is one of the ways to display the detailed breakdown of the model’s performance for all the classes in dataset. It highlights the insight of the scene or object categories which confuses with one another. It displays true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN) for each class present in the dataset.

The confusion matrices reveal that bedrooms in both datasets suffer from FP with living rooms due to shared structural features, such as beds and sofas. Similarly, kitchens and dining rooms exhibit mutual confusion, primarily caused by overlapping objects like countertops and tables. In the NYU-Dv2 dataset, living rooms are often misclassified as dining rooms (15.5% FP rate) due to shared furniture layouts. On the SUN RGB-D dataset, bathrooms face challenges with other categories (13.1% FP rate), potentially due to clutter or poor lighting. Lastly, offices and house offices show mutual misclassification across both datasets, reflecting difficulties in distinguishing desk-based environments with similar setups.

The performance of a model is often evaluated using key metrics: precision, recall and mAP as displayer in Tables 5 and 6. Precision measures the accuracy of the positive predictions made by the model. It indicates the proportion of true positive instances among all instances that the model identified as positive. This means that a high precision score reflects a model that returns mostly relevant results, minimizing the number of incorrect positive predictions. Conversely, recall assesses the model’s ability to identify all relevant instances within a dataset. It captures how many true positive instances were correctly predicted by the model out of all actual positive instances. The mAP incorporates the trade-off between precision and recall and considers both false positives (FP) and false negatives (FN).

(18) $$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$$

(19) $$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(20) $$mAP={\displaystyle \frac{1}{N}\sum _{i=1}^{N}A{P}_i.}$$

Our proposed model performs remarkably in both scene classification and object segmentation tasks on the challenging NYU-Dv2 and SUN-RGBD datasets. Notably, the model identifies specific scene types like bedrooms, kitchens, bathrooms, closets, and bookstores, exhibiting near-perfect precision and high recall. However, it encounters some challenges with visually similar or cluttered scenes such as living rooms, dining rooms, offices, and home offices, particularly on the NYU-Dv2 dataset, where either precision or recall suffers. Interestingly, the model performs relatively better across all scene categories on the SUN-RGBD dataset, potentially due to cleaner or more distinct scene representations.

Ablation study

In Fig. 6, we present the ablation study of our proposed model, comparing the scene classification performance using RGB images, depth images, and a fused approach. The results demonstrate that the fused images consistently yielded the highest classification accuracy across both datasets. Specifically, RGB images achieved 72% accuracy on the SUN RGB-D dataset and 82% on the NYU v2 dataset. Depth images, on the other hand, produced 67% and 76% accuracy on the respective datasets. In contrast, the fused RGB-D images significantly outperformed the individual modalities, achieving 78.56% accuracy on the SUN RGB-D dataset and 88.79% on the NYU v2 dataset. The better performance of the fused approach highlights the complementary nature of RGB and depth information and when combined, it provides a better understanding of the scene. This fusion leverages the strengths of both modalities, capturing both the color and texture information from RGB images and the spatial and structural details from depth images.

Table 7, shows the comparison of our proposed model with the state of the art model. Our model outperformed SOTA methods in both of the datasets by achieving the 78.56% and 88.79% scene classification accuracies on SUN RGB-D and NYUv2 datasets respectively. These results are credited to our novel feature extractor methods which extract features for classification and also as we provide very enrich input to CRF model for scene classification which comprises of multi-object classification probabilities, contextual information from the Vision Transformer, and depth gradient statistics.

Table 7:

Comparison of proposed method with existing approaches.

SOTA methods	SUN RGB-D	NYUv2
Song, Chen & Jiang (2017)	--	66.9
Song et al. (2018)	53.8	67.5
Xiong, Yuan & Wang (2020)	56.2	68.1
Du et al. (2019)	56.7	69.2
Rafique et al. (2022)	63.1	72.8
Cai & Shao (2019)	48.7	79.3
Seichter et al. (2022)	61.8	76.5
Pereira et al. (2024a)	62.3	77.8
Pereira et al. (2024b)	63.7	80.1
Proposed model	78.56	88.79

DOI: 10.7717/peerj-cs.2796/table-7

Discussion

The experiments conducted on our novel MAPE-ViT model for multimodal scene understanding have yielded promising results, validating the effectiveness of our approach in addressing key challenges in RGB-D image classification specifically issues related to sensor misalignment, noise distortion, and the fusion of multi-modal data. It demonstrates exceptional performance across various environmental conditions and complex scene structures. The MAPE-ViT model, with its innovative combination of MSER and wavelet coefficients, extracts important information from complex multimodal data. By integrating RGB and depth information through our adaptive patch embedding technique, it captures both local and global features which is one of the fundamental factor for accurate scene classification. The incorporation of Vision Transformers for processing these patch embedding further enhances the model’s ability to handle intricate spatial relationships and sensor misalignment issues which is one of the common challenge in RGB-D fusion. We used the Grey Wolf Optimizer for feature selection and the extreme learning machine for final classification which is shown significant improvements in both accuracy and efficiency compared to traditional approaches. As a result it helps to take informed decision for applications like autonomous navigation and augmented reality as it enhances the object detection and classification process.

Limitations

Although our model has produced good results across both the datasets. They key aspects for achieving state of the art results across both dataset is the multi modal data fusion and novel MAPE-ViT based feature extraction. However, there are some challenges and limitations like our model is evaluated on two datasets SUN RGB-D and NYU v2, both these datasets focus mainly on the indoor scenes. So, this limits our model generalizability on outdoor scenes which will be addressed in future work. Another important thing to note is that the attention method used in MAPE-ViT, heavily relies on global self attention. This make it less focused on local details as a result the objects with larger size and dominate features plays vital role in feature extractions which some times can add bias towards larger objects in scenes. The proposed framework is specifically designed and tested for RGB-D images. It does not currently support data from other sensors, such as LiDAR or thermal imaging. Adapting the model to handle such modalities would require additional research and modifications to the data fusion and feature extraction mechanisms, which we plan to explore in subsequent studies. The attention method in MAPE-ViT relies heavily on global self-attention, which can sometimes overshadow local details. This may introduce bias toward larger or more dominant features in the scene, potentially underrepresenting smaller or subtler objects. We aim to refine the attention mechanism by incorporating localized attention strategies or hierarchical attention layers in future iterations to mitigate this issue.

Conclusion and future work

In this article, we presented a novel scene classification method that uses novel Multimodal Adaptive Patch Embedding with Vision Transformer for feature extraction which combines MSER, wavelet transforms, and Vision Transformers to effectively address challenges in RGB-D image analysis such as sensor misalignment, depth noise, and object boundary preservation. In our method, we use the MSER stable regions connected with the wavelets coefficients to construct more comprehensive descriptors. The generation of rich, multi-modal patch embedding and their processing by Vision Transformers allow our method to learn complex relationships between RGB and depth information, enabling dual outputs for multi-object and scene classification. In addition, the anisotropic diffusion and Gray Wolf Optimization improve the stability and identification capability of the proposed method as shown in experimental results.

Our research opens up several promising directions for future work, particularly in extending our model’s capabilities to diverse domains. A primary direction is adapting the MAPE-ViT framework for outdoor scene understanding, where challenges include varying lighting conditions, weather effects, and dynamic object interactions. This would require enhancing our RGB-D fusion technique to handle larger depth ranges and variable environmental conditions. Another significant direction is extending our framework to remote sensing applications. This would involve adapting MAPE-ViT to process multi-spectral and hyperspectral imagery alongside RGB-D data. The model’s wavelet-based feature extraction and adaptive patch embedding could be particularly valuable for analyzing satellite and aerial imagery, enabling applications such as land use classification, urban planning, and environmental monitoring. We plan to modify our architecture to handle the unique characteristics of remote sensing data, including different spatial resolutions, multiple spectral bands, and varying viewing angles.

Supplemental Information