ROM-Pose: restoring occluded mask image for 2D human pose estimation

Human pose estimation

HPE is a critical task in computer vision, with various methods developed to detect and associate body parts. These methods are typically classified into two broad categories: top-down and bottom-up approaches, each exhibiting unique strengths and weaknesses concerning speed, scalability, and accuracy.

The top-down approach, as described by Huang, Gong & Tao (2017), Papandreou et al. (2017), Sun et al. (2019a), Wang et al. (2020), initially detects individuals in an image before applying a bounding box for pose estimation. This method provides beneficial scalability but heavily relies on the precision of the bounding box. To increase precision, (Fang et al., 2017) resolved challenges in pose estimation, including those with imprecise bounding boxes, and (Khirodkar et al., 2021) suggested deploying multiple bounding boxes during non-maximum suppression (NMS) to better detect poses, particularly in occluded subjects. This top-down approach aligns well with the methodologies advocated in this article as it relies on bounding box-based human pose estimation.

In contrast, the bottom-up approach, as discussed by Pishchulin et al. (2016), Insafutdinov et al. (2016), Cao et al. (2017), Kreiss, Bertoni & Alahi (2019), Cheng et al. (2020), Geng et al. (2021), and Jin et al. (2020), focuses on detecting key points for body parts and grouping them to estimate the full pose. Although highly accurate, this method is computationally intensive because it entails detecting all key points before grouping them. Efforts like those by Debapriya et al. (2022) and PersonNet by Papandreou et al. (2018) have enhanced efficiency through multilayer feature extraction and the application of 2D offset fields to facilitate key point grouping. However, bottom-up methods demand extensive computation and are unsuitable for the applications proposed in this study.

HPE involves estimating the pose of humans in an image. It faces particular challenges in detecting peripheral body parts like hands and feet, which are often smaller and more easily occluded than other body parts such as the head or torso. To enhance accuracy in HPE, various backbone models with improved architectures have been proposed.

For example, Newell, Yung & Deng (2016) introduced stacked hourglass networks to address resolution discrepancies stemming from the varying scale of detected individuals. By transforming input images into multiple scales, the model aggregates results from each to improve pose estimation accuracy. Sun et al. (2019b) developed HRNet, which maintains high resolution for feature extraction, regardless of person sizes in the image. Additionally, Gao et al. (2019) proposed Res2Net, a residual connection-based structure to deepen networks and enhance feature extraction, which serves as the foundation for the simple baseline, while Su et al. (2019) introduced the channel shuffle module (CSM) to improve cross-channel information sharing between low and high-level feature maps.

Another breakthrough in HPE is the introduction of Vision Transformers (ViT) by Dosovitskiy et al. (2021), which process images using a decoder-encoder structure. This model, as seen in models such as Yufei et al. (2022), outperforms traditional convolutional neural networks (CNN) models in feature extraction. Recently, Zhou et al. (2023) have developed hybrid models that combine bottom-up detectors with attention-based top-down methods (BUCTD) to integrate the strengths of both approaches.

Amodal instance segmentation

Amodal instance segmentation as described in Li & Malik (2016) aims to restore occluded parts of objects in an image, enabling the model to deduce hidden parts based on available visual cues. This approach mirrors human cognitive capabilities, which allow individuals to perceive an entire object even when parts are concealed. Various studies, such as those by Zhan et al. (2020), have explored amodal instance segmentation through self-supervised learning to anticipate occluded object sections. Qi et al. (2019) introduced the KITTI (Geiger, Lenz & Urtasun, 2012) INStance dataset (KINS), augmenting it with detailed pixel-level annotations for visible and occluded object regions, enabling advancements in amodal instance segmentation. Ke, Tai & Tang (2021) uses a bilayer structure to decouple occluder and occludee boundaries, improving segmentation, especially in heavy occlusion cases. Nonetheless, these methodologies are often challenged by lengthy restoration periods and substantial computational demands.

To mitigate these challenges, Nguyen & Todorovic (2021) introduced a technique that utilizes uncertainty loss to accelerate restoration, concentrating on boundary information and thus diminishing the computational burden. Conversely, our study automates dataset generation, obviating the need for manual annotation and optimizing the training regimen for amodal instance segmentation, making the process more rapid and efficient.

In our work, we propose a novel method that combines a restored mask with an RGB image to enhance pose estimation accuracy. This method enables more precise detection of key points, even in the presence of occluded parts and inaccurate bounding boxes, setting it apart from traditional approaches in HPE.

Whole COCO mask dataset

To restore the mask image, it is essential to train the restoration model using both full-mask and occluded images. Full-mask images are grayscale mask images that encompass both occluded and visible parts of the target. Since the COCO 2017 dataset lacks these full-mask images, we propose a method to generate them in order to effectively train the restoration model.

An annotation file from the COCO dataset, including image ID, key-point coordinates, and mask image coordinates, is needed to produce full-mask images. The existing mask images in the COCO dataset are inadequate because they fail to cover occluded areas. For instance, Fig. 2C displays a gap in the COCO mask within a red circle, with ground truth key points (pink dots) present, underscoring the necessity for full-mask images to enable the restoration model to recover missing parts.

Furthermore, experiments conducted using the initial mask images from the original COCO dataset revealed that there was no significant difference in performance between models trained with these masks and those trained without masks.

To solve this issue, we suggest an automated method for creating full-mask images, which drastically cuts down on the time and labor associated with manual annotation. However, the effectiveness of this method is constrained by the accuracy and completeness of the existing key-point annotations, which may limit its general applicability.

Our full-mask generation method is applicable not only to the COCO dataset but also to other COCO-style formatted datasets like OCHuman (Zhang et al., 2019), which includes human key-point pairs. This wide applicability demonstrates the method’s potential value in occlusion-aware human pose estimation across various datasets.

The process for generating the key points image starts by loading the original image onto an empty canvas. A circle is then drawn at each key point’s coordinates, as illustrated in Fig. 3A. Subsequently, ellipses are configured based on key-point relationships, using the midpoint as the origin. In a deviation from simply drawing circles, ellipses connect coordinates representing body parts to create a complete mask, as depicted in Fig. 3B.

If the mask were created strictly following the ground truth, as demonstrated in Fig. 3B, fine lines from the original COCO dataset would be covered by the overlaid key points image, resulting in the visibility of white areas. Nonetheless, as evidenced in Fig. 3C, our method retains the integrity of the original COCO dataset’s mask.

Given that this technique is designed to enhance the original COCO dataset, it is not intended to replace it directly. Consequently, we suggest producing the key points image as depicted in Fig. 3D, identifying human body key points and surrounding them with ellipses. The red and blue ellipses and lines in Fig. 3D demonstrate this technique. In Fig. 3E, the created key points image enhances the detailed mask of the COCO dataset, supporting more precise ground truth identification. While shapes like triangles or rectangles can also be used to represent human pose, we found that ellipses are more effective in capturing the natural relationships between key points. Therefore, in this study, we use ellipses for mask generation. However, depending on the dataset being applied, using rectangles or other shapes for mask generation is also feasible, providing flexibility in dataset creation.

Ellipses are drawn using Eqs. (1) and (2). In Eq. (1), $R_{\mathrm{f}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}}$ modifies the dimensions of circles and lines in proportion to the bounding box’s size, where $w$ represents the width and $h$ the height. Equation (2) computes the circle’s radius used for displaying key points, ensuring it ranges between 4 and $R_{\mathrm{f}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}}\times 10$.

(1) $$R_{\mathrm{f}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}}=\sqrt{\frac{({\mathrm{w}}_{max}-{\mathrm{w}}_{min})\times ({\mathrm{h}}_{max}-{\mathrm{h}}_{min})}{\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{g}{\mathrm{e}}_{height}\times \mathrm{i}\mathrm{m}\mathrm{a}\mathrm{g}{\mathrm{e}}_{width}}}$$ (2) $$R_{\mathrm{p}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{t}}=min\left(10,max\left(R_{\mathrm{f}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}}\times 10,4\right)\right)$$ (3) $$\mathrm{F}\mathrm{u}\mathrm{l}\mathrm{l}\text{-}\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}=\mathrm{M}\mathrm{a}\mathrm{s}\mathrm{k}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{g}\mathrm{e}\odot 0.5+\mathrm{k}\mathrm{e}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{g}\mathrm{e}\odot 0.5.$$

The final key points image is combined with the mask image through element-wise multiplication and summation, as defined in Eq. (3), to produce a full-mask image. This procedure blends the mask and key points images with equal weight, achieving a semi-transparent appearance. These full-mask images are produced for each image in the COCO 2017 dataset, creating the comprehensive Whole COCO mask dataset.

Restoration model training

We trained the restoration model using the Whole COCO mask dataset. Figure 4A depicts an object obstructing the target. Figure 4C displays the grayscale mask image of the obstructing object, termed the occludee mask image. Figure 4C also shows the target area that needs restoration. Figure 4B illustrates the area where the restoration model identifies the target in the image. At this point, the selection of the occludee mask from the dataset pair is essential. For instance, the first line of Fig. 4 presents two men in the picture, limiting the maximum mask information obtainable from the image to 2. Therefore, if the individual at the back is the target, the occludee must be the person in front in the image. In scenarios with more than two individuals, like in the 2nd or 3rd row of Fig. 4, the nearest person is selected as the occludee mask. After selecting the occludee target and the occluded target, the model is trained to determine the boundary between the occluded mask and the occludee mask. Figure 4D demonstrates the boundary line as identified by the model. The dataset does not include ground truth data for the boundary line.

To achieve this, we leveraged the Amodal Segmentation with Boundary Uncertainty (ASBU) (Nguyen & Todorovic, 2021) method. The ASBU method extends UNet (Ronneberger, Fischer & Brox, 2015) to output an $H\times W\times 2$ feature map with two channels: one for

(4) $$L=\sum _{i=1}^N\frac{1}{u_i}⊮(m_i^{\ast }=0)L_{i,0}+\lambda ⊮(m_i^{\ast }=1)L_{i,1},$$

In Eq. (4), the following symbols are used: $u_i$ is a weighting factor for the $i$-th pixel, $m_i^{\ast }$ is the ground truth mask for the $i$-th pixel, where $m_i^{\ast }=0$ indicates the pixel is outside the mask and $m_i^{\ast }=1$ indicates the pixel is inside the mask, $L_{i,0}$ is the loss for the $i$-th pixel when it is outside the mask, $L_{i,1}$ is the loss for the $i$-th pixel when it is inside the mask, and $\lambda$ is a regularization factor applied to the loss for pixels inside the mask.

Figure 4E represents the ground truth mask that the model aims to predict, while Fig. 4F displays the reconstructed mask generated by the model. The ground truth masks in Fig. 4E were created using the newly proposed COCO whole dataset, an extension of the original COCO dataset developed specifically for this study. Please refer to the description in Fig. 3 for more details on the COCO whole dataset and the mask generation process.

To evaluate the model’s performance, we compare the area covered by the predicted mask in Fig. 4F with the area of the ground truth mask in Fig. 4E, calculating the loss using two specific functions. $\mathrm{L}\mathrm{o}\mathrm{s}{\mathrm{s}}_1$ (Eq. (5)) assesses reconstruction error within the target object by focusing on regions where mask $=1$ and applies inmask_weight to emphasize restoration within the object’s boundaries. On the other hand, $\mathrm{L}\mathrm{o}\mathrm{s}{\mathrm{s}}_2$ (Eq. (6)) addresses error outside the target in regions where mask $=0$, using outmask_weight to avoid unnecessary reconstructions. Through iterative refinement, the model minimizes discrepancies between the reconstructed and ground truth masks (Figs. 4F and 4E).

Figure 4 also includes intermediate steps such as the occluder (Fig. 4C) and the overlap region (Fig. 4D), which assist the model in boundary detection and accurate restoration.

(5) $$L_{\mathrm{i}\mathrm{n}}={\lambda }_{\mathrm{i}\mathrm{n}}\frac{1}{|M|}\sum _{i\in M}\frac{1}{2}{\left(\frac{y^i-\mu }{\sigma }\right)}^2$$ (6) $$L_{\mathrm{o}\mathrm{u}\mathrm{t}}={\lambda }_{\mathrm{o}\mathrm{u}\mathrm{t}}\frac{1}{|B|}\sum _{i\in B}\frac{1}{2}{\left(\frac{y^i-\mu }{\sigma }\right)}^2.$$

Here, ${\lambda }_{\mathrm{i}\mathrm{n}}$ is a weighting factor applied to the loss computed for pixels inside the mask region M, where M denotes the set of indices corresponding to the masked region ( $m_i=1$). This ensures that the model prioritizes accurately reconstructing occluded regions. Conversely, ${\lambda }_{\mathrm{o}\mathrm{u}\mathrm{t}}$ is applied to the loss for pixels in the background region B, where B denotes the set of indices corresponding to the background region ( $m_i=0$), aiding the model in differentiating the target from the background. The mean squared error (MSE) is employed to measure the overall discrepancy between the predicted mask and the target full-mask image.

The model iteratively refines the restored mask image until it closely resembles the full-mask image. During testing, the restoration model no longer requires the full-mask image, as it has gained sufficient confidence in identifying boundary lines. Through these adaptations, our method utilizes the ASBU framework while integrating key point information and dataset-specific adjustments to improve the accuracy and reliability of mask restoration in occluded scenarios.

Pose estimation with restored image

To estimate human pose, the pose estimation model first detects the bounding box around the target object within the input image. This image is then enhanced with a restored grayscale mask, generated by our restoration model, to capture occluded regions. Within each identified bounding box, a $64\times 48$ heatmap is generated for each joint, and the coordinates of each estimated joint are derived from these heatmaps.

For pose estimation, we utilized the Simple Baseline method, which employs ResNet (He et al., 2016), a commonly used backbone network for image feature extraction. This method adds several deconvolutional layers above ResNet’s final convolutional layer, known as C5, to produce heatmaps from deep, low-resolution features. This simple design closely aligns with state-of-the-art methods like Mask R-CNN (He et al., 2017).

The Simple Baseline model, by default, comprises three deconvolutional layers equipped with batch normalization (Ioffe & Szegedy, 2015) and ReLU activation (Nair & Hinton, 2010). Each layer utilizes 256 filters with a $4\times 4$ kernel and a stride of 2, succeeded by a $1\times 1$ convolutional layer that generates the predicted heatmaps, $\{H_1,\dots ,H_k\}$, for all $k$ key points. This efficient architecture eliminates the complexity of skip connections found in other models.

Our method enhances the Simple Baseline approach by incorporating the newly developed Whole COCO dataset, an improved version of the COCO dataset that features both original RGB images and restored grayscale mask images. This integrated dataset enables the model to learn from both visible and occluded key points, thereby improving its pose estimation capabilities under conditions of occlusion.

To develop the Whole COCO dataset, we processed original COCO images to create restored grayscale mask images using our restoration model, and then merged them with the RGB images. The pose estimation model underwent training on these combined images using standard augmentation techniques to boost robustness and prevent overfitting. We assessed the model’s performance on a test set, focusing on accuracy in pose estimation under various occlusion conditions. Evaluation metrics included average precision (AP) and average recall (AR).

We employed MSE as the loss function. A metric extensively used in machine learning, MSE measures the mean squared difference between predicted and actual values. In pose estimation, MSE quantifies the discrepancy between the coordinates predicted by the model ( ${\hat{Y}}_i$) and the actual ground truth coordinates ( $Y_i$) for each of the 17 joints considered, with the average MSE across all joints serving as the loss metric. The targeted heatmap ${\hat{H}}_k$ for each joint $k$ is formed by applying a 2D Gaussian centered at the ground truth location of the $k$-th joint.

Finally, the pose estimation process involves both the restoration and pose estimation models, as illustrated in Fig. 1. The occluded mask and occludee mask images are extracted from the original RGB image via Faster R-CNN and then restored by the restoration model. The restored mask image and the RGB image are subsequently input into the pose estimation model. The restored mask image is overlaid onto the RGB image at full opacity and scale. Without the restored grayscale mask, the model cannot recognize occluded regions as part of the target. However, with the restored grayscale mask, the model can accurately define a bounding box that includes occluded regions, enabling it to estimate hidden key points of the target. This enhancement substantially improves pose estimation accuracy.

Experimental setup

Experiment result

As shown in Table 1, AP increased by 1.6% with ResNet-50, by 1.6% with ResNet-101, and by 1.8% with ResNet-152. Although the improvement in AP may seem modest, other metrics, particularly AP-medium (AP-m), demonstrated significant enhancements, increasing by over 10%. This result underscores the model’s ability to effectively utilize overlaid mask information for medium-sized objects, better capturing occluded parts in the restored mask and thus aiding in accurate posture estimation. For small-sized objects, the mask’s impact was minimal due to their lower resolution and smaller dimensions.

However, for large-sized objects, applying a mask could occasionally obstruct feature extraction. The mask’s increased size, based on the object’s dimensions, results in an expanded masking area, similar to the effect shown in Fig. 3C. In such cases, the mask image may obscure essential features of the target object, potentially reducing the accuracy of posture estimation.

Compared to state-of-the-art models like ViTPose (Yufei et al., 2022), our model exhibits a modest decrease in AP performance. However, our method provides significant computational efficiency benefits, as it has considerably fewer parameters. Moreover, our model adopts a top-down approach, which simplifies the pipeline by separating person detection and key point estimation. This modularity not only enhances interpretability but also allows for flexible integration into existing systems. As shown in Table 1, this reduced parameter count means our model does not require high-spec hardware, thus it is more accessible and practical for deployment in resource-constrained environments compared to Vision Transformer-based models. Despite these specifications, the model shows significant improvements in certain evaluation metrics, such as AP-m, suggesting it can achieve competitive performance at much lower computational costs.

Figure 5 is a qualitative result of performance improvement, where Fig. 5A visualizes the correct ground truth locations of 17 key points in the OCHuman dataset. Figure 5B displays results using the existing baseline, while Fig. 5C illustrates 17 key points identified by ROM-Pose.

Comparing the color boxes of each image, despite the unchanged model size, the key points predicted by ROM-Pose were closer to the correct ground truth than those predicted by the baseline model. Several observations can be made from the color boxes of each image. In the first image on the left, where two individuals overlapped, attention shifts to the second yellow box. With the baseline model, joints overlapped, as indicated by the presence of white and blue dots within the same box. However, ROM-Pose accurately identified this overlap as a single joint, aligning closely with the correct depiction shown in Fig. 5A. In the second image from the left, the positioning of the man’s arm (blue dot) within the orange box was closer to the correct answer. In the third image from the left, the focus went to the green box, where the key point (white point) on the lower body of the individual in red attire was more accurately captured by ROM-Pose, aligning well with the correct answer. In the fourth image from the left, attention was drawn to the red box marking the foot, which was correctly recognized as belonging to a single individual rather than two separate people’s feet. Regarding the sixth image from the left, the number of blue dots within the blue box increased compared to the baseline model, indicating that the model accurately identified the presence of two joints within the box. This aligned with the correct answer and confirmed that ROM-Pose demonstrated greater robustness than the baseline model in estimating posture in occluded environments.

Ablation study

To assess the impact of varying epsilon values (the transparency ratio between the grayscale mask and the RGB image) on model performance, a detailed ablation study was conducted. Specifically, the effects of different epsilon values, including small values such as 1e−7 and 1e−5, were examined to evaluate their influence on training. Additionally, when the epsilon value exceeds 0.1, the model achieves an accuracy above 85%. However, this results in excessive reliance on the mask image, leading to an overfitted model. Therefore, in this ablation study, we set the maximum epsilon value to 0.1 to ensure balanced model generalization. Furthermore, additional experiments were performed to investigate whether the mask image significantly influences the training process. This study demonstrates that incorporating the mask image as part of the input enhances the model’s ability to estimate human pose. By varying the intensity levels of the mask image, we analyzed its direct contribution to the model’s capacity to effectively capture structural information.

(7) $$\mathrm{i}\mathrm{n}\mathrm{p}\mathrm{u}\mathrm{t}=\left(\mathrm{r}\mathrm{g}\mathrm{b}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{g}\mathrm{e}\odot (1-ϵ)+\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{g}\mathrm{e}\odot ϵ\right).$$

The epsilon values were adjusted as specified in Eq. (7) and the input was subsequently normalized to the range of [0, 255] to ensure consistent model processing. The findings summarized by different epsilon values are recorded below:

• Epsilon = 0.000001: The results (denoted by $\star$ in Table 2) showed no noticeable improvement over the baseline. With such a small epsilon value, the mask image had minimal effect, and the model’s performance was similar to that of the baseline.

• Epsilon = 0.00001: The results (denoted by $†$ in Table 2) demonstrated a significant enhancement in model accuracy. This increase in transparency enabled the model to integrate more features from the occluded portions of the object, consequently improving performance.

• Epsilon = 0.0001: Results (indicated by $\bullet$ in Table 2) showed a moderate improvement in performance compared to epsilon = 0.00001. Although AP did not significantly improve, AR increased, illustrating the model’s enhanced capability to estimate a wider range of poses by utilizing occluded parts.

• Epsilon = 0.001: Experimental results (indicated by ▴ in Table 2) displayed no significant enhancement with ResNet50. However, significant improvements were observed with ResNet101 and ResNet152, suggesting that deeper ResNet architectures are more effective at exploiting mask image features.

• Epsilon = 0.1: Experimental results marked with $\mathrm{♢}$ in Table 2 and illustrated in Fig. 6, optimal performance was achieved when the mask and RGB images were equally weighted. An example of the most effective input image is displayed in Fig. 3E. Higher epsilon values resulted in enhancements in pose estimation, confirming the effectiveness of superimposing the grayscale mask image.