TGMF-pose: text-guided multi-view 3D pose estimation and fusion network for online sports instruction

Monocular pose estimation

Two-dimensional pose estimation aims to detect the positions of human body joints from a single image and serves as the foundation for three-dimensional reconstruction. In recent years, Transformer-based architectures have achieved remarkable progress in this domain. PoseFormerV2 (Zhao et al., 2023) enhances the efficiency of long-sequence modeling and robustness to noise by incorporating frequency-domain representations, significantly improving estimation accuracy under occlusions and detection errors. Additionally, MHFormer (Li et al., 2022) introduces a multi-hypothesis Transformer structure that models multiple pose hypotheses in parallel, thereby strengthening the model’s ability to represent complex movements.

Although the Transformer-based models effectively capture global temporal dependencies through self-attention mechanisms, thereby enhancing long-range contextual modeling, they lack explicit constraints on the geometric priors of the human skeleton. This often leads to joint position predictions that deviate from anatomically plausible skeletal structures. In contrast, Graph Convolutional Networks (GCNs) (Li et al., 2025) excel at modeling local connectivity between joints by leveraging skeletal topology, reinforcing structural priors through message-passing mechanisms. However, GCNs struggle to model global temporal dynamics and long-range dependencies across distant joints. To harness the strengths of both approaches, hybrid architectures, such as SPGformer (Fang et al., 2024) and MotionAGFormer (Mehraban, Adeli & Taati, 2024) have been proposed, which integrate GCNs in parallel with Transformer encoders. These models retain structural priors from the graph while benefiting from the global feature learning capacity of Transformers. However, in multi-action instruction scenarios, they failed to propose effective methods for aligning graph structure priors with action semantic features, which often resulted in insufficient matching between keypoint structure information and action categories, thereby leading to ambiguity in semantic representation and action recognition.

Recently, diffusion-based approaches have emerged as a new paradigm for monocular 3D pose estimation by treating pose reconstruction as a generative denoising process. Diff3DHPE (Zhou et al., 2023) leverages the noise distillation capabilities of diffusion models and integrates them with graph neural diffusion to robustly refine noisy 2D keypoints into accurate 3D structures. Similarly, FinePOSE (Xu, Guo & Peng, 2024) introduces fine-grained part-aware textual prompts to guide diffusion-based denoising, demonstrating the potential of coupling language priors with probabilistic motion modeling. KTPFormer (Peng, Zhou & Mok, 2024) further combines kinematic and trajectory priors within a Transformer framework, embedding domain knowledge of human motion into the self-attention process to improve both spatial and temporal reasoning. These methods represent an important evolution beyond deterministic regression approaches by incorporating uncertainty modeling and semantic cues. However, they remain limited to single-view or monocular input settings and often depend on pretrained text encoders that do not explicitly fuse geometric constraints, making them less effective in occluded or multi-action scenarios relevant to online physical education. In contrast, our TGMF-Pose jointly leverages text-guided geometry modeling and multi-pseudo-view generation fusion, thereby enhancing both the semantic and spatial consistency of 3D pose reconstruction.

Multi-camera pose estimation

To overcome the inherent depth ambiguity of monocular vision, multi-camera and multi-frame methods have been widely explored for improving the accuracy and robustness of 3D human pose estimation. Traditional multi-view frameworks such as VoxelPose (Tu, Wang & Zeng, 2020; Moliner, Huang & Åström, 2024) and FastPose (Dong et al., 2019) explicitly reason in 3D space by aggregating features across calibrated cameras. While these methods are robust to occlusions, their reliance on voxelized representations and exhaustive cross-view matching results in high computational cost and limited scalability. MvP (Wang et al., 2021) addressed this limitation through projective attention, directly regressing multi-person 3D poses from multiple views without volumetric reconstruction, achieving both efficiency and accuracy. However, these approaches remain sensitive to view calibration errors and perform poorly under partially overlapping or sparse camera setups.

Recent studies have increasingly adopted Transformer-based architectures to implicitly capture geometric consistency across multiple sensors and views. For instance, MVGFormer (Liao et al., 2024) integrates geometry- and appearance-aware Transformer modules to jointly reason about multi-view and temporal cues, enhancing robustness to occlusion and viewpoint variation. Dual Transformer Fusion (DTF) (Ghafoor, Mahmood & Bilal, 2024) further advances this direction by generating two intermediate views and fusing them via a dual-branch Transformer. These approaches collectively demonstrate that multi-view feature fusion through attention-based modeling can effectively alleviate depth ambiguity while maintaining temporal coherence. Although these hybrid design yields strong robustness under complex occlusions, it substantially increases training cost and system complexity. Moreover, the model’s reliance on synchronized multi-sensor input makes deployment infeasible for single-camera or consumer-level environments typical of online sports instruction.

Beyond explicit geometric fusion, new paradigms have emerged. MV-SSM (Chharia, Gou & Dong, 2025) reframes multi-view estimation as a state-space modeling problem, employing a Projective State Space (PSS) block with bidirectional token scanning. This design markedly improves generalization to unseen camera configurations and demonstrates strong robustness under view sparsity. However, the state-space formulation treats joint dynamics as purely spatial–temporal signals, omitting the semantic or instructional context that defines the intent behind an action. Consequently, while MV-SSM achieves cross-dataset robustness, it remains limited in interpretability and fine-grained motion discrimination.

In contrast, TGMF-Pose eliminates the dependency on multi-sensor configurations by introducing a text-guided multi-view generation and fusion strategy within a single-camera framework. Rather than relying on calibrated cameras or synchronized sensors, it synthesizes pseudo-views through temporally conditioned attention guided by textual prompts and geometric priors. This design allows the model to infer missing keypoints and recover depth with comparable accuracy to multi-sensor systems while maintaining the deployability required for real-time online instruction. By jointly leveraging semantic cues and implicit multi-view reasoning, TGMF-Pose bridges the gap between physically constrained multi-sensor fusion and semantically aware single-view pose estimation.

Online athletic instruction technology

Online physical education (He, 2024) imposes higher demands on feedback accuracy and system deploy ability. PoseCoach (Liu et al., 2022) provides visual feedback for running instruction by comparing 3D skeletal animations, yet its reliance on offline processing makes it unsuitable for real-time interaction in large-scale online teaching. Zhao et al. (2022) proposed a 3D pose feedback framework based on graph convolutional networks, capable of detecting user errors during exercise and offering corrective suggestions, thereby significantly enhancing the safety and effectiveness of personalized training. However, this approach primarily targets specific movements and does not adequately address the challenge of recognizing subtle differences across multiple action categories.

Recent advancements are beginning to explore multimodal and privacy-aware instructional setups. For instance, Lupión et al. (2024) employed multi-view thermal vision sensors to preserve privacy while achieving robust 3D estimation in low-light environments. Meanwhile, ChatPose (Feng et al., 2024) demonstrated the integration of large language models (LLMs) for reasoning about human poses from textual or visual inputs, suggesting a new direction for interactive and explainable sports coaching systems. Although these methods improve interpretability or privacy, they often sacrifice temporal coherence or introduce high computational costs that impede real-time deployment.

Building on these insights, there is an urgent need for a semantically aware 3D pose estimation framework that operates reliably with a single standard camera, enabling real-time interaction and fine-grained movement assessment.

Overall architecture

TGMF-Pose takes a monocular video sequence as input and aims to estimate high-precision 3D human poses for each frame. The overall framework is composed of four key stages: 2D pose extraction, geometric-semantic modeling, 3D candidate generation, and multi-view fusion optimization. Given a sequence of video frames $\mathcal{V}=\{I_1,I_2,\dots ,I_n|n\in N^{+}\}$, we first use ViTPose (Xu et al., 2022) as the backbone network for 2D pose estimation to detect keypoints in each frame, yielding a set of skeletal joints $\xi =\{o_1,o_2,...,o_J\}$ and their connectivity, where $J$ is the number of joints. Based on this skeletal structure, we construct two geometric measurement branches: the first calculates the Euclidean distances between all pairs of joints to form a spatial distance matrix. The second computes angles between each non-leaf joint and its neighboring joints to capture joint-angle constraints in the pose. These geometric features are encoded through a one-dimensional convolutional network to generate structural embedding representation $F_{geom}\in {\text{ℝ}}^{J\times d}$, which serves as the foundational feature for subsequent geometric modeling.

Second, to introduce motion semantic information, we designed a text feature embedding branch. Users or the system can provide a natural language description of the physical activity being performed in the video. We use the parameter-frozen CLIP model (Radford et al., 2021) to generate semantic vectors. Aligning text prompts with geometric structural features effectively enhances the model’s ability to perceive action-specific geometric characteristics. To address the issue of occlusion commonly encountered in 2D pose estimation, we design a Transformer-based multi-view generation module (MVG). This module utilizes self-attention mechanism to model temporal context dependencies and queries available frames near the central frame. Each combination of the central frame and available adjacent frames is treated as a pseudo-multi-view. By embedding temporal position, text prompt, and geometric structure information, MVG generates sets of 3D pose candidate solutions. All generated candidate 3D poses are subsequently refined through a multi-view fusion module (MVF). This module performs feature-level interaction and consistency optimization across views using the cross-attention mechanism to aggregate complementary information among views. MVF can effectively complete the structure of occluded areas and corrects depth estimation, which made the output are better suited for subsequent tasks such as motion correction and performance feedback in online physical education.

Notably, a potential concern when employing pseudo-view generation is that the model may overfit to the synthetic views instead of learning view-invariant 3D geometry. To avoid this risk, the pseudo-view in our work differ from traditional view fusion methods refers to feature-level pseudo-views produced by cross-frame feature querying and aggregation, not to pixel- or image-level synthesis. Concretely, MVG constructs pseudo-view candidates by attending to and re-weighting latent joint embeddings drawn from nearby temporal frames; no decoder is trained to render new images or to fabricate novel visual appearances. This feature-space design prevents the model from memorizing synthetic image artifacts and keeps the learned representations grounded in real observed evidence, improving robustness and avoiding distributional shift associated with pixel-level view generation.

Text-guided geometric embedding module

This module is designed to leverage the spatial structural features between joints and incorporate semantic guidance from text to enable sensitive modeling of inter-class differences in actions. From 2D poses, we extract two types of structural features: the Euclidean distance matrix and the angle feature matrix. For any two joints $p_i$ and $p_j$, the normalized Euclidean distance matrix is defined as:

(1) $$D_{ij}={\displaystyle \frac{\Vert p_i-p_j{\Vert }_2}{ma{x}_{k,l}\Vert p_k-p_l{\Vert }_2},\mathrm{\forall }i,j\in \{1,\dots ,J\}.}$$

Among them, $k,l$ represents the indices of the two joints with the furthest spatial distance. For non-leaf $p_j$ joints, the angle between the adjacent joints $p_{j+1}$ and $p_{j+1}$ is defined as:

(2) $$A_j={\displaystyle \frac{(p_{j-1}-p_j)\cdot (p_{j+1}-p_j)}{\Vert p_{j-1}-p_j{\Vert }_2\cdot \Vert p_{j+1}-p_j{\Vert }_2}.}$$

These normalized geometric descriptors are encoded into feature vectors through convolutional layers $Conv(\cdot )$. Then passed through a multi-layer perceptron (MLP) composed of two fully connected layers with Gaussian Error Linear Unit activations and residual normalization. This MLP projects the geometric features into a latent vector space, ensuring each joint is represented by a compact embedding that preserves relative spatial structure.

(3) $$F_D=Conv(D),F_A=Conv(A)$$

(4) $$F_{geom}=Concat(F_D,F_A)\in {\text{ℝ}}^{J\times d}.$$

Given an action semantic text prompt (e.g., “playing basketball”), we encode it using a pre-trained CLIP text encoder to obtain its embedding $F_{text}$, then expand this embedding into multi-head prompt vectors that match keypoint dimension. To align the two modalities, a cross-modal attention block is introduced, where text features act as queries and geometric embeddings serve as keys and values. This operation generates a text-guided geometric feature, emphasizing spatial regions relevant to the described action.

(5) $$F_{joint}=Attn(F_{geom},F_{text}).$$

$Attn(\cdot )$ denotes the multi-head attention module, $F_{joint}$ represents the semantic perception embedding of joints under text prompts and serving as the input to the subsequent 3D pose generation module.

Multi-view generator

The multi-view generator is designed to infer missing or occluded body parts and to simulate multiple complementary pseudo-views of the same motion from a single input sequence. Its role is to enhance depth perception and occlusion robustness by reconstructing consistent 3D representations from limited 2D evidences. Let the central frame be at timestamp $t$, and a total of $2K$ adjacent frames before and after it form a temporal window:

(6) $$V_t=\{F_{t-K}^{joint},\dots ,F_{t+K}^{joint}\},F_t^{joint}\in {\text{ℝ}}^{J\times d}.$$

Here, $F_t^{\mathrm{j}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{t}}$ represents the geometric feature embedding of the frame after the text prompt is fused. The corresponding text prompt embedding is $F_{\mathrm{t}\mathrm{e}\mathrm{x}\mathrm{t}}$, and the temporal position encoding $E_t^{\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}}$ is used to indicate the relative position of frames. We utilize the self-attention mechanism to model cross-frame contextual dependencies, search for nearby available frames, and concatenate them with the above feature embeddings as input to the MVG to generate pseudo 3D views $P_t^{m,n}$.

(7) $${\tilde{F}}_t=Concat(F_t^{joint},E_t^{time},F_{text})$$

(8) $$P_t^{m,n}=Decoder([V_{t-m},V_t,V_{t+n}],{\tilde{F}}_t)m,n\in [0,K].$$

The architecture of MVG is illustrated in Fig. 1. The input is first normalized and then passed through a Multi-Head Self-Attention (MSA) module. This is followed by another normalization layer and a Multi-Layer Perceptron (MLP) for further feature transformation. Residual connections are employed throughout the process to facilitate gradient flow and improve information propagation. This process essentially leverages the temporal differences in frame perspectives to simulate spatial perspective differences and generate pseudo views.

Multi-view fusion module

This module is designed to perform feature-level fusion of 3D pose estimations obtained from multiple different viewpoints, with the goal of refining predictions and enhancing the accuracy of both depth estimation and keypoint localization.

Taking the fusion of two pseudo-views as an example, let them be denoted as $P_t^{(1)}$ and $P_t^{(2)}$ respectively. A pose encoder $f_{\varphi }$ is introduced to project them into a shared latent space $H$. Using $H_t^{(1)}$ as the query and $H_t^{(2)}$ as the context, a Multi-Head Cross-Attention (MCA) mechanism is employed to fuse structural information from the two viewpoints. The MCA module is composed of multiple parallel attention heads that jointly capture and quantify the inter-view correlations.

(9) $${\tilde{H}}_t=H_t^{(1)}+CrossAttn(LN(H_t^{(1)}),LN(H_t^{(2)})).$$

This process enables structural alignment and contextual information sharing, which refines the 3D pose representation. The fused features are then fed into a 3D pose reconstructor $r_{\psi }$, which regresses the final 3D pose estimation for the central frame, denoted as ${\hat{P}}_t=r_{\psi }({\tilde{H}}_t)$.

To optimize the proposed TGMF-Pose network, a multi-objective loss function is employed to jointly supervise joint consistency and semantic-text alignment. The total loss is computed as follows:

(10) $$L=L_{3D}+\lambda L_{text}$$where $\lambda$ is the loss-term balancing hyperparameter. $L_{3D}$ measures the mean squared error between the predicted (P) and ground-truth (T) 3D joint positions. It is defined as follows:

(11) $$L_{3D}={\displaystyle \frac{1}{J}\sum _{j=1}^J{\Vert T_j^{3D}-P_j^{3D}\Vert }_2.}$$

$L_{text}$ minimizes the cosine distance between the predicted pose and the text embedding in the shared latent space. It is defined as follows:

(12) $$L_{text}=1-\cos (f_{text}(T),f_{pose}({\hat{P}}^{3D}))f_{pose}$$where $f_{text}(\cdot )$ denotes the CLIP-based text encoder, and $f_{pose}$ is a shallow MLP implemented as three feed-forward layers equipped with GELU activations and layer normalization. This module projects the fused 3D pose features into the shared semantic embedding space of the text encoder, thereby facilitating cosine-similarity-based alignment between the visual pose representations and their corresponding textual descriptions.

Action evaluation

The Artificial Intelligence (AI) Teacher System is an intelligent, interactive instructional engine built upon the existing large language model (LLM) architecture, enhanced through Low-Rank Adaptation (LoRA) (Hu et al., 2022), which enables efficient fine-tuning on domain-specific datasets without compromising general linguistic capability. By performing instruction tuning on standardized 3D pose data collected from professional athletes, the system acquires both structural understanding and semantic expressiveness of athletic movements. In practical applications, the system takes as input a sequence of joint coordinates generated by a 3D pose estimation network for each frame, along with user-provided natural language prompts such as the name of the movement. This enables the system to perform a comprehensive analysis that integrates temporal motion trajectories with semantic expectations. The system evaluates the execution quality of key parts (e.g., head, arms, body, legs, action and speed) on an individual basis, producing structured scores and personalized textual feedback. This facilitates fine-grained, interpretable automatic error correction and instructional support.

Notably, Through LoRA-based fine-tuning, the system achieves efficient parameter adaptation while preserving the general language understanding capability of the base LLM, thereby enhancing robustness to real-world textual inputs. It can handle natural language instructions containing typos, informal expressions, or non-technical vocabulary, and maps diverse phrases to standardized motion categories through contextual semantic alignment. The low-rank adaptation also maintains the model’s disambiguation ability, allowing the system to infer ambiguous actions by integrating linguistic semantics with 3D pose data. Furthermore, the attention-based confidence mechanism enables adaptive weighting or clarification when textual uncertainty is high, ensuring stable motion evaluation performance across diverse natural language environments.

Datasets and evaluation metrics

Human3.6M (Ionescu et al., 2013) is one of the most widely used large-scale datasets for 3D human pose estimation. Captured in a controlled laboratory setting, it features 11 subjects performing 15 everyday activities such as walking, sitting, phoning, and eating. Each sequence is recorded from four synchronized camera views and annotated with high-precision 3D motion capture data. The dataset contains approximately 3.6 million image frames, providing per-frame 3D joint coordinates for 17 body joints, along with corresponding 2D projections and RGB images. In our experiments, we follow Protocol #1, where subjects S1, S5, S6, S7, and S8 are used for training, and S9 and S11 are reserved for testing. Owing to its accurate annotations and structured setup, Human3.6M serves as a strong benchmark for evaluating the 3D reconstruction accuracy of pose estimation models under controlled conditions.

MPI-INF-3DHP (Mehta et al., 2017) presents a more challenging benchmark designed to assess the generalization ability of pose estimation algorithms, particularly under unconstrained scenarios such as outdoor environments, occlusions, and viewpoint transitions. The dataset includes 17 activity classes performed by eight actors across diverse indoor (e.g., green screen) and outdoor (e.g., street, grassland) scenes. Several sequences involve varying clothing, props, and partial occlusions. Compared to Human3.6M, MPI-INF-3DHP offers greater variability in background complexity, lighting conditions, and camera angles, as well as a broader range of body poses. It is therefore well-suited for testing the robustness of TGMF-Pose to diverse viewpoints, motion amplitudes, and environmental conditions.

To comprehensively evaluate the performance of our model, we employ three standard metrics: Mean Per Joint Position Error (MPJPE), Percentage of Correct Keypoints (PCK), and Area Under the Curve (AUC). Specifically, MPJPE measures the average Euclidean distance between the predicted and ground-truth 3D joint positions, with lower values indicating higher accuracy in pose estimation. PCK quantifies the proportion of correctly predicted keypoints within a specified error threshold, serving as an indicator of overall pose correctness. AUC represents the area under the PCK curve across varying thresholds and provides a holistic assessment of the model’s keypoint prediction stability and robustness.

Implementation details

All experiments were conducted under a unified hardware and software environment to ensure reproducibility and fair comparison. We used an NVIDIA A100 GPU (40 GB) and ran our code in Python 3.9 with PyTorch 2.0. Each input frame was resized to 256 × 192 pixels and normalized to the [−1, 1] range. A pretrained ViTPose backbone then extracted 17 joint coordinates and their confidence scores. Confidence values were applied as attention masks in downstream modules to enhance robustness against low-quality detections. Action prompts, each limited to 20 Chinese characters to avoid redundancy, were authored by professional sports instructors. We encoded these prompts using a frozen CLIP text encoder (ViT-B/32), producing 512-dimensional embeddings. Each embedding was then expanded into multi-head prompt vectors matching the number of joints for cross-modal attention. Both the Multi-View Generator (MVG) and the Multi-View Fusion Module (MVF) employ a 4-layer Transformer, with 8 attention heads per layer and a hidden dimension of 256. Joint features are encoded via two stacked 1D convolutional layers (kernel = 3, stride = 1). Cross-modal fusion is implemented through multi-head cross-attention, integrating textual and geometric features.

For each centre frame $t_0$, we sample five preceding and five succeeding frames to form a 10-frame temporal window. At video boundaries, missing frames are duplicated. All 2D joint coordinates are normalized to [−1, 1], and their confidence scores serve as optional attention masks. We optimized all models using Adam with an initial learning rate of $1\times {10}^{-4}$. A linear warm-up was applied for the first 20 epochs, followed by cosine-annealing decay over a total of 80 epochs. In the live sports-teaching system, TGMF-Pose splits uploaded video clips into frames, extracts 2D skeletons, and generates fused 3D poses. These 3D poses are then fed into an AI instructor module, which produces natural-language feedback and localized action scores for interactive, interpretable teaching.

Comparison with state-of-the-art methods

To assess the effectiveness of our proposed TGMF-Pose model in online sports instruction scenarios, we conducted a comprehensive comparison using standard benchmarks and compared it with a series of state-of-the-art 3D human pose estimation methods, including Anatomy3D (Chen et al., 2021), PoseFormer (Zheng et al., 2021), MHFormer (Li et al., 2025), P-STMO (Shan et al., 2022), MixSTE (Zhang et al., 2022), and PATA (Xue et al., 2022).

1) Results on Human3.6M Dataset:

All compared methods are evaluated under Protocol #1, which computes the Mean Per Joint Position Error (MPJPE) between the predicted and ground-truth 3D joint coordinates without any alignment. Table 1 presents action-wise MPJPE results for each method across eight commonly benchmarked activities. Our method, TGMF-Pose, achieves the lowest average MPJPE of 56.81 mm, demonstrating a notable margin of improvement over all other methods. In particular, TGMF-Pose outperforms Anatomy3D by 19.85 mm, and reduces error by over 18.88 mm compared to PoseFormer, confirming the efficacy of our multi-view guided estimation and prompt-based skeletal modeling. Compared with Anatomy3D and MHFormer, both of which depend on single-frame or short-range temporal information, our method exhibits stronger resilience in capturing long-term dependencies and handling self-occlusion. For actions involving intricate hand or upper-body interactions (e.g., “Greet” and “Phone”), where keypoints are frequently occluded or spatially entangled, TGMF-Pose achieves superior accuracy by effectively leveraging surrounding frames to reconstruct the occluded joints. While MixSTE also utilizes a spatio-temporal encoder, its simplified token mixing mechanism lacks the fine-grained semantic modeling and cross-view fusion capabilities found in our architecture. Consequently, although MixSTE performs well on relatively structured actions such as “Eating” and “Directions”, it suffers from depth ambiguity in actions like “Photo” and “Pose”, where our approach consistently produces more reliable 3D reconstructions. A key factor contributing to the superior performance of TGMF-Pose is the incorporation of textual prompts into the geometric reasoning process. Unlike PATA, which introduces prompt attention in a relatively shallow fashion, our method integrates text-guided joint relationships into both the view generation and refinement stages. This design allows the model to adaptively emphasize discriminative spatial patterns that differentiate actions with subtle inter-joint variations.

Additionally, the multi-view generator in TGMF-Pose synthesizes diverse pose hypotheses from temporally adjacent frames, capturing alternate perspectives and mitigating the impact of missing or noisy joints. The subsequent fusion module then aggregates these candidate views into a unified representation, significantly enhancing depth estimation robustness. Notably, our method exhibits stable performance across a broad range of action categories. For example, in “Discussion” and “Photo”—scenarios prone to partial occlusion or limited body movement—TGMF-Pose maintains competitive accuracy, thanks to its temporal reconstruction capabilities. In contrast, prior methods often misinterpret such inputs due to over-reliance on frame-local information or limited structural priors.

Figure 2 illustrates one representative case involving a male subject seated with legs crossed and facing away from the camera, where severe occlusion of the lower limbs poses significant challenges to accurate 3D pose recovery. In this example, all baseline methods fail to correctly localize the joints of the occluded leg. Their predictions exhibit noticeable artifacts such as misaligned knee positions, inverted lower limbs, or unrealistic joint angles, which deviate significantly from the ground truth. These errors highlight the limitations of existing models in handling occluded or self-overlapping joint regions, particularly when lacking multi-view context or fine-grained geometric priors. In contrast, our TGMF-Pose model reconstructs the full 3D pose with remarkable accuracy, even in the absence of visible leg joints in the current frame. The predicted keypoints for the occluded knee and ankle align closely with the ground-truth annotations, demonstrating our model’s ability to infer plausible joint configurations based on temporal cues and structural reasoning.

2) Results on MPI-INF-3DHP Dataset:

Table 2 shows that TGMF-Pose achieves the best performance across all evaluation criteria, with a PCK of 98.6%, AUC of 83.3, and MPJPE of only 25.7 mm. These results represent significant improvements over the closest competitor, P-STMO, which achieves a PCK of 97.9% and an MPJPE of 32.2 mm. MHFormer, though effective in clean sequences, suffers performance degradation with an MPJPE of 58.0 mm and lower AUC (63.3), reflecting limited robustness to occlusion and appearance confusion. MixSTE, which employs mixed spatio-temporal encoding, shows better resistance to pose jitter but lacks sufficient semantic modeling, leading to higher MPJPE (54.9 mm). Anatomy3D and PoseFormer perform reasonably but are evidently less capable of handling visual ambiguities, as indicated by their lower PCK and higher MPJPE values.

To complement the numerical findings, Fig. 3 showcases a challenging outdoor scenario captured from the MPI-INF-3DHP dataset. The subject is a woman walking across a grassy field, partially occluded by her own scarf, which drapes over her right arm and introduces a confusing visual cue. In this scene, Baseline models such as MHFormer, PoseFormer, and P-STMO consistently misidentify the scarf as part of the right arm, resulting in visible misalignment in elbow and wrist localization. These methods erroneously extend the arm’s trajectory along the scarf contour. In contrast, TGMF-Pose accurately distinguishes between clothing and actual limb structure, producing a 3D pose that closely aligns with the ground-truth. The wrist and elbow are correctly inferred despite the absence of direct visual evidence, thanks to temporal cue propagation and structural constraints learned through prompt-guided modeling. This qualitative example highlights a critical advantage of TGMF-Pose: semantic-awareness and structural integrity. By embedding text-based priors and enforcing multi-view consistency, our model avoids common confusions induced by background objects or accessories.

In addition, to further assess the generalization capacity and practical utility of the proposed TGMF-Pose framework beyond conventional benchmarks, we conducted an additional evaluation using sports imagery sourced directly from the internet, as illustrate in Figs. S1, S2, and Supplemental Material 1.

Ablation

To thoroughly evaluate the contribution of each core component within the proposed TGMF-Pose framework, we conducted a series of ablation experiments on the MPI-INF-3DHP dataset. All experiments adopt ground-truth 2D keypoints as input to exclude the interference of detection errors and focus purely on the impact of each architectural component.

1) Effectiveness of Text-Guided Prompt Embedding:

We first remove the TGPE module, which is responsible for modeling semantic differences in joint relationships based on textual prompts. As shown in Table 3, this modification causes a performance drop across all evaluation metrics. These results indicate that the introduction of text-based guidance significantly enhances the model’s sensitivity to fine-grained variations in human motion, especially for semantically similar but technically distinct actions. The lack of TGPE weakens the model’s ability to incorporate human-level prior knowledge in joint distance and angle modeling, leading to increased ambiguity in pose interpretation.

To further validate the contribution of TGPE in capturing technical nuances across similar motions within the same sport category, we conducted an additional comparative study across four exercise pairs that are commonly confused in instructional settings due to their visual similarity but require different technical execution: sit-up vs. crunch, squat vs. lunge, push-up vs. plank, and jump vs. high-knee running. As summarized in Table 4, the full model consistently exhibited improved pose accuracy in these ambiguous cases. Notably, in the sit-up vs. crunch pair, the MPJPE decreased from 31.5 to 25.8 mm when TGPE was included, primarily due to better estimation of upper-body joint trajectories where trunk flexion angles are critical. Similar improvements were observed in the squat vs. lunge comparison (from 32.3 to 27.0 mm), where TGPE helped differentiate foot placement and knee alignment that are otherwise visually similar in monocular input.

These results support the hypothesis that TGPE enhances the model’s fine-grained perception by integrating motion-specific semantic priors into geometric reasoning. By embedding text-derived intent into the joint embedding space, TGPE enables the network to interpret not only how a pose appears, but also what it is intended to represent. This effectively bridges the gap between kinematic similarity and task-level meaning. Such capability is particularly beneficial in online sports instruction environments, where subtle misalignments in joint behavior, such as premature trunk elevation during a crunch or insufficient hip descent in a lunge, can result in incorrect form or even injury. TGPE provides a semantic scaffold that allows the model to resolve motion ambiguity that cannot be reliably addressed through spatial features alone.

2) Effectiveness of Multi-View Generator:

To evaluate the impact of the Multi-View Generator, we disable the generation of pseudo-views from adjacent frames, forcing the model to rely solely on the central frame’s features. As reflected in Table 3, the model performance declines significantly. This suggests that MVG plays a crucial role in addressing occlusion and depth ambiguity by reconstructing occluded joints using temporal context. The removal of MVG deprives the network of critical auxiliary views, reducing its capacity to infer accurate spatial structures under motion continuity.

3) Effectiveness of Multi-View Fusion Module:

Lastly, we analyse the role of MVF by replacing it with a simple averaging operation across pseudo-view outputs. This simplification results in a measurable decline in performance, demonstrating that the proposed fusion strategy is not merely a redundant integration step but a key component that adaptively refines pose hypotheses by resolving inconsistencies across views. The MVF enhances the robustness of final predictions by aggregating cross-view evidence in a context-aware manner. The above results clearly demonstrate that all three components, TGPE, MVG, and MVFM, contribute synergistically to the performance of TGMF-Pose. TGPE enhances semantic understanding and structural awareness, MVG provides temporal redundancy and occlusion recovery, and MVF ensures robust view-wise information aggregation. Disabling any of these modules leads to a consistent and non-trivial degradation in pose estimation accuracy, underscoring their necessity for effective application in multi-action online sports instruction scenarios.

In Supplemental Materials 2–6, we present a more comprehensive ablation study covering hyperparameter sensitivity, temporal performance, computational complexity, robustness to textual inputs, and the effect of text-guided prompts on action differentiation. Overall, these findings demonstrate that TGPE enhances the AI teacher’s ability to provide context-aware, task-specific feedback. While feedback without TGPE tends to be generalized, the inclusion of text-based prompts allows the system to offer actionable, expert-like guidance. This makes TGPE a valuable tool in online instruction, where visual cues alone may not be sufficient to address subtle technical issues.