Cloud-to-Thing continuum-based sports monitoring system using machine learning and deep learning model

Aim and scope

This research aims to develop a comprehensive sports monitoring system that seamlessly integrates cloud and edge processing for real-time analysis. The scope encompasses data acquisition from various sources, preprocessing for quality enhancement, feature extraction for essential information capture, cloud-based processing for in-depth analysis, continuum paradigm integration for localized processing, and decision-making based on integrated analysis. The system focuses on basketball sports events and aims to improve player performance evaluation, team strategies, and tactical adjustments.

Research objectives

• 1)

  To acquire sports video data from diverse sources including live feeds, recorded matches, and archival footage.

• 2)

  To preprocess acquired data for noise reduction, frame alignment, and resolution adjustment.

• 3)

  To extract relevant features using ML and DL techniques for player, action, and event identification.

• 4)

  To perform cloud-based processing for in-depth analysis and application of machine learning algorithms for activity recognition and event detection.

• 5)

  To integrate edge devices within the continuum paradigm for localized processing, ensuring real-time analysis and reducing latency.

• 6)

  To make informed decisions regarding player performance, team strategies, and tactical adjustments using fuzzy decision-making techniques.

Overall problem statement

Sports monitoring systems often face real-time analysis, resource utilization, and decision-making challenges. Latency issues, data quality, and inadequate feature extraction techniques hinder accurate performance evaluation and strategic decision-making. Additionally, integrating cloud and edge processing seamlessly poses technical challenges. This research addresses these challenges by proposing an innovative system that leverages ML, DL, and fuzzy decision-making techniques to achieve real-time analysis, optimize resource utilization, and improve decision-making in sports monitoring.

Particle swarm algorithm

The stochastic optimisation technique known as the particle swarm algorithm takes its cues from the population dynamics of many animal groups, including schools of fish and flocks of birds. There are two primary study methods that are often associated with PSO algorithms, and these are evolutionary algorithms and artificial algorithms. In terms of optimisation, this technique outperforms genetic optimisation approaches. This operation is carried out using the velocity vector and the swarm size of each particle’s location vector in a finite dimension. From the starting point, the swarm experiences the ideal positions of each person and updates their optimal positions. Revising, taking into account both position and velocity, the ideal location of each individual using swarm. As an expression of the position update formula,

(2) $$y_{\mathrm{i}}^{q+1}=y_{\mathrm{i}}^q+{\mathrm{P}}_{\mathrm{i}}^{q+1}.$$

Iteration is represented by q, particle by i, the particle’s location by $y_{\mathrm{i}}^q$, and the particle’s velocity by ${\mathrm{P}}_{\mathrm{i}}^q$. And to top it all off, the following is the updated velocity formula that takes inertia weight into account:

(3) $${\mathrm{P}}_{\mathrm{i}}^{q+1}=\mathrm{s}\ast [{\mathrm{P}}_{\mathrm{i}}^q+\mathrm{l}{3}_1{\eta }_1\left({\mathrm{R}}_{\mathrm{i}}^q-y_{\mathrm{i}}^q\right)+\mathrm{l}{3}_2{\eta }_2\left({\mathrm{R}}_{\mathrm{i}}^q-y_{\mathrm{i}}^q\right)].$$

The current stored velocity is represented by $\mathrm{l}{3}_1{\eta }_1$ and $\mathrm{l}{3}_2{\eta }_2$, the iteration’s pbest and gbest are ${\mathrm{R}}_{\mathrm{i}}^q$ and ${\mathrm{R}}_{\mathrm{i}}^q$, the social factors of the stochastic algorithm are ${\eta }_1$ and ${\eta }_2$, and the weight factor of inertia is $\mathrm{s}$. It gets around the problems with the HSA algorithm. Compared to other heuristic optimization methods, this one offers superior computing efficiency because of its efficient control settings.

It circumvents the problems with the HSA algorithm. Because of its efficient control settings, this method offers superior computing efficiency compared to other heuristic optimization methods.

Data acquisition

The first step is to gather video data related to sports from several sources, like live streams, recorded contests, or archive material. Additional information, such as game outcomes, ambient factors, and player stats, may also be included in the data.

Dataset: The APIDIS dataset contains seven camera images and is open to the public. The video files are captured in MPEG-4 at a frame rate of 25 fps with a resolution of 800 $\times$ 600, and they include 1,500 frames. There are 16 unlabeled periods in the dataset as well. The dimensions of the court for basketball are 2,797 cm $\times$ 1,499 cm. Two officials, two five-player teams, and a total of twelve players make up the court. Lighting, reflections, and shadows make the publicly accessible dataset difficult to work with. Our group at Shantou University has just amassed a new dataset, which we call the STU dataset. There are eight cameras in this time-synchronized dataset. The MPEG-4 video files are captured at a frame rate of 24 fps with a resolution of 1,280 $\times$ 720. Two time periods, totaling 2,500 frames, have been used in our investigations. The basketball court’s dimensions are 2,800 cm $\times$ 1,500 cm. Two referees and two teams of eight make up the sixteen players on the court. Eleven whole periods make up the dataset. We plan to release it at a later date.

The basketball court dimensions used in this study must align with standard measurements for accurate analysis and application. In the National Basketball Association (NBA), the standard court size is 94 feet by 50 feet (approximately 28.7 m by 15.2 m). Under the rules of the International Basketball Federation (FIBA), the court is slightly smaller, measuring 28 m by 15 m. To ensure consistency with these standards, the system in this study has been calibrated to support data collection and analysis based on these official court dimensions. This standardization facilitates the applicability of the proposed system in professional and international basketball settings.

This research follows ethical norms with great respect regarding data collection and usage, which is very important because player data is delicate. Before any data collection, an ethical review, for instance, by an Institutional Review Board (IRB) or any appropriate ethical review body, is performed to evaluate and accept the study design. Specifically, all subjects who participated, including athletes and athletic trainers, were oriented with the reason, extent, and localization associated with the utilization of gathered information. Clear instructions on how to pursue ethical practices concerning voluntary participation are given to everyone involved in the investigation. Special identification restrictions are imposed wherever applicable, especially in the preprocessing period, to protect participant information from being traced to specific persons. Further, only authorized personnel have access to the data, and safe data storage methods are followed to avoid misuse of the information. The research also considers local legislation on protecting data collected during the study, which applies to all the studies that intersect the boundaries of the European Union or other regional laws. These measures protect and journalist all participants in the research and the rights of all participants involved in the research which helps the advancement of ethics in the research and development of the sports monitoring system.

Preprocessing

The collected data is preprocessed to improve the quality and make further analysis easier. Adjusting the resolution, stabilizing the video, reducing noise, and aligning the frames are all examples of preprocessing operations. The first step is to use the threshold to filter out the noise. The image’s noise is also eliminated to improve the picture quality. To achieve this goal, we put forth the Thresholding Adaptive median filter (TAMF), which can remove noise and produce point clouds of excellent quality.

The noisy picture is denoted as $\check{\mathrm{K}}(\mathrm{l})$, while the noisy word is $\check{\mathrm{K}}(\mathrm{r})$. Here is the representation of these inputs in formulation form:

(4) $$\check{\mathrm{K}}\left(\mathrm{l}\right)=\mathrm{S}\left(\mathrm{o}\right)+N(8)$$

(5) $$\check{\mathrm{K}}\left(\mathrm{r}\right)=\mathrm{S}\left(\mathrm{o}\right)+N(8).$$

In this case, $\mathrm{S}\left(\mathrm{o}\right)$ stands for the initial data and $N(8)$ for the background noise. The noise reduction process begins with converting the photos to greyscale. The median filtering is subjected to a sliding window with dimensions $(\mathrm{\partial }=vxv)$. The RGB-D picture’s pixels, which are ranked from tiny to big according to their grey value, are represented by the symbols $\check{\mathrm{K}}\left(\eta \right)$. The number that represents the median of a pixel is ${\check{\mathrm{K}}}_{med}$.

With ${\check{\mathrm{K}}}_{min}$ being the lowest grey value in the window and ${\check{\mathrm{K}}}_{max}$ being the highest, we can express the grey value of the centre pixel as ${\check{\mathrm{K}}}_{\zeta }$. The original information of the picture ${\check{\mathrm{K}}}_{\zeta }$ is characterised as $\zeta$ if it is not equal to ${\check{\mathrm{K}}}_{min}$ or ${\check{\mathrm{K}}}_{max}$; otherwise, it is regarded as noise. To accomplish denoising, replace ${\check{\mathrm{K}}}_{\zeta }$, with ${\check{\mathrm{K}}}_{med}$.

The adaptive adjustment is made to the window size so that the filtering template may be larger. The symbol $\check{\mathrm{K}}\left(\xi \right)$ represents the pixel value in this filter window. The two stages of adaptive median filtering are as follows:

Step 1: If ${\check{\mathrm{K}}}_{min}<{\check{\mathrm{K}}}_{med}<{\check{\mathrm{K}}}_{max}$ shift to step 2 otherwise, adjust the window as $v=v+2$. When $\mathrm{\partial }>{\mathrm{\partial }}_{max}$, then the output is ${\check{\mathrm{K}}}_{med}$ otherwise, repeat step 1.

Where, ${\mathrm{\partial }}_{max}$ denotes the window’s upper limit.

Step 2: If ${\check{\mathrm{K}}}_{min}<{\check{\mathrm{K}}}_{med}<{\check{\mathrm{K}}}_{max}$ then the output is $\check{\mathrm{K}}\left(\xi \right)$ otherwise ${\check{\mathrm{K}}}_{med}$.

Step 2 doesn’t save borders and other information that are crucial for picture smoothing, and it may lower the window’s upper limit. To reduce the residual noise while maintaining the edge and other crucial information, the enhanced wavelet threshold function is used. There are two kinds of frequency coefficients in this threshold function: low-frequency coefficients, which include image information, and high-frequency coefficients, which contain noise and edge information. Applying the wavelet cutoff function to high-frequency coefficients helps to remove noise while preserving relevant information. There are two types of threshold functions that are taken into account: soft and hard. A representation of the hard threshold function would be,

(6) $$h_{th}=\{\begin{array}{cc}h\hfill & \left|h\right|\ge t\hfill \\ 0\hfill & \left|h\right|<t\hfill \end{array}$$where $h_t$ denotes the wavelet threshold. The soft threshold function is represented as,

(7) $$h_{ts}=\{\begin{array}{c}sign(h)(\left|h\right|-t),\left|h\right|\ge t\\ 0,\left|h\right|<t.\hfill \end{array}$$

At $\left|h\right|=t$, the enhanced wavelet threshold function becomes continuous, and it gains the overall definition domain with different sorts of slopes from $\left|h\right|<t$ and $\left|h\right|\ge t$. This is the revised wavelet threshold function put into formula form:

(8) $$h_{ti}=\{\begin{array}{c}sign(h)(\left|h\right|-t),\left|h\right|\ge t\\ \left(1-\lambda \right)|h|,\left|h\right|<t\hfill \end{array}$$where $\lambda$ stands for the factor that may be adjusted. During the interval $\left|h\right|<t$, the threshold function is not equal to zero because of the adjustment factor $\lambda (0<\lambda <1)$. Although the soft threshold function removes the majority of the noise from pictures and noisy points in point clouds, this demonstrates that the wavelet coefficients are somewhat less compressed. To increase detection accuracy, we convert points to voxels at the second level using cloud points to provide a better perceptual picture. The three coordinates $(X,Y,Z)$ that each cloud points to have three sets of corresponding values, such as $(\left[x_{min},x_{max}\right],\left[y_{min,}{y}_{max}\right],\left[z_{min},z_{max}\right])$. The point cloud data is first organised into voxel subsets according to the Cartesian coordinate system, with each subset being represented by an index $U\left(d,f,c\right)$.

Where $d\in \left[0;L_x-1\right],f\in \left[0;L_y-1\right]$, and c $c\in \left[0;L_z-1\right]$. The count of voxels $({\mathrm{\Delta }}_x,{\mathrm{\Delta }}_y,{\mathrm{\Delta }}_z)$ in each direction is given according to the individual voxels’ dimensions $(L_x,L_y,L_z)$.

(9) $$L_x=\frac{(x_{max}-x_{min})}{{\mathrm{\Delta }}_x}+1$$

(10) $$L_y=\frac{(y_{max}-y_{min})}{{\mathrm{\Delta }}_y}+1$$

(11) $$L_z=\frac{(z_{max}-z_{min})}{{\mathrm{\Delta }}_z}+1$$

In this context, ${\mathrm{\Delta }}_x,{\mathrm{\Delta }}_y,{\mathrm{\Delta }}_z$ indicate the voxel sizes, while $L_x,L_y,L_z$ stand for the voxel counts from left to right. The following description of pseudocode serves as a concise explanation of noise reduction (Algorithm 1).

Feature extraction

The next step is to extract important characteristics from the cleaned data in order to record crucial details about the participants, their activities, and the events that took place. Image processing, object identification, motion estimation, and posture estimation are some of the approaches that feature extraction may use. We generate region boxes using improved Mask R-CNN (IMR-CNN), which helps to achieve high accuracy of pose estimate by eliminating the misalign issue of classic Mask RCNN, and we consider both point-based and local-based semantic information for semantic segmentation.

In classic Mask R-CNN, the classification head is responsible for ROI classification and bounding box regression. A quicker R-CNN-based instance segmentation approach, the improved Mask-R-CNN is an expansion of that model. A modification to the mask’s head block allows for more accurate boundary identification, which in turn improves the mask-RCNN. The first step in learnable upsampling is to add a decoder layer to feature maps; this effectively increases the spatial resolution. The second step is for the ROI align block with skip connections to align the ROI. This will provide features with high resolution, which the adjusted mask head may then use.

Cloud-based processing

For further processing and analysis, the extracted characteristics are sent to the cloud. The use of deep learning and machine learning algorithms in cloud computing allows for the analysis of player motions, identification of activities, and performance evaluation. Activity identification and event detection may be achieved using techniques like GAN with a hybrid metaheuristic algorithm. The fundamental concept of GANs is the rivalry between two neural networks. Proposed architecture is shown in Fig. 2. The generator and discriminator are the two main components of GANs. The generator’s job is to trick the discriminator into thinking the inputs are real, while the discriminator’s job is to tell the difference between the two. Think of a neural network (generator) $gene(\mathfrak{Z};{\theta }_{\mathfrak{G}})$ that maps the output sample of $(\mathfrak{Z}\sim {\mathcal{P}}_{\mathfrak{Z}})$ to the input samples of $(\mathfrak{x}\sim {\mathcal{P}}_{\mathfrak{G}})$. The output of this neural network called $Discr(\mathfrak{x};{\theta }_{\mathfrak{O}})$, is a binary value that is determined by the inputs. The GAN’s goal is to train both neural networks in tandem using each other’s outputs; this training process is like a min-max game. The goal statement is,

(18) $$\underset{gene}{min}\underset{Dicr}{max}\mathcal{V}(Discr,gene)={\mathbb{E}}_{\mathfrak{x}\sim {\mathcal{P}}_{data}}\left(\mathfrak{x}\right)\left[\log Discr\left(\mathfrak{x}\right)\right]+{\mathbb{E}}_{\mathfrak{Z}\sim {\mathcal{P}}_{\mathfrak{Z}}}(\mathfrak{Z})\left[\log (1-Discr(gene(\mathfrak{Z})))\right]$$

The loss function of GANs is among the distributions of ${\mathcal{P}}_{\mathfrak{Z}}and{\mathcal{P}}_{\mathfrak{G}}$ respectively based on the kullback-Leibler distribution. The loss function can be formulated as,

(19) $$Disc{r}_{kl}\left({\mathcal{P}}_{\mathfrak{Z}}||{\mathcal{P}}_{\mathfrak{G}}\right)={\int }_{-\mathfrak{x}}^{\mathfrak{x}}{\mathcal{P}}_{\mathfrak{G}}(\mathfrak{x})\mathrm{l}\mathrm{o}\mathrm{g}\frac{{\mathcal{P}}_{\mathfrak{G}}(\mathfrak{x})}{{\mathcal{P}}_{\mathfrak{Z}}(\mathfrak{x})}d\mathfrak{x}.$$

An algorithm that combines the HSA with the PSO is a hybrid metaheuristic. We need to update the HS parameters frequently using the PSO algorithm since the HSA method quickly slips into local optima. There are three stages to the proposed HSA: initiation, improvisation, and updating. In this case, improvision is revised for three procedures: random selection, pitch adjustment, and memory consideration. Harmony memory (HM) is a part of the HSA algorithm that stores the method’s goal function in harmony vectors. First, set up the HM and swarm size to start with so you can evaluate the fitness function, which is stated as follows:

(20) $$\left[\begin{array}{cccc}{h}_1^1& h_2^1& \cdots & h_d^1\\ h_1^2& h_2^2& \cdots & h_d^2\\ ⋮& ⋮& ⋮& ⋮\\ h_1^x& h_2^x& \cdots & h_d^n\end{array}\right]\left[\begin{array}{c}{f}_1\\ f_2\\ ⋮\\ f_n\end{array}\right]=\left[\begin{array}{c}{F}_1\\ F_2\\ ⋮\\ F_n\end{array}\right]$$

In this case, distance and location assess the fitness function. A fresh harmony vector $[h_1^{\prime },h_1^{\prime }\dots ,h_t^{\prime }]$. is generated by performing harmonic improvisation once initialization is finished. A new harmony vector component $h_j^{\prime }$ is created using the following formula.

(21) $$h_j^{\prime }\leftarrow \{\begin{array}{c}{h}_j^{\prime }\in P_{HM}withPbofHMCR\hfill \\ h_j^{\prime }\in h_{j}withPbof(1-HMCR)\end{array}$$where HMCR stands for the component selection probability defined by the harmonic memory consideration rate, The formula for determining the pitch adjustment for the chosen $h_j^{\prime }$ is given below.

(22) $$h_j^{\prime }\leftarrow \{\begin{array}{c}h{n}_j\in P_{HM}withPbofPAR\\ h_j^{\prime }withPb(1-PAR).\hfill \end{array}$$

The parameter PAR stands for the pitch adjustment rate, $P_{HM}$ for the suggested hybrid HSA and PSO method, and $h_j^{\prime }$ for the ideal controller position.

The goal function value for each $P\mathrm{\_}Best$ is used to compute the new hybrid harmony vector. For $P_{HM}$ to take into account a new harmony vector, its objective value must be higher than that of the worst harmony. According to $P_{HM}$, the harmony vector with the poorest value is discarded. The best particles in the swarm, denoted as $P\mathrm{\_}Best$, are used to identify the ideal location for the controller to be placed. It iteratively creates n particles, with the best one being named $G\mathrm{\_}Best$. Every velocity updates the particle’s location and velocity. Here is how the current location and velocity are defined:

(23) $$v_{ij}\left(R+1\right)=W\left(R\right)v_{ij}\left(R\right)+C_1{R}_1\left(P\mathrm{\_}Bes{t}_{ij}-P_{ij}\left(R\right)\right)+C_2{R}_2(G_{Best}-P_{ij}\left(R\right))$$

(24) $$P_{ij}\left(R+1\right)=P_{ij}\left(R\right)+v_{ij}\left(R+1\right)$$where $v_{ij}(R)$ and $P_{ij}(R)$ are the velocities and positions of the particles, $W\left(R\right)$ is the weight value, $C_{1}and{C}_2$ are the acceleration constants, $R1,R2$ are the random values between $[0,1]$, and $P\mathrm{\_}Best$ and $G\mathrm{\_}Best$ are the local and global positions of the particles, respectively. The following is a definition of the weight value calculation:

(25) $$W\left(t\right)=W_u-(W_u-W_l)\left(\frac{i}{I_{max}}\right)$$where represent the total count of iterations and $i$ represent the current iteration, and represent the upper and lower limit of the weight values. These processes are continued till met the termination criteria. Table 1 describes the parameters of HSA algorithm.

In this context, $I_{max}$ stands for the maximum number of iterations, i for the current iteration, and $W_u$ and $W_l$ for the maximum and minimum values of the weights, respectively. Until the termination requirements are satisfied, these procedures will be maintained. Figure 3 shows the process flow for hybrid optimization. Figure 4 illustrates the latency.

Continuum paradigm integration

In parallel, edge devices within the continuum paradigm receive a subset of the data for localized processing.

Edge devices perform real-time analysis to provide immediate feedback and support time-sensitive applications. This integration ensures a balance between centralized cloud processing and distributed edge processing, optimizing resource utilization and reducing latency. The results from cloud-based and edge-based processing are fused and integrated to comprehensively analyze sports events. Fusion techniques combine insights from both sources to enhance accuracy, completeness, and reliability.

Decision making

Based on the integrated analysis, decisions are made regarding player performance, team strategies, and tactical adjustments. Decision-making algorithms may consider player statistics, historical data, game context, and coach preferences. Evaluate the fuzzy weights and fuzzy alternative rates by edge decision makers using sets of linguistics variables $\left(L{V}_2\right)$, which can be used to improve the fuzzy model by instances.

The fuzzy weight instances are Very low, low, medium-low, low, very high, medium-high, and medium, whereas the fuzzy alternatives instances are very poor, poor, fair, good, medium good, very good, medium poor, and good.

The decision matrix of fuzzy weights of the users and applications is represented as $w_h\left(h=1,2,\dots ,n\right)$, for the $r-th$ norm provided by the $h-th$ decision maker can be formulated as,

(26) $$W^{rh}=\left[w_{1h}\dots w_{rh}\dots w_{lh}\right].$$

The policy ratings of User (U) and Application (A) decision matrix can be represented as $U{A}_h\left(h=1,2,\dots ,n\right)$ for the $i-th$ alternative with respect to $r-th$ criterion provided by the $h-th$ decision maker can be represented as,

(27) $$U{A}_h=\left[\begin{array}{ccccc}u{a}_{11h}& \cdots & u{a}_{1rh}& \cdots & u{a}_{1lh}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ u{a}_{i1h}& \dots & u{a}_{irh}& \dots & u{a}_{ilh}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ u{a}_{b1h}& \dots & u{a}_{brh}& \dots & u{a}_{blh}\end{array}\right].$$

Step 2: Construction and weighted matrix defuzzification

The weighted policy rating matrix can be computed by multiplying each set of policy ratings $\left\{{\mathrm{\exists }}^{irh}=\left({\mathrm{\exists }}^{irh1},{\mathrm{\exists }}^{irh2},{\mathrm{\exists }}^{irh3},{\mathrm{\exists }}^{irh4}\right)\right\}$ with fuzzy weights $W^{rh}=W^{rh1},W^{rh2},W^{rh3},W^{rh4}$ which can be calculated by,

(28) $${\mathrm{\exists }}_{irh}^W={\mathrm{\exists }}^{irh}\times W^{rh}=\left({\mathrm{\exists }}_{irh1}^W,{\mathrm{\exists }}_{irh2}^W,{\mathrm{\exists }}_{irh3}^W,{\mathrm{\exists }}_{irh4}^W\right).$$

The matrix form of weighted policy rating is represented by,

(29) $$U{A_h}^D=\left[\begin{array}{ccccc}u{a}_{11}^z& \cdots & u{a}_{1r}^z& \cdots & u{a}_{1l}^z\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ u{a}_{i1}^z& \dots & u{a}_{ir}^z& \dots & u{a}_{il}^z\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ u{a}_{b1}^z& \dots & u{a}_{br}^z& \dots & u{a}_{bl}^z\end{array}\right].$$

The normalization of $U{A_h}^D$ can be computed as follows,

$${\mathrm{\exists }}_{irh4}^{z+}=\underset{i}{\mathrm{m}\mathrm{a}\mathrm{x}}\left\{{\mathrm{\exists }}^{irh4}\right\},forbenefitnorm$$

$${\mathrm{\exists }}_{irh1}^{z+}=\underset{i}{\mathrm{m}\mathrm{i}\mathrm{n}}\left\{{\mathrm{\exists }}^{irh1}\right\},forcostnorm.$$

The max, min function is used for differentiate the gathered fuzzy values. The normalized matrix can be represented as,

(30) $${{\overline{UA}}_h}^D=\left[\begin{array}{ccccc}{\overline{ua}}_{11}^z& \cdots & {\overline{ua}}_{1r}^z& \cdots & {\overline{ua}}_{1l}^z\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ {\overline{ua}}_{i1}^z& \dots & {\overline{ua}}_{ir}^z& \dots & {\overline{ua}}_{il}^z\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ {\overline{ua}}_{b1}^z& \dots & {\overline{ua}}_{br}^z& \dots & {\overline{ua}}_{bl}^z\end{array}\right].$$

The defuzzification of normalized weighted policy rating matrix is normalized which can be represented as,

(31) $$\overline{DF}\left({{\overline{UA}}_{ih}}^D\right)=\frac{DF\left({{\overline{UA}}_{ih}}^D\right)}{{\sum }_{L=1}^{b}DF\left({{\overline{UA}}_{Lh}}^D\right)},h=1,2,\dots ,n.$$

The normalized $\overline{DF}\left({{\overline{UA}}_{ih}}^D\right)$ can be expressed as,

(32) $${{\overline{UA}}_h}^D=\left[\begin{array}{ccccc}{\overline{DF(ua}}_{11}^z)& \cdots & \overline{DF}({\overline{ua}}_{1r}^z)& \cdots & {\overline{DF(ua}}_{1l}^z)\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \overline{DF}{\overline{(ua}}_{i1}^z)& \dots & \overline{DF}({\overline{ua}}_{ir}^z)& \dots & \overline{DF}{\overline{(ua}}_{il}^z)\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \overline{DF}{\overline{(ua}}_{b1}^z)& \dots & \overline{DF}({\overline{ua}}_{br}^z)& \dots & \overline{DF}{\overline{(ua}}_{bl}^z)\end{array}\right].$$

Algorithm 2 computes fuzzy weights, normalizes decision matrices, and applies defuzzification to aid in decision-making processes.

Data privacy and security considerations

Protecting susceptible player information is paramount in the designed Cloud-to-Thing Continuum-based Sports Monitoring System. To ensure that the data’s confidentiality and integrity are not compromised, various security measures have been built along the data lifecycle. During the exchange and transfer of data between the edge devices and the cloud, all the data is secured through encryption known as Advanced Encryption Standard AES-256, and secure communications like TLS 1.3 are used so that no interception and/or tampering is possible. Furthermore, to preserve the communication and policy analysis of the individual players’ data, parameters like k-anonymity and differential privacy are adhered to to avoid disclosing identifiable features. Access to the system is implemented through multi-level permissions, multi-factor authentication, and role-based access control RBAC, so authorized users only access sensitive information. Data at rest is also safe with the assistance of strong encryption algorithms. Observance of regulatory cycle time managing prohibitions ensures that retained data is only limited to what is strictly necessary, and afterward, comprehensive data deletions are carried out. There is also an exoneration of the relevant requirements for the institution regarding international data protection laws such as General Data Protection Regulation (GDPR) and Health Insurance Portability and Accountability Act (HIPAA), which show positive attitudes to data and player privacy protection.

Several cyber threats have been assessed and described using a systematic risk management framework. Regular security reviews and penetration testing are carried out to enhance the system. The system proposed in this work complements real-time analysis and low latency with privacy and security and the proper management of sensitive player data.

Experimental setup

The proposed Cloud-to-Thing continuum-based sports monitoring system has a practical approach, and it is expected that its hardware setup is valid and adequate in scale to provide effective data collection, processing, and analysis as well as fit in a future Centria cloud aided smart platform. To the data acquisition layer, HD cameras with frame rates of not less than 25–30 fps are suggested for detailed recording of the player movements. Support for such cameras should be ensured to a great extent around the playing area to reduce the problem of occlusions. In the case of data preprocessing for images acquired, small-scale computers along the lines of NVIDIA Jetson Nano or Raspberry Pi with Intel Movidius Neural Compute Stick can carry out Network Virtual Internet Protocol (NVIP) tasks such as noise removal and simple feature extraction. Such devices are effectively suited for edge computing since they are not only low-power but no brute force computing can be undertaken. As pose estimation and action recognition relying on deep learning demand more resources, computational resources of a centralized cloud infrastructure are necessary. This can be achieved with the help of cloud services such as AWS or Google Cloud or Microsoft Azure, backed up with GPU instances such as NVIDIA Tesla V100 or A100, capable of computing quite expansive deep learning models. The architecture of the cloud facility needs to include phones to support auto-scaling to enable optimization of infrastructure by shifting only the required resources when necessary. It is also vital for edge devices and the cloud to have a good and high bandwidth communication link to avoid data transfer delays between the cloud and edge processing. A hybrid approach is optimal for data retention, where some copies are physically held on municipal gaining devices for temporary storage while the rest are archived in standard databases. Contemporary installations must have local storage for high-definition video images of about 1 TB capacity or greater AWS or Google Cloud for devices like Amazon S3 or Google Cloud Storage, especially for Cloud computing devices. The training also incorporated data augmentation techniques, such as brightening, rotation, translation, etc, to increase the training set’s effectiveness and avoid overfitting. The data was distributed into train/test/validation sets in the following proportions: 70%, 15%, and 15%, respectively. The training was conducted in deep learning models for pose estimation based on Improved Mask R-CNN architecture and classification using hybrid metaheuristic GAN with dominantly applied supervised and unsupervised learning methods. The models were trained using the Adam optimizer with the following configurations: default base learning rate 0.001, batch size 16, and early stopping by validation loss for overfitting prevention. The training used NVIDIA’s Tesla V100 GPU with Tensor Flow and Pytorch libraries. The average training time in a single epoch of the models was about 45 min between epochs, and 50 epochs were applied for the model’s training.

Result analysis

The performance of the Cloud-to-Thing continuum-based sports monitoring system was validated using basketball games to determine player tracking, action recognition, and accuracy of decision-making. The system attained a latency of about 5.1 ms as compared to other methods like deformable convolution and adaptive multiscale features which recorded rather higher latency of 10.2 and 8.3 ms for recurrent neural networks. This further indicates that it is possible to get a system that can allow time-sensitive feedback, which is critical for sports monitoring. The system was relatively accurate with a score of about 94.25% while other methods scored about 78.34 for DC-AMF and 83.2 for RNN methods showing a great step in enhancement for analyzing player performance as well as detecting events. Besides, there was also the consideration of how the system would be fair with respect to different volumes of data. The team achieved a scalability of 6.75 while DC-AMF achieved around 4.5, and for RNN, it was 5.5, implying that the system can be effective when applied to more significant amounts of data without drastically deteriorating performance.

Evaluating the system’s performance entailed several measurements, including accuracy, precision, recall, F1 score, and latency. In measurement set A, accuracy was described as a fraction of the number of relatable player actions and movements against the total number of instances in the data. Accuracy considered how many correctly predicted events in a case were true, whereas recall was used to determine the fraction of the exciting events captured. Using the F1 score was ideal in this case as it was a better alternative in incorporating the aspects of recall and precision. Latency was also a key and necessary metric that was included, and it is the average period it takes between a collection of the data and its activity in actionable insights; we aimed at less than 10 ms of latency. To assess the system’s robustness, it was subjected to increasing data loads and processing times without modifications, and the respective level of accuracy was evaluated.

The system achieved an accuracy of 94.25%, with a precision of 93.8% and a recall of 94.7%. The F1-score of 94.25% indicates a well-balanced model performance. The latency was measured at 5.1 ms, demonstrating the system’s capability for real-time processing. In terms of scalability, the system maintained a high accuracy (above 90%) even with a 50% increase in data volume, showcasing its robustness and efficiency in handling large-scale datasets.

The proposed flow undergoes iterative refinement based on feedback and performance evaluation. Continuous improvement efforts focus on enhancing accuracy, reducing latency, improving scalability, and addressing emerging sports monitoring and analysis challenges. By following this proposed flow, sports organizations can effectively leverage cloud-to-continuum computing paradigms to enhance sports monitoring and analysis, improving player performance, team strategies, and overall fan engagement. When compare the existing methods such as deformable convolution and adaptive multiscale features (DC-AMF) (Xiao et al., 2023) and recurrent neural network (RNN) (Alghamdi, 2023). Latency is defined as the amount of delay time taken between the users’ request and the applications’ response. In general, latency is caused due to policy generation and trust management. Low latency provides efficient network. The comparison of latency is based on number of tasks in Table 2 and Fig. 4

Accuracy is defined as how the proposed methods are performed in proposed and existing methods and are represented in Fig. 5 and Table 3.

The significant reduction in latency and improved accuracy can be attributed to the efficient integration of cloud and edge processing, which allows for a balanced distribution of computational tasks. Improved Mask R-CNN for pose estimation and the hybrid metaheuristic algorithm combined with GAN for classification enhanced the precision of player tracking and action recognition. The system’s scalability score suggests it can be effectively deployed in different sports environments with varying data volumes, broadening its applicability beyond basketball to other sports with similar monitoring needs.

The proposed system demonstrates a clear advantage over existing methods regarding real-time processing and accuracy. For example, the improved latency makes it suitable for applications that require immediate feedback, such as in-game tactical adjustments and player health monitoring. The high accuracy also implies that the system can reliably identify complex actions and events, essential for detailed performance evaluation and strategic planning.

Scalability is high in proposed methods because in the cloud to the continuum is defined as how proposed methods are performed in proposed and existing methods are represented in Fig. 6 and Table 4.

Scalability is a decisive factor to consider within the context of the proposed Cloud-to-Thing continuum-based sports monitoring system, especially in its use across different sports environments with varying amounts of data being generated. To this end, the system can scale up efficiently because of the use of a modular design, which allows for altering the system architecture depending on the nature of the sport being monitored. For example, in situations such as in the sport of basketball, where players are constantly moving with frequent contacts, the system uses edge devices to process high-definition video feed to reduce latency and the amount of feed sent to the cloud. On the other hand, in sports such as golf, the athletes’ movements are more discrete, and the data gathered will not be as much in that situation. The system can afford to leverage cloud processing more. To accommodate different volumes of data, the system is built on SoftLayer infrastructure, including cloud infrastructure, which has automatic resizing functionality where virtual machine deployment and resource allocation adjust to the level of demand. This means that the system will be able to cater to high and low system loads during on and off-peak operating hours without degradation of speed. Further, the system is equipped with a data fusion layer, which facilitates the use of devices with heterogeneous data sources, such as video feeds, various sensors, and contextual information, by converting the input data formats into the agreed interface and incorporating them into the processing pipeline. This variability allows the system to be used in different sports, from confined halls with controlled lighting and fewer variables to outdoor sports with several complexities. In future work, we aim to test the system under large-scale scenarios in different sports and environments and its ability to provide effective low latency even in such conditions. Considering these aspects, it is evident that the system is not deficient but capable of scaling data size and application utilization in various sports scenarios.

The results presented in Table 5 demonstrate the superior performance of the proposed system across multiple evaluation metrics compared to existing methods like deformable convolution and adaptive multiscale features (DC-AMF) and recurrent neural networks (RNN)-based models. The proposed system achieved an accuracy of 94.25%, significantly higher than the 78.34% and 83.2% obtained by DC-AMF and RNN models, respectively. This indicates the system’s ability to identify player movements and actions in diverse sports scenarios accurately. Precision and recall were also notably higher for the proposed system, with values of 93.8% and 94.7%, respectively, reflecting its capability to minimise false positives and ensure comprehensive detection of relevant events. The F1-score, which balances precision and recall, was 94.25%, further confirming the system’s robustness in handling complex player actions.

Table 6 clearly and concisely presents the mean performance values and their variability, enhancing the validity and reliability of the findings.

While the current study focuses on basketball data to demonstrate the system’s capabilities, the proposed Cloud-to-Thing continuum-based sports monitoring system is designed to be adaptable to a wide range of sports environments. The modular architecture and flexible data processing framework enable the system to accommodate varying requirements, such as different types of player movements, game dynamics, and environmental conditions in sports like soccer, tennis, or cricket. For example, in soccer, the system can be applied to larger fields and accommodate more players, while in tennis, it can focus on high-precision tracking of player positions and ball trajectory in a more confined space. To enhance the generalizability of the findings, future work will involve testing the system on diverse datasets from multiple sports, including contact and non-contact sports, individual and team sports, and indoor and outdoor environments. This will include acquiring and integrating datasets from publicly available sources and collaborations with sports organizations to gather sport-specific data. By validating the system’s performance across various sports, we aim to demonstrate its robustness and versatility, ensuring that the proposed framework can provide accurate, real-time analysis in diverse sports scenarios. Expanding the dataset diversity will improve the system’s applicability and allow for the development of sport-specific features, thereby enhancing the overall effectiveness of the monitoring system.

Challenges and limitations

While the system showed promising results, several challenges were encountered during the implementation. For instance, occlusions and overlapping players occasionally led to identity-switching errors in player tracking. To mitigate this, additional post-processing steps, such as trajectory smoothing and re-identification algorithms, were implemented, resulting in a 5% reduction in tracking errors. Moreover, the system’s performance in highly cluttered environments, such as crowded basketball courts with multiple non-player objects, slightly decreased due to the difficulty distinguishing between relevant and irrelevant objects. Future work will focus on enhancing object detection capabilities and incorporating context-aware models to improve performance in such scenarios.