UltraTimTrack: a Kalman-filter-based algorithm to track muscle fascicles in ultrasound image sequences

Introduction

Brightness-mode (B-mode) ultrasonography, or ultrasound, is a non-invasive method for looking under the skin to image the human body’s tissues, which can be applied to study skeletal muscle function during movement (Rutherford & Jones, 1992; Fukunaga et al., 2001). During passive movements and active muscle contraction, ultrasound can be used to visualize the connective tissue around muscle fascicles—bundles of muscle fibers—and their changes in length and orientation. Fascicle length and angle affect a muscle’s force potential (Azizi, Brainerd & Roberts, 2008; Bohm et al., 2019) and metabolic energy expenditure (Joumaa & Herzog, 2013; van der Zee, Lemaire & van Soest, 2019; van der Zee & Kuo, 2021; Beck et al., 2022), thereby affecting movement performance and economy (Fletcher & MacIntosh, 2017; Swinnen et al., 2021; Schwaner et al., 2024). Consequently, quantifying fascicle length and angle changes is important for improving our understanding of in vivo muscle-tendon function (Cronin & Lichtwark, 2013). Recent years have hence seen an increase in algorithms to automate muscle ultrasound image analysis and fascicle tracking, which had previously been limited by the laborious and subjective nature of manual labelling (van Hooren, Teratsias & Hodson-Tole, 2020). However, existing fascicle tracking algorithms are still error prone and have limitations that prevent complete automatization (Ritsche et al., 2024). It would thus be helpful to develop an algorithm that automatically and robustly tracks muscle architectural changes from ultrasound image sequences to more quickly and easily interpret muscle-tendon function during movement.

Optical flow is a commonly used method to track muscle fascicles in a ‘semi-automated’ manner (Gillett, Barrett & Lichtwark, 2013). Optical-flow-based algorithms such as UltraTrack (Farris & Lichtwark, 2016) make a best guess (i.e., least-squares approximation) of the apparent motion between consecutive ultrasound images to track muscle fascicles (Farris & Lichtwark, 2016; Drazan, Hullfish & Baxter, 2019). This method is relatively insensitive to the speckle noise present in ultrasound images because comparing two consecutive frames effectively removes common noise. However, small tracking errors can occur in each frame, because of image brightness changes or variable local pixel motion. Integrated over many frames, these tracking errors accumulate, causing fascicle length and fascicle angle estimates to ‘drift’ away from their original values (Magana-Salgado et al., 2023). To correct for this drift, additional information (e.g., timing of external forces) is required. Additionally, if there is substantial motion between consecutive frames, manual corrections are required to avoid underestimation of fascicle length and fascicle angle changes. Fascicles also need to be defined before tracking, which is typically done via manual labelling. Consequently, while the noise insensitivity of optical-flow-based methods is an important advantage for tracking small fascicle displacements (e.g., those that occur during healthy postural sway (Day et al., 2017)) over a few frames, drift sensitivity and manual labelling requirements limit their accuracy, objectivity, repeatability, and time effectiveness in other conditions (e.g., large fascicle displacements during locomotion).

Unlike optical-flow-based methods, line-detection-based and artificial-intelligence (AI)-based methods attempt to automatically identify line segments in individual muscle ultrasound images. Line-detection-based algorithms such as TimTrack (van der Zee & Kuo, 2022) analyse each ultrasound image independently, making them insensitive to drift (Zhou & Zheng, 2008; Rana, Hamarneh & Wakeling, 2009; Zhou et al., 2012; Zhou, Chan & Zheng, 2015; Marzilger et al., 2018; Ryan et al., 2019; Seynnes & Cronin, 2020; van der Zee & Kuo, 2022). However, because similarities and differences between consecutive images are not considered, these methods are sensitive to the speckle noise present in ultrasound images. Similarly, recently-proposed AI-based algorithms (Bao et al., 2023; Ritsche et al., 2024; Yuan et al., 2024) also analyse each image independently, yielding noisier estimates compared with optical-flow-based methods (Ritsche et al., 2024). Both line-detection-based and AI-based algorithms thus yield relatively noisy estimates of muscle architectural changes during movement that require low-pass filtering within a trial or averaging over multiple trials. Consequently, while line-detection-based and AI-based methods achieve high degrees of automation favoring objectivity, repeatability, and time effectiveness, their sensitivity to noise reduces tracking accuracy and usability. This may explain why recent biomechanical studies (e.g., Swinnen et al., 2024; van Hooren et al., 2024; Beck, Schroeder & Sawicki, 2024; Raiteri, Lauret & Hahn, 2024; Farris et al., 2024) still use UltraTrack for tracking muscle architectural changes during movement, despite its manual labelling requirements and drift sensitivity.

Here, we propose to leverage the key advantages and overcome the main limitations of existing fascicle tracking algorithms by combining existing techniques into a new Kalman-filter-based fascicle tracking algorithm. We chose Kalman filtering (Kalman, 1960) because it is a popular sensor-fusion method that has been successfully employed to correct drift (e.g., Lee & Jung, 2009) and reduce noise (e.g., Alfian, Ma’arif & Sunardi, 2021) in robotics and other fields. The proposed algorithm combines estimates from a noise-insensitive, but drift-sensitive, optical flow method (i.e., Kanade-Lucas-Tomasi optical flow (Lucas & Kanade, 1981; Shi & Tomasi, 1994)) with drift-free, but noise-sensitive, line-detection methods (i.e., object detection and Hough-transform methods (Duda & Hart, 1972)) to yield improved estimates of muscle architectural changes (Fig. 1). We re-used and modified open-source code from existing UltraTrack (Farris & Lichtwark, 2016) and TimTrack (van der Zee & Kuo, 2022) algorithms, and therefore coined the proposed algorithm ‘UltraTimTrack’. The proposed algorithm and its parent algorithms were tested on various B-mode ultrasound image sequences collected from the left-sided medial gastrocnemius muscle of healthy human participants. The proposed algorithm was also applied to two previously collected ultrasound image sequences from the human tibialis anterior and medial gastrocnemius muscles and compared with two recently-proposed algorithms. One of these algorithms considers a hybrid method that combines optical-flow-based and line-detection-based algorithms (Verheul & Yeo, 2023), but lacks a Kalman filter (here referred to as HybridTrack) and the other is the AI-based DL_Track algorithm (Ritsche et al., 2024). We expected UltaTimTrack to be noise-insensitive like optical-flow-based algorithms, and to be drift-free like line-detection-based and AI-based algorithms. Portions of this manuscript were previously published as part of a preprint (https://doi.org/10.1101/2024.08.07.607010).

Methods

Proposed algorithm

To improve the accuracy of the proposed UltraTimTrack algorithm, we first modified one of its parent algorithms as discussed immediately below. We then describe the Kalman filter, followed by the graphical user interface.

Parent algorithms of UltraTimTrack

The proposed algorithm has UltraTrack and TimTrack modules (Fig. 1), which are based on the corresponding original algorithms. However, notable modifications were made to UltraTrack to improve its fascicle tracking performance. Firstly, the UltraTrack module of UltraTimTrack employs Kanade-Lucas-Tomasi optical flow (Shi & Tomasi, 1994), while the original UltraTrack algorithm (Farris & Lichtwark, 2016) employs Lucas-Kanade optical flow (Lucas & Kanade, 1981). The main difference is that Kanade-Lucas-Tomasi only tracks ‘good feature points’ that can be tracked well (in our case, corner points), which improves tracking accuracy while reducing computational cost. The Kanade-Lucas-Tomasi method has been used in more recent optical-flow-based fascicle tracking algorithms (e.g., Drazan, Hullfish & Baxter, 2019), including later and improved, but non-peer-reviewed, versions of UltraTrack (Bakenecker et al., 2022; Raiteri, Lauret & Hahn, 2024; Tecchio, Raiteri & Hahn, 2024). Similar to these versions, MATLAB’s (MathWorks, Inc., Natick, MA, USA) built-in detectMinEigenFeatures, the PointTracker and estimateGeometricTransform2D functions were used to detect feature points and compute optical flow with an affine transformation type, respectively. Optical flow parameters were set to values used in the most recent version of UltraTrack (Raiteri, Lauret & Hahn, 2024). Secondly, unlike any previous UltraTrack version, the UltraTrack module of UltraTimTrack separately computes optical flow for fascicles and aponeuroses. It leverages TimTrack’s detection of fascicle and aponeurosis locations to more precisely identify feature points that are specific to these structures. More specifically, it uses the TimTrack module to identify superficial- and deep-aponeurosis locations within user-specified regions (default: 5-30% and 30-70% of vertical image range, with 0% referring to the top of the cropped ultrasound image). With the region of interest (ROI) type (Fig. 2) set to ‘Hough-local’ (default), feature points are then selected in distinct, local fascicle regions that are based on TimTrack’s detected fascicle locations. These fascicle regions are rectangles of fixed width (default: 10 pixels), centered around the locations of the most frequently occurring lines (default: 10) detected by TimTrack. Alternatively, ROI type can be set to ‘Hough-global’ to allow feature points to be detected within the whole ROI between aponeuroses. In both cases, regions outside the middle portion (default: 80%) of the overall region between TimTrack’s aponeuroses are excluded to avoid detecting feature points on aponeuroses, like in an existing algorithm (Drazan, Hullfish & Baxter, 2019). A fixed number of feature points is selected within the remaining fascicle region (default: 300) using MATLAB’s built-in selectStrongest function.

Kalman filter

We now briefly describe the Kalman filter and then discuss how it was applied to track muscle fascicles. The reader is referred to other literature for a more detailed description of Kalman filters (Kalman, 1960), summarized in Article S1.

A Kalman filter considers the state (vector) z of a dynamical system at time i. It assumes that the current state (vector) z_i depends on state-transition model A, control-input model B, the previous state (vector) z_i−1, the current input u_i, and the current process noise w_i: (1)${\displaystyle z_i=A{z}_{i-1}+B{u}_i+w_i.}$ The process noise w_i is unknown, and assumed to be drawn from a zero-mean normal distribution with variance Q_i. Because the process noise w_i is unknown, state z_i cannot be computed directly. Instead, the Kalman filter can predict state z_i, and update this prediction using a measurement ${\tilde{z}}_i$ to yield an optimal state estimate ${\hat{z}}_i$.

In the proposed implementation, state z_i is obtained through time-integrating a rate-based input u_i. Thus, the modelled system is an integrator, with the state-transition model A equal to unity and the control-input model B equal to the time difference between i and i − 1.

States and measurements

The proposed fascicle tracking algorithm performs Kalman filtering on both aponeuroses and fascicles. For aponeuroses, the states are the vertical (y) positions of the superficial (S) and deep (D) aponeuroses at the left (L) and right (R) image boundaries (i.e., S_L,y, S_R,y, D_L,y, D_R,y). For the fascicle, the states are the fascicle orientation (α, with respect to the horizontal) and the horizontal (x) position of a point p on the fascicle (p_x). For both aponeuroses and the fascicle, UltraTrack’s optical flow is treated as input u, and TimTrack’s line detection is treated as the measurement $\tilde{z}$. An a priori state estimate is obtained by applying UltraTrack to the previous state estimate (equivalent to Az_i−1 + Bu), which is then updated using TimTrack to yield an a posteriori state estimate. More specifically, aponeurosis locations and the fascicle orientation are updated with TimTrack’s aponeurosis location estimates and fascicle orientation estimate, respectively. The fascicle point estimate is updated using a constant that is equal to that defined in the first frame. Finally, a Rauch-Tung-Striebel smoother (Rauch, Striebel & Tung, 1965) is applied to smooth the state estimates (see Article S1 for details).

Fascicle and aponeurosis locations

The vertical positions of the superficial aponeurosis, deep aponeurosis, and fascicle can be computed from the states and expressed as functions of horizontal (x) position, yielding $S\left(x\right)$, $D\left(x\right)$ and $F\left(x\right)$, respectively: (2)${\displaystyle S\left(x\right)=S_{L,y}+\frac{S_{R,y}-S_{L,y}}{r}x}$ (3)${\displaystyle D\left(x\right)=D_{L,y}+\frac{D_{R,y}-D_{L,y}}{r}x}$ (4)${\displaystyle F\left(x\right)=p_y+tan\left(\alpha \right)\left(x-p_x\right).}$ Here, r is the image width and p_y is the vertical position of point p (Table 1 for symbol definitions).

Fascicle tracking estimates

Two main fascicle tracking estimates are derived from the proposed method: (1) fascicle angle ϕ and (2) fascicle length L. Here, we define fascicle angle ϕ as the angle of the fascicle relative to the straight-line projection of the deep aponeurosis in the ultrasound image. First, the superficial and deep attachment points [s_x, s_y] and [d_x, d_y] are found by solving $F\left(x\right)=S\left(x\right)$ and $F\left(x\right)=D\left(x\right)$, respectively. Next, these attachment points are combined with the aponeurosis locations to compute fascicle angle ϕ (i.e., with respect to the deep aponeurosis) and fascicle length L: (5)${\displaystyle L=\sqrt{{\left(s_y-d_y\right)}^2+{\left(s_x-d_x\right)}^2}}$ (6)${\displaystyle \varphi =\text{atan}\left(\frac{s_y-d_y}{s_x-d_x}\right)-\text{atan}\left(\frac{D_{R,y}-D_{L,y}}{r}\right).}$ For UltraTrack, the second term of Eq. (6) was set to 0 because UltraTrack requires additional inputs to track aponeuroses.

Noise estimation

To determine the Kalman gain (see Article S1), the process noise and measurement noise covariances need to be estimated. The measurement noise covariance reflects the uncertainty in TimTrack’s line detection, which is affected by image (speckle) noise. We estimated the covariance of this noise by taking the variance of the high-pass filtered TimTrack aponeurosis locations and fascicle orientations, using a second-order dual-pass Butterworth filter (default cut-off frequency f_c: 2 Hz). The measurement noise covariance of the horizontal position of the fascicle point p reflects how well the initial position value reflects the instantaneous position value. A large covariance allows more movement of this point, but at the cost of more drift (default: 1000 pixels). Considering that optical flow is most accurate for small displacements, the process noise Q was assumed to increase with the square of the optical flow input u: (7)${\displaystyle Q_i=c{u}_i^2.}$ The proportionality constant c is unknown, and it is chosen through trial-and-error (default: 0.01). We later report on a sensitivity analysis on the effect of parameter c on tracking accuracy.

Table 1:

Variables of the UltraTimTrack algorithm.

Symbol	Type	Meaning
SL,y	State	Vertical position of the superficial aponeurosis at the left image boundary
SR,y	State	Vertical position of the superficial aponeurosis at the right image boundary
DL,y	State	Vertical position of the deep aponeurosis at the left image boundary
DR,y	State	Vertical position of the deep aponeurosis at the right image boundary
α	State	Fascicle orientation with respect to the horizontal
px	State	Horizontal position of the fascicle point p
py	Constant	Vertical position of the fascicle point p
r	Constant	Image width
sx	Variable	Horizontal position of the superficial fascicle attachment point
sy	Variable	Vertical position of the superficial fascicle attachment point
dx	Variable	Horizontal position of the deep fascicle attachment point
dy	Variable	Vertical position of the deep fascicle attachment point
L	Variable	Fascicle length
ϕ	Variable	Fascicle angle with respect to the deep aponeurosis

DOI: 10.7717/peerjcs.2636/table-1

Initial state estimate

The initial state estimate ${\hat{z}}_1$ is automatically determined by evaluating the TimTrack module on a user-defined number of stationary initial frames (default: 1). The initial state estimate ${\hat{z}}_1$ equals the mean TimTrack module estimate over these initial frames. Initial frames must be stationary (e.g., at a steady-state passive or active muscle force during no muscle length change). If no stationary frames are available, the initial state estimate ${\hat{z}}_1$ should be determined based on the first frame only.

Graphical user interface

UltraTimTrack has a similar graphical user interface (GUI) as UltraTrack (Fig. 2). It displays the current ultrasound image and the tracked muscle fascicle, alongside the estimated fascicle lengths and fascicle angles from the image sequence. The algorithm can be started using the “Process all” button and reset using the “Clear” button. The “Detect fascicle” button detects a single fascicle from the current image, and the “Process folder” button allows all videos in a specified folder to be processed. Settings include the frame rate and resolution (obtained from the video file), and the image depth to scale pixels to mm (note that if a TVD file is converted to an MP4 file using the provided function, TVD2ALL.m, this information is stored in a MAT file and automatically loaded along with the video). The image resolution can be optionally decreased with a specified factor (“Resolution reduction”) to speed up TimTrack. Under Processing, the user can specify the estimated optical flow process noise covariance parameter c (see “Noise estimation”), the cut-off frequency of the high-pass filtering, the superficial attachment point measurement noise covariance, and the number of stationary frames (see “Initial state estimate”). The “SA” button performs a sensitivity analysis on process noise covariance parameter c. Under “Options”, the user can also check whether to perform parallel processing to speed up computation, and whether to flip the image about its middle vertical axis. Currently, only forward tracking from the first frame of the image sequence is supported.

Results

Participants produced an average ankle plantar flexion MVC torque of 114.5 ± 47.7 N⋅m at 0 deg plantar flexion, and produced mean torques similar to the mean target torques (Table S1) in the sinusoidal trials and ramp-and-hold trials (Fig. 3). Muscle activity changed cyclically over time (Fig. S1), similarly to torque (Table S1). As expected during the sinusoidal trials, UltraTrack’s estimates drifted, TimTrack’s estimates were noisy, while UltraTimTrack’s estimates were insensitive to both drift and noise (Fig. 3A). Similar results were obtained from the ramp-and-hold (Fig. 3B) and passive trials (Fig. 4).

UltraTimTrack’s estimates had lower or similar cycle-to-cycle variability and cumulative deviation compared with estimates of TimTrack and UltraTrack (Fig. 5). Cycle-to-cycle variability of UltraTimTrack’s fascicle length and fascicle angle estimates (1.4 ± 0.4 mm and 0.6 ± 0.3 deg, respectively) was smaller (p = 0.001 and p = 0.002, paired t-tests with Holm-Bonferroni corrections) than TimTrack’s (3.5 ± 1.0 mm and 1.4 ± 0.5 deg), but not different (p = 0.082 and p = 0.195) from UltraTrack’s (1.1 ± 0.3 mm and 0.5 ± 0.1 deg). UltraTimTrack’s cumulative deviation of fascicle length and angle estimates during 0-20% MVC sinusoidal contractions (2.1 ± 1.3 mm and 0.8 ± 0.7 deg) was smaller (p = 0.018 and p = 0.028, respectively) than UltraTrack’s (67.0 ± 59.3 mm and 9.3 ± 8.6 deg), but not different (p = 0.623 and p = 0.476) from TimTrack’s (1.9 ± 2.2 mm and 0.9 ± 1.0 deg). Unlike its parent algorithms, UltraTimTrack’s estimates had both low cycle-to-cycle variability and low cumulative deviation, indicating insensitivity to noise and drift, respectively.

Overall variability of fascicle tracking estimates from UltraTimTrack was generally comparable to or lower than that of its parent algorithms, and less sensitive to sequence duration, image-to-image dissimilarity, and image quality (Fig. 6). For sinusoidal trials, UltraTimTrack was less sensitive to both the sequence duration and image-to-image dissimilarity than UltraTrack, as indicated by significant interaction effects between cycle number and torque range with algorithm type on fascicle length and fascicle angle variability (linear mixed-effects regression with repeated measures, Table 2). For sinusoidal trials, UltraTimTrack had lower overall variability than TimTrack as indicated by independent effects of algorithm, but there were no significant interactions with algorithm type (Table 3). For symmetric ramp-and-hold trials, UltraTimTrack was less sensitive to image-to-image dissimilarity than UltraTrack, as indicated by interactions between ramp rate and algorithm type on overall variability (Table 2). There was no significant interaction between image quality and algorithm type on overall variability, which indicates that UltraTimTrack and UltraTrack had a similar sensitivity to image quality. For ramp-and-hold trials and in comparison to TimTrack, UltraTimTrack had lower overall variability as indicated by independent effects of algorithm, but was more sensitive to both image quality for fascicle length estimates and to image-to-image dissimilarity for fascicle angle estimates (Table 3). For the asymmetric ramp trial, UltraTimTrack had a lower fascicle length and angle variability than both UltraTrack (Table 2) and TimTrack (Table 3), with no differences in sensitivity to image quality (Tables 2–3). For the passive rotation trials, UltraTimTrack was less sensitive to image-to-image dissimilarity than UltraTrack for fascicle length and angle estimates (Table 2). UltraTimTrack had lower overall variability than TimTrack for passive trials, but there was no significant difference in sensitivity to image-to-image dissimilarity (Table 3). Compared with UltraTrack, UltraTimTrack was less sensitive to image-to-image dissimilarity in all three types of conditions (Table 1 and Fig. S2). Overall, UltraTimTrack had similar or lower overall variability than its parent algorithms across a range of conditions, indicating robust fascicle tracking.

UltraTimTrack also compared favorably against recently-proposed HybridTrack and DL_Track fascicle tracking algorithms for the example videos that accompanied these publications (Figs. 7 and 8). Despite notable out-of-plane motion in the tibialis anterior video, both UltraTimTrack and HybridTrack yielded low-noise (i.e., smooth) and drift-free fascicle tracking estimates, but with different amplitudes (Fig. 7). For the medial gastrocnemius video, both UltraTimTrack and DL_Track agreed relatively well with manual tracking, but UltraTimTrack had considerably less noise (Fig. 8). For both videos, UltraTimTrack yielded a smaller RMSD relative to manual estimates than both HybridTrack and DL_Track (Table 4). A sensitivity analysis revealed that UltraTimTrack yielded smaller RMSD’s than both HybridTrack and DL_Track for a range of estimated process noise covariance parameter c values spanning at least ∼4 orders of magnitude (Fig. 9). Videos of fascicle tracking by UltraTimTrack, HybridTrack and manual observers are available as Supplemental files (Videos S1–S2). UltraTimTrack’s processing time was 0.2 s per image (Intel Core i7-10700 CPU @ 2.90 GHz), and thereby 5 times and 8 times shorter compared with HybridTrack (1.0 s per image), and DL_Track (1.7 s per image), respectively (Table 4). When parallel processing was employed (8 cores), UltraTimTrack’s processing time was reduced to 0.1 per image, and thereby 10 times and 17 times shorter compared with these algorithms, respectively. UltraTimTrack thus yields better agreement with manual tracking compared with openly-available state-of-the-art algorithms, at lower computational cost.

Discussion

We proposed a Kalman-filter-based fascicle tracking algorithm that combines optical-flow-based methods with line-detection methods to improve muscle fascicle tracking in ultrasound image sequences. The proposed UltraTimTrack algorithm employs existing UltraTrack and TimTrack algorithms as modules for optical flow estimation and line detection, respectively, and combines outputs from both algorithms to reduce the time burden of fascicle tracking. Unlike existing algorithms, UltraTimTrack’s fascicle length and fascicle angle estimates are low noise and drift free and are obtained at a low computational cost.

The proposed fascicle tracking algorithm yielded estimates without the drift of UltraTrack and without the noise of TimTrack. Less noise and drift were observed for time series from a representative participant (Figs. 3–4) and from participant-average summary measures (Figs. 5–6). Unlike TimTrack, UltraTimTrack’s Kalman filter exploits information from optical flow to reduce noise. This allows UltraTimTrack’s estimates to be smooth even for data from a single contraction. UltraTrack also uses this optical flow information, but without an automatic update step to correct drift. UltraTrack’s estimates therefore drift with each contraction, causing its contraction average to be offset compared with drift-free algorithms (Fig. 3). Averaged over participants, UltraTimTrack had less drift than UltraTrack and less noise than TimTrack (Fig. 5). Less noise and drift resulted in similar or lower overall variability of UltraTimTrack’s estimates across contraction cycles compared with its parent algorithms (Fig. 6). Furthermore, the proposed algorithm was generally less sensitive to factors that are known to affect tracking performance, including sequence duration, image-to-image dissimilarity, and image quality. UltraTimTrack thus provides robust estimates of fascicle length and angle changes during movement compared with existing algorithms.

The proposed fascicle tracking algorithm also performed well compared with a recently-proposed fascicle tracking algorithm that also combined optical flow and line detection procedures. Like UltraTimTrack, a recently-proposed hybrid method (Verheul & Yeo, 2023), here referred to as HybridTrack, yielded low-noise and drift-free estimates, albeit with smaller amplitudes of fascicle length and angle changes (Figs. 7–8). We believe that HybridTrack yields smaller amplitudes because it (1) filters the Hough angle quite heavily, and (2) it uses the similarity transform instead of the affine transform. Its filtering of Hough angles includes a smoothing spline curve, and a 10-point moving average. Employing a similarity transform instead of an affine transform may yield smaller fascicle displacements because shear is neglected, which can be an important contributor to fascicle length and fascicle angle changes (Finni et al., 2017). We would like to point out that this may be an error in HybridTrack, because the accompanying publication mentions that an affine transformation was used. Compared with HybridTrack, UltraTimTrack generally yielded better agreement with manual tracking, while processing time was 5 times shorter (i.e., 0.2 s versus 1.0 s per image, Table 4). UltraTimTrack thus improves upon both tracking accuracy and processing time compared with a state-of-the-art hybrid method.

The proposed fascicle tracking algorithm also performed well compared with a recently-proposed fascicle tracking algorithm that employed machine learning. The recently-proposed DL_Track algorithm (Ritsche et al., 2024) is highly automated, but its estimates are quite noisy (Figs. 7–8). DL_Track’s sensitivity to noise may originate from the fact that it tracks multiple (and different) fascicles in each image. The heterogeneity between line segments of the same and different fascicles can yield a noisy estimate of the dominant fascicle angle and subsequently of fascicle length. DL_Track performed better on the image sequence of medial gastrocnemius (Fig. 8) from its accompanying publication, compared with the image sequence of tibialis anterior (Fig. 7) from the HybridTrack publication (Verheul & Yeo, 2023). For the latter sequence, DL_Track could not detect a single fascicle or the deep aponeurosis in some images and misidentified the aponeurosis in others. Its performance on this sequence may be expected to improve when optimizing some built-in parameters and when re-trained on similar data, but this would require (1) those data being available, and (2) manual labelling. In contrast to DL_Track, UltraTimTrack showed good agreement with manual tracking in both image sequences (Table 4), despite using the same set of parameters. Furthermore, UltraTimTrack’s processing time was eight times shorter than that of DL_Track (i.e., 0.2 s versus 1.7 s per image, Table 4). Overall, UltraTimTrack appears to have clear advantages compared with a state-of-the-art AI-based algorithm in terms of both fascicle tracking accuracy and computational cost.

Ultrasound-based estimates of muscle architectural changes have many potential applications in muscle physiology and movement science fields. For example, musculoskeletal simulations (Falisse et al., 2019; D’Hondt, De Groote & Afschrift, 2024) may benefit from empirical estimates of muscle architectural changes, which could inform model parameters (Delabastita et al., 2020; van der Zee, Wong & Kuo, 2024) or improve model predictions (Dick, Biewener & Wakeling, 2017; Zhang et al., 2022). On the other hand, ultrasound tracking may also benefit from insights obtained through musculoskeletal simulations. For example, future versions of Kalman-filter-based ultrasound tracking algorithms may leverage insights into the dynamics of muscle force development derived from musculoskeletal simulations (van der Zee, Wong & Kuo, 2024) to predict muscle architectural changes from measurements of muscle excitation. Such algorithms could complement existing machine learning algorithms that predict muscle excitations, joint angles and joint torques from ultrasound data (Cunningham & Loram, 2020). Considering the low computational cost (i.e., 0.1–0.2 s per image) and high apparent accuracy of the proposed algorithm (Table 4), ultrasound-derived estimates may also be used as real-time feedback to help control movements, either by a user or an external controller. The feasibility of such a ‘muscle-in-the-loop’ approach was previously shown for a machine-learning algorithm (Rosa et al., 2021). Altogether, ultrasound-based computational methods appear key to understanding the relation between muscle architectural changes and the biomechanics of human movement.

UltraTimTrack has limitations that could be addressed in future updates of the algorithm. For example, the algorithm assumes that fascicles and aponeuroses are straight lines, while they may be curved in certain cases (e.g., at rest and low activation levels (Rana, Hamarneh & Wakeling, 2014; Heieis et al., 2023)). Furthermore, while UltraTimTrack is less sensitive to image-to-image dissimilarity and image quality compared with UltraTrack and TimTrack, respectively, the algorithm still requires clearly identifiable fascicles and aponeuroses. It therefore remains important to capture ultrasound images with both high frame rate and high line density, to allow both small image-to-image dissimilarity and high image quality. If users need to choose between high frame rate and high line density, this decision should be based on the anticipated rate of change in fascicle length and fascicle angle. For image sequences with poor contrast, high levels of blur, and large image-to-image dissimilarities, manual tracking may be necessary. Another limitation is that both the process noise covariance parameter c and superficial aponeurosis measurement noise covariance parameter are unknown. While UltraTimTrack yielded good agreement with manual tracking for a range of parameter c values (Fig. 9), it may be better to inform parameter c based on independent data rather than through trial-and-error. Furthermore, we assumed a constant measurement noise covariance, even though it may be expected to vary between images. In future versions of the algorithm, measurement noise may be estimated for each image individually (e.g., using the variance of a certain number of Hough angles).

Conclusion

We developed a Kalman-filter-based fascicle tracking algorithm that combines existing optical-flow-based and line-detection-based algorithms to yield low-noise and drift-free estimates of fascicle length and fascicle angle from B-mode ultrasound image sequences. The proposed UltraTimTrack algorithm has a low computational cost compared with state-of-the-art algorithms, and may be adapted for real-time fascicle tracking. UltraTimTrack is openly available to facilitate its continuous improvement, and to allow users to utilize its benefits and make modifications to address their specific needs.

Supplemental Information