Automated identification and segmentation of urine spots based on deep-learning

Animal

All animal procedures were strictly conducted according to institutional guidelines and protocols, having obtained approval from the Institutional Animal Care and Use Committee of the Guangxi University (GXU-2024-003). Four male and female mice, aged between 2 and 5 months with a C57BL/6J genetic background were obtained from the Laboratory Animal Center at the Guangxi University. Subsequently, the mice were housed in groups, with a maximum of five individuals per cage, under a 12-hour light-dark cycle while having ad libitum access to water and food (Keller et al., 2018). Euthanasia of the animals was carried out using an intraperitoneal injection of a dosage three times higher than that of pentobarbital sodium before finishing the experiments. This method was employed to ensure humane and ethical treatment of the mice during the euthanasia process.

Spontaneous void spot assay

In this experiment, a double-layer structure device was designed to observe the urination behavior of mice. The size of the upper structure was 27  ×15  × 20 cm. The lower compartment was equipped with a 365 nm ultraviolet (UV) lamp (DJ Black-24BLB; ADJ Lighting, Los Angeles, CA, USA). Neutral filter paper (Xinxing) absorbed the urine of the mice, and the lower compartment had mirrored walls. The UV lamp was activated and videos were captured using a Lenovo computer equipped with Logitech software and a video camera with 1,280 × 720 pixel resolution. Under these conditions, urine spots deposited on the filter paper could be detected by UV illumination. The experimental animals were domesticated in the behavioral chamber for three consecutive days before the experiment (Fig. 1A) (Verstegen et al., 2019).

Image acquisition and analysis

After the experiment was completed, the recording device was turned off and a video describing the urine diffusion process was generated. The filter paper at the bottom was air-dried and returned to its original position, while the overhead infrared light was turned off. UV light was utilized to enhance the visibility of the urine spots, and images were captured with a camera. Throughout the process, we ensured that the pixel dimensions of both images and videos were 1,280 × 720, and collected mouse urination videos as training samples (Fig. 1B). The boundary information of each urination point was manually labeled (Fig. 2A). Next, we used our customized AutoVSA software for analysis. Figure 1 shows an example of filter paper under UV irradiation. Filter paper photos were saved in JPEG format.

Conversion from area to volume

Mouse urine was used as an intermediate medium to enable quantitative determination of urine volume. Different volumes of urine were pipetted onto experimental neutral filter paper, and the spot area was converted to volume by constructing a standard curve correlating the known urine volume to the corresponding urine spot area on the filter paper (Fig. 1C) (Sugino et al., 2008). When the volume was below 100 microliters, there was a linear relationship between volume and area with an R² value of 0.9973 (Fig. 1D). Thus, urine volume can be accurately calculated based on different filter paper types and urine spot sizes.

Statistical analysis

Statistical analysis was performed by comparing the two independent groups. For normally distributed data, the Student’s t-test was used. In cases where the data deviated from a normal distribution, a nonparametric test for independent samples was used. p-values below 0.05 were considered statistically significant differences.

AutoVSA

AutoVSA is a deep learning-based approach for automatically segmenting VSA images and videos. It comprises a detection network and a tracking network to perform instance segmentation and location tracking of urine spots, utilizing the YOLACT (Bolya et al., 2019) and DeepSort (Pujara & Bhamare, 2022) frameworks.

Detection framework

In this study, the YOLACT algorithm is employed for the spot detection. The algorithm integrates rapid localization with precise pixel-level segmentation techniques. YOLACT uses Feature Pyramid Networks (FPNs) and Depth Separable Convolution for feature extraction. An anchor frame mechanism predicts the target’s location and species. The second stage involves a series of prototypes and a novel linear combination of coefficients, significantly speeding up the segmentation process by dynamically generating segmentation masks for each instance and predicting mask coefficients concurrently (Fig. 2B).

Tracking framework

The tracking framework primarily relies on the DeepSort algorithm. This algorithm encompasses state estimation, trajectory processing, and matching challenges. It leverages target motion and appearance data to reduce object ID switches. The Kalman filter predicts the target position and is integrated with the Hungarian algorithm and Intersection of Union (IoU) for trajectory estimation. Time and position data are recorded to track urination points in videos (Yadav et al., 2022). The marsupial distance quantifies motion information correlation, reflecting uncertainty in state measurements (Durve et al., 2023; Veeramani, Raymond & Chanda, 2018). Given the relatively slow diffusion rate of urine spots, emphasizing motion information correlation as the primary matching metric is instrumental in analyzing the dynamics of the urine spots (Fig. 2C).

Training of AutoVSA

Due to the limited data, image enhancement techniques like translation, rotation, and affine transformation were applied during training to produce three derivative images from each original image. A total of 2,600 images were utilized for the training and analysis of the deep learning model. All images have a uniform resolution of 1280*720 pixels .The LabelMe image annotation tool generated a JSON format file, segmenting the image dataset into a 9:1 training-to-validation ratio. The pre-trained YOLACT model and PyTorch framework were selected for parameter tuning, employing a stochastic gradient descent (SGD) optimization algorithm with a learning rate of 0.001, momentum of 0.9, and weight decay of 0.0001. Post-training, the model was applied to urine spot images and validated against manually labeled data (Guan et al., 2017; Xu et al., 2023). Leveraging the robust performance of ResNet-50 in image classification, YOLACT was chosen to achieve accurate pixel-level segmentation.

Postprocessing

Watershed algorithms have proven effective in manually analyzing overlapping urine spots, particularly in quantifying and resolving overlaps. We utilized these algorithms for automated segmentation to address the separation of overlapping mouse urine spots. The method applies a watershed algorithm to automatically segment the target region’s gray gradient by mimicking water flow, eroding filled edges, locating the center of mass, and reconstructing the boundary. This automated process effectively distinguishes non-concentric overlapping circles into distinct regions, accurately representing individual mouse urine spots (Fig. 3).

Evaluation and statistical analysis

To ensure the accuracy of the results, this study conducted experimental evaluations on a standard test dataset that is widely recognized as a benchmark for rigorously testing the capabilities of instance segmentation methods. The raw image data collected were organized into the standard COCO dataset format after data preprocessing of the images. In these experiments, this article uses commonly used metrics such as AP (average precision) and the mean average precision (mAP) to evaluate the accuracy of the detection algorithms (Ran et al., 2023). mAP, as a commonly used evaluation metric in the field of computer vision, is widely used in tasks such as instance segmentation, object detection, and image categorization, and can provide a more comprehensive evaluation result (Cattaneo et al., 2020; Sanchez, Romero & Morales, 2020).

The study employs a suite of metrics—Precision (P), Recall (R), F1 score, and mean, average precision (mAP)—to benchmark the efficacy of surgical tool detection algorithms, see [22] for more details on these parameters. ${\displaystyle \mathrm{P}\left(\text{\%}\right)=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}\times 100}$

${\displaystyle \mathrm{R}\left(\text{\%}\right)=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}\times 100}$ ${\displaystyle \mathrm{F}1\left(\text{\%}\right)=\frac{2\mathrm{PR}}{\mathrm{P}+\mathrm{R}}}$ ${\displaystyle \mathrm{AP}={\int }_0^1\mathrm{PdR}}$ ${\displaystyle \mathrm{mAP}=\frac{1}{\mathrm{n}}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{AP}}_{\mathrm{i}}.}$

Precision quantifies the ratio of accurately identified positives within the set of predicted positives, while Recall assesses the proportion of true positives out of all actual positives. In this context, true positives (TP) denote correctly identified positive instances, false positives (FP) denote incorrectly labeled positives, true negatives (TN) refer to correctly labeled negatives, and false negatives (FN) indicate wrongly labeled negatives. The computation of P, R, and F1 follows established formulae. Importantly, mAP serves as a metric for evaluating object detection accuracy, averaging the AP across categories, derived from the area under the Precision-Recall (PR) curve. Here, ‘n’ represents the category count. The research adopts prevalent COCO challenge metrics for object detection assessment, including AP and its variants AP₅₀, AP₇₅, AP_S (small object area), AP_M (medium object area), and AP_L (large object area), with AP₅₀ and AP₇₅ denoting AP at IoU thresholds of 0.5 and 0.75, respectively. The metrics AP_S, AP_M, and AP_L denote the Average Precision (AP) for object detections of varying scales, with AP_S applying to bounding boxes under 32² pixels, AP_M to those within 32² to 96² pixels, and AP_L to those exceeding 96² pixels. Given the dataset’s exclusive inclusion of large surgical instruments, characterized by bounding boxes surpassing 96² pixels, computation of AP_S and AP_M is rendered irrelevant. Moreover, within this dataset, the AP_L metric is observed to converge with the overall AP measure. AP₅₀ represents the average accuracy when the IoU threshold is set to 50%. If the IoU between the predicted bounding box and the true bounding box is greater than or equal to 50%, it is considered to be a correct detection. AP₅₀ is more lenient, which reflects the model’s ability to detect larger-sized speckles or targets with more distinctive boundaries. AP₇₅ denotes the average accuracy when the IoU threshold is set to 75%, which is a more stringent evaluation criterion. AP₇₅ can be a more precise measure of the model’s detection accuracy for small-sized spots or targets with less distinct boundaries.

Comparison with different architectures & Fiji

Comparative analyses using Fiji and AutoVSA showed that the points on the scatterplot were in close agreement with the equation line, indicating that there were no significant differences between the mice and the corresponding controls in terms of number, area, and frequency across the 30 images (Fig. 5).

Furthermore, this study implemented dynamic detection and segmentation of urine spots in video data through a GUI interface using AutoVSA (Fig. 6). The method has the advantage to accurately detect and segment detailed information for each urination event, including spot number, frequency, interval, and location (Fig. 7). In summary, AutoVSA demonstrates robust performance on both image and video data at various scales, offering significant advantages over other tools. This evidence supports the method’s feasibility and validity in assessing the volume of tiny cavities in normal mice (Table 2).

The results indicate that AutoVSA offers higher accuracy and reliability in urine spot detection and segmentation tasks compared to the traditional ImageJ method. It effectively recognizes and segments urine spots in various images and excels across different image scales, significantly outperforming existing tools. The study confirms AutoVSA’s feasibility and effectiveness in assessing urinary output in mice, providing a precise and efficient tool for studying lower urinary tract function.