Deep learning-based fine-grained assessment of aneurysm wall characteristics using 4D-CT angiography

Dataset

This retrospective study included 52 unruptured cerebral aneurysms treated at Asahikawa Red Cross Hospital, Osaka University Hospital, Saiseikai Kumamoto Hospital, and Mie Chuo Medical Center. All patients are Japanese with a gender ratio of 30% male and 70% female. The mean and standard deviation of age are 63.33 and 8.87, respectively. Prior to analysis, patient records underwent a rigorous anonymization process to safeguard privacy.

The 4D-CTA imaging data was acquired using 320-detector CT scanners (Aquilion ONE GENESIS, and Aquilion ONE, Canon Medical Systems Corporation, Otawara, Japan). The collected imaging data was analyzed using a specialized medical image-processing software. The aneurysm-containing region was then precisely identified and extracted. Two key outputs were generated: (1) a 4D-CTA geometry file representing the spatial characteristics of the aneurysm and surrounding structures, and (2) a comprehensive point-trajectory file capturing the dynamic vascular behavior within the region of interest. To analyze vascular motion, we collected the 3D trajectories of each observation point, corresponding to a single pixel in the imaging data on the aneurysm walls, such as middle cerebral artery (MCA; 30 cases), anterior cerebral arteries (ACA; two cases), internal carotid artery—posterior cerebral artery (IC-PC; 14 cases), and anterior communicating artery (A-com; six cases). Voxel tracking was employed for trajectory extraction, and Fig. 1B provides a visual representation of one such trajectory. The trajectories were sampled at a rate of 100 Hz with each trajectory spanning a duration of 1 s. To achieve higher temporal resolution for accurate tracking, a nonlinear interpolation method was applied, with its precision benefits outweighing potential errors. Phase diagrams, derived from the interpolated data and representing displacement, velocity, and acceleration of membrane motion, demonstrated strong consistency with established physical principles. This consistency confirms that enhanced temporal sampling preserves critical biomechanical characteristics while enabling a more detailed analysis of dynamic vascular behaviors.

We annotated each point in aneurysm walls based on RGB images recorded during craniotomies. Note that all surgeries used the same medical microscope model. These microscopes had professional lighting systems with constant illumination, which ensured stable light intensity, color temperature, and uniformity during image capture. As a result, variations from natural or indoor lighting were eliminated. To achieve point-to-point matching between the 4D-CTA data and the labeled points from the RGB image, we implemented a dual-approach point-cloud generation process. First, we created the primary point cloud directly from the original 4D-CTA data. Then, we generated a corresponding point cloud from the RGB images using a pattern-matching approach. This method involved identifying anatomical or vascular structures that were present in both the intraoperative RGB images and the 3D geometry reconstructed from 4D-CTA data. By detecting these shared features and establishing spatial correspondences, we accurately aligned the RGB images with the 3D geometry. This alignment enabled the precise projection of RGB-derived information onto the 4D-CTA geometry file, effectively integrating color and texture details into the 3D representation. Finally, to transfer the color labels to the original point cloud, a nearest-point search was conducted between the primary 4D-CTA point cloud and the newly generated colored point cloud. This comprehensive process resulted in a unified point cloud that seamlessly integrated the color labels and accurately represented the aneurysm’s structural and visual characteristics.

There are 11 annotators with experience ranging from 10 days to 5 years, 6 months, and 24 days. The average expertise is 1 year, 2 months, and 17 days. At least three annotators are involved in a majority vote, and they consistently receive advice from the doctors from the mentioned hospitals throughout the entire annotation process. Additionally, another three to five annotators are engaged in discussions regarding essential parts that require judgment. We performed a thorough verification process including manual review and automated checks to confirm the dataset is complete and free from missing data. We labeled points that exhibited an apparent blue or red coloration, corresponding to thin-walled (TW) or hyperplastic-remodeling (HR) regions, as illustrated in Figs. 2A–2B. Notably, we do not measure the actual thickness for TW and HR; instead, we conduct a relative evaluation as described above. Moreover, the total count of points as TW and HR amounted to 104,000 each. The remaining 312,000 points have no class labels. This study was approved by the Osaka University Research Ethics Committee Institutional Review Board (Reference Number: R 48-7, Approval Number: 201905, Ethical Application Reference: 15000107), and written informed consent was obtained from all participants.

Neural network model

Capturing long- and short-term temporal dynamics plays a pivotal role in point-level assessment. As shown in Fig. 3, we have devised a neural network composed of 1D convolution (CNN) layers and long- short-term memory (LSTM) layers. This architecture enables us to capture short- and long-term trends, respectively. Additionally, we introduced attention layers to focus on important time steps (Vaswani et al., 2017). Note that we employed Glorot uniform initialization (Glorot & Bengio, 2010) for the weights and zero initialization for the biases. The input of the model is a time series of 3D speed, acceleration, and smoothness of motion calculated from a 3D trajectory of a point of interest. The smoothness of motion can be represented by the angles between three adjacent points (p_t−1, p_t, and p_t+1, where p_t is the coordinate at time t) in the trajectory. Consistent angles indicate smooth motion, while highly variable angles suggest unstable or jerky movement. In this study, we considered the angles between three adjacent points in 2D planes of the 3D space: XY, XZ, and YZ planes. The output is a numeric value showing the status of the point: 0 for TW or 1 for HR. In this study, we developed and trained the neural network model for classification¹ using TensorFlow 2.6.2 with the Keras API in Python 3.6.13. The training was conducted in a GPU environment equipped with a ROG-STRIX-RTX4090-O24G (24 GB) GPU, an Intel Core i9-13900KF (13th Gen, 5.8 GHz) CPU, and 64 GB of Trident Z5 RGB DDR5 6,400 MHz RAM.

Model training

The Adam optimizer (Kingma, 2014) was used to train the network based on the following loss function. (1)${\displaystyle E\left({\theta }_f,y_i,\widehat{y_i}\right)=\frac{1}{N}\sum _{i=1}^N{\left(y_i-\widehat{y_i}\right)}^2+\frac{1}{N}\sum _{i=1}^N{L}_p\left({\theta }_f\right)-\frac{1}{N}\sum _{i=1}^N{L}_m\left({\theta }_f\right)}$ where θ_f represents network parameters of the feature extraction layer (i.e., Dense_1 layer), and N denotes the number of training instances. The first term calculates the mean absolute error (MAE) between the ground-truth label (y_i) and estimate (${\hat{y}}_i$). The second term was introduced to extract patient-independent features and L_p() denotes a loss term for a patient-independent feature that calculates the Euclidean distance between data points from different patients with the same class labels in the feature space (output of dense_1 layer). Minimizing this term ensures that points from different patients with the same labels have similar movement features, i.e., patient-independent features. The third term was introduced to leverage unlabeled data and L_m() denotes a loss term for differences in movement features between labeled and unlabeled data that calculates the Euclidean distance between a labeled point (TW or HR) and an unlabeled point in the feature space. Because the unlabeled points do not belong to TW or HR, this term ensures that the unlabeled points have different movement features from the labeled points. Also, we believe that this term helps us acquire the latent relationship between points in TW and HR regions by leveraging the ambiguous points as a proxy.

Evaluation methodology

To evaluate the effectiveness of the proposed method, we prepared the following methods:

• Proposed: The proposed method.

• Baseline: This method is based on an LSTM-based regression model composed of an LSTM layer and two dense layers. The model is trained by minimizing the loss function that calculates only the MAE.

• Only MAE: This is a variation of the proposed method. This method involves training a CNN-LSTM-based attention model by minimizing the loss function, which calculates only the MAE.

• W/o patient-Ind: This is a variant of the proposed method, but it disables the term for patient-independent feature extraction, i.e., the second term in Eq. (1).

• W/o unlabeled: This is a variant of the proposed method, but it disables the term responsible for leveraging unlabeled data, i.e., the third term in Eq. (1).

The above methods were evaluated on the leave-one-patient-out cross-validation. We used accuracy, precision, recall, and F1-score as evaluation metrics. Note that the output of each method is a numeric value. In this study, we used a threshold of 0.5 to distinguish between TW (0) and HR (1).