Segmentation and classification of colon glands with deep convolutional neural networks and total variation regularization

CNN architecture

The general architecture of both CNNs is motivated by the LeNet-5 architecture (LeCun et al., 1998), and consists of K = 7 (k = 1, …, K) layers: four convolutional layers (Conv k) for feature learning and three fully connected (FCk) layers as feature classifiers, see Fig. 4. Rectified linear unit (ReLU) nonlinearities (f(x) = max(0, x)) are used as activation functions throughout all layers in the networks. All convolutional layers consist of a set of learnable square 2D filters, followed by ReLU activation. Subsampling (max-pooling) layers (Subk, 2 ×2 ) are used after the first three convolutional layers. The final pixel-wise classification of an input image is obtained by sliding a window of size 101 ×101 pixels over the image, and classifying the center pixel of that window. Differences between the two CNN architectures are due to the smaller field of view that is required for modeling the boundaries between glands, as opposed to the classification into benign/malignant glands or background.

For training, minibatch stochastic gradient descent with momentum, weight decay, and dropout regularization is used to minimize a negative log-likelihood loss function. Training the networks for classification is thus equivalent to minimizing a cross-entropy loss. For a more detailed explanation of this widely used setup and the involved parameters, we refer the reader to (LeCun et al., 1998) and (Goodfellow, Bengio & Courville, 2016).

Object-Net: classifying gland objects

The goal of the Object-Net is to predict the probability of a pixel belonging either to a gland or to any other tissue structure, i.e., background. Although this could be formulated as a binary classification task, the unique features of benign and malignant tissues, which are not found in the other tissue type, allow a more specific exploitation of these features. We therefore formulate a classification problem, in which we distinguish four classes C_l, l = {0, …, 3}: benign background (C₀), benign gland (C₁), malignant background (C₂), and malignant gland (C₃). This requires a transformation of the ground truth labels available in the Warwick-QU dataset, which now also indicate whether the gland is of benign or malignant type. Hence, a new label is assigned to pixels belonging to each class C_l, see Fig. 5.

The input to the CNN is an image patch I(x) of size 101 × 101 pixels, centered at an image location x = (u, v)^⊤, where x ∈ Ω and Ω denotes the image domain. A given patch I(x) is convolved with 80 filters (11 × 11) in the first convolutional layer, in the second layer with 96 filters (7 × 7), in the third layer with 128 filters (5 × 5), and in the last layer with 160 filters (3 × 3), see Fig. 4A. The three subsequent fully connected layers FC5-FC7 of the classifier contain 1,024, 512, and four output units, respectively. The output of FC7 is fed into a softmax function, producing the center pixel’s probability distribution over the labels. The probability for each class l is stored in a corresponding map I_Cl(x).

Separator-Net: classifying gland-separating structures

Initial experiments have shown that taking pixel-wise predictions only from the Object-Net were insufficient in order to separate very close gland objects. Hence, a second CNN, the Separator-Net, is trained to predict structures in the image that separate such objects. This learning problem is formulated as a separate binary classification task using the manually created label annotations defined in ‘Training dataset sampling’.

As depicted in Fig. 4B, the CNN structure is similar to the Object-Net: a given input image patch I(x) of size 101 × 101 pixels is convolved with 64 filters (9 × 9) in the first convolutional layer, in the second layer with 96 filters (7 × 7), in the third layer with 128 filters (5 × 5), and in the last layer with 160 filters (3 × 3). The three subsequent fully connected layers FC5-FC7 of the classifier contain 1,024, 512, and two output units, respectively. The output of the last layer (FC7) is fed into a softmax function to produce the probability distribution over the labels for the center pixel. The probability for a pixel x belonging to a gland-separating structure is stored in the corresponding probability map S(x).

Refining Object-Net outputs

Once all probability maps have been obtained, the Object-Net predictions I_Cl(x) are refined with the Separator-Net predictions S(x) in order to emphasize the gland borders and to prevent merging of close objects. The subsequent figure-ground segmentation algorithm requires a single foreground and background map to produce the final segmentation result, so outputs are combined as follows.

The foreground probability map p_fg is constructed by (1)${\displaystyle p_{fg}\left(\mathbf{x}\right)=max\left\{\left(\sum _{l\in \left\{1,3\right\}}{I}_{C_l}\left(\mathbf{x}\right)\right)-S\left(\mathbf{x}\right),0\right\},}$ and the background probability map p_bg accordingly: (2)${\displaystyle p_{bg}\left(\mathbf{x}\right)=min\left\{\left(\sum _{l\in \left\{0,2\right\}}{I}_{C_l}\left(\mathbf{x}\right)\right)+S\left(\mathbf{x}\right),1\right\}.}$

Gland segmentation

Quantitative evaluation metrics were computed for gland detection (F1-score), segmentation overlap (Dice index) and shape similarity (Hausdorff distance) at the object-level. The manually annotated object having the maximum overlap with a segmentation hypothesis is considered the associated ground truth for that segmentation. A minimum area overlap of 50% between them is required to consider a detection as true positive (TP), otherwise it is considered a false positive (FP). Remaining ground truth objects having less than 50% overlap with a segmentation are counted as false negative (FN). The F1-score measures the gland detection performance and is defined by (10)${\displaystyle \mathrm{F1}=\frac{2\cdot \mathrm{PRC}\cdot \mathrm{REC}}{\mathrm{PRC}+\mathrm{REC}},}$ where precision is defined by (11)${\displaystyle \mathrm{PRC}=\frac{TP}{TP+FP}}$ and recall by (12)${\displaystyle \mathrm{REC}=\frac{TP}{TP+FN}.}$

The pixel-level segmentation performance is evaluated using the Dice coefficient (Dice, 1945), which measures the overlap between two sets of pixels: (13)${\displaystyle \mathrm{Dice}\left(G,S\right)=\frac{2\cdot |G\cap S|}{\left|G\right|+\left|S\right|},}$ where G and S denote the set of ground truth and segmented pixels, respectively, and |⋅| denotes the cardinality of a set. The object-level Dice coefficient represents an integrated value for how well ground truth and segmentation, and segmentation and ground truth overlap, respectively. It is defined by (14)${\displaystyle {\mathrm{Dice}}_{\mathrm{obj}}\left(G,S\right)=\frac{1}{2}\left[\sum _{i=1}^{n_G}{\tilde{\omega }}_i\mathrm{Dice}\left({\tilde{G}}_i,{\tilde{S}}_i\right)+\sum _{i=1}^{n_S}{\omega }_i\mathrm{Dice}\left(G_i,S_i\right)\right],}$ where ${\tilde{G}}_i$ denotes the set of pixels belonging to the ith ground truth object, and ${\tilde{S}}_i$ the set of pixels in a segmented object that maximally overlaps with ${\tilde{G}}_i$. Conversely, S_i denotes the set of pixels belonging to the ith segmented object, and G_i the set of pixels in a ground truth object that maximally overlaps with S_i. n_G and n_S are the total numbers of individual ground truth objects and segmented objects, respectively. The weighting terms ${\tilde{\omega }}_i=\left|{\tilde{G}}_i\right|∕{\sum }_{j=1}^{n_G}\left|{\tilde{G}}_j\right|$ and ${\omega }_i=\left|S_i\right|∕{\sum }_{j=1}^{n_S}\left|S_j\right|$ capture the relative area of an object i in the respective sets.

Shape similarity is assessed using the Hausdorff distance, which is defined by (15)${\displaystyle \mathrm{HD}\left(G,S\right)=max\left\{{\sup }_{x\in G}\underset{y\in S}{inf}\parallel x-y\parallel ,{\sup }_{y\in S}\underset{x\in G}{inf}\parallel x-y\parallel \right\}.}$ At the object-level, the shape similarity is measured between all segmented objects and all ground truth objects using the object-level Hausdorff distance: (16)${\displaystyle {\mathrm{HD}}_{\mathrm{obj}}\left(G,S\right)=\frac{1}{2}\left[\sum _{i=1}^{n_G}{\tilde{\omega }}_i{\mathrm{HD}}_{\mathrm{obj}}\left({\tilde{G}}_i,{\tilde{S}}_i\right)+\sum _{i=1}^{n_S}{\omega }_i\mathrm{HD}\left(G_i,S_i\right)\right].}$

Tissue classification

The classification performance for benign and malignant tissue is computed from a 2 × 2 confusion matrix M in terms of overall accuracy (17)${\displaystyle \mathrm{ACC}=\frac{\mathrm{tr}\left(\mathbf{M}\right)}{\sum _i{\mathbf{M}}_i},}$ where tr(M) denotes the trace of the confusion matrix, and i the ith element of the matrix. Similarly, tissue classification performance is reported class-wise as F1-score, precision, and recall using Eqs. (10)–(12). Please note that different to the definition given for segmentation above, here we refer to TP as true positive, FP as false positive, and FN as false negative classified cases, i.e., entire images.

Training dataset sampling

For the sake of execution speed when using a sliding window approach for pixel-wise classification, the images are rescaled to half resolution prior to classification with the CNNs, and upsampled with bilinear interpolation to their original size before applying the TV segmentation. The size of the input patch I(x) is chosen to be 101 × 101 pixels, such that sufficient contextual information is available to classify the center pixel.

The majority of training images (79) have a size of 775 × 522 pixels, and rescaling reduces them to 387 ×261 pixels. If only the valid part without border extension would be considered for sampling the patches for the training dataset, approximately 46% of the labeled pixels would be lost when using a patch size of 101 ×101 pixels. On the other hand, a significant number of boundary artifacts would be introduced by artificially extending the border. Fortunately, most images in the training set are tiles of a bigger image and can thus be stitched seamlessly to obtain a total of 19 images (Fig. 6). From these images, enough patches can be sampled without heavily relying on artificial border extension. In one case full stitching was not possible, since only three tiles were available. These three tiles, and the remaining six images that were not part of a bigger image, were treated as individual images.

In principle, the same sampling strategy was pursued for the Separator-Net, but it was necessary to create the ground truth labels manually. We annotated all pixels that belong to a structure very close to two or more gland borders as separating structures. The green lines in Fig. 6 illustrate the additional manual annotation of the separating structures (note that the green lines were increased in thickness for the figure to improve visualization). Due to the low number of foreground samples when compared to the Object-Net, the number of foreground samples for the Separator-Net was artificially increased by exploiting rotation-invariance, and adding nine additional rotated versions of the patch, i.e., every 36°.

CNN training

Both CNNs were trained on a balanced training set of 125, 000 image patches per class. Patches in the training sets were sampled at random from the available pool of training images. Training and test sets reflect approximately the same distribution of samples over images. The size of the minibatches in the stochastic gradient descent optimization scheme was set to 200 samples and the networks were trained until the stopping criterion was met: no further improvement of the error rate on a held-out validation set over 20 epochs. We set the initial learning rate to η₀ = 0.0025, with a linear decay saturating at 0.2η₀ after 100 epochs. For all layers, a weight decay was chosen to be 0.005 and the dropout rate was set to 0.5. An adaptive momentum term was used, starting at 0.8 and increasing to 0.99 after 50 epochs, such that with progressing training the updates are influenced by a larger number of samples than at the beginning.

Figure 7 shows the classification error rate as a function of the training duration in epochs. Each class was represented by 5,000 samples in the validation set for the Object-Net, and 10,000 for the Separator-Net, respectively. The training error was estimated on a fixed subset of the training data (20,000 samples), to get an intuition when overfitting starts. The Object-Net achieved the best performance after 43 epochs, with a minimum training error of 4.75% and a minimum validation error of 4.92%. Training of the Separator-Net continued until the lowest training error of 2.31% and validation error of 6.24% was reached after 119 epochs. The trained networks were evaluated on a representative test set of 20,000 samples, which shares the same class distribution, but does not overlap with training and validation set. On this test set, the best Object-Net achieved an error rate of 4.71%, whereas the Separator-Net achieved 5.58%. The CNN models were implemented in Pylearn2 (Goodfellow et al., 2013), a machine learning library built on top of Theano (Bergstra et al., 2010; Bastien et al., 2012).

Influence of the Separator-Net

Compared to predictions arising from the Object-Net alone, the segmentation performance improved with separator refinement. This procedure further decreases the Hausdorff measure, since the TV segmentation can better utilize predictions in narrow regions between borders formed by epithelial nuclei. Figure 8 shows a qualitative comparison of segmentation results with and without the Separator-Net refinements.

Malignant cases are harder to segment due to their irregular shape and pathological variations in the tissue. In general, the separator refinement works as expected and allows a better separation of adjacent glands than with Object-Net predictions alone. Border regions are more pronounced after the refinement and allow the TV segmentation to better adapt to the true gland borders. However, in cases where two glands are located very closely and there is no significant visual cue for a border or a gland-separating region, the separator does not have any negative influence on the final segmentation result. This is illustrated in Fig. 9.

As indicated by the precision measures in Table 1, including the separators sometimes leads to an over-segmentation of the image and causes multiple detections on a single object, see Fig. 10. Over-segmentation increases the number of false positives, and at the same time may also decrease the number of true positives, when the overlap between segmentation and ground truth drops below 50%. A possible reason may be that the Separator-Net predicts high probabilities for interiors of glands that show highly irregular shape.

Despite having a negative effect on precision, the overall detection performance in terms of F1-score increases when the Separator-Net predictions are included. Employing the TV segmentation on the Object-Net predictions alone produces fewer, but more extensive segmentation objects. Therefore, the number of false positives is quite low (high precision), while the number of false negatives rises (lower recall). From the object-level perspective, segmentation accuracy and shape similarity measures benefit most from the separator refinements. These findings suggest that—when compared to using only predictions from the Object-Net—applying the refinements usually leads to better overall performance.