A novel hyperspectral compressive sensing framework of plant leaves based on multiple arbitrary-shape regions of interest

HCSMAROI

For most hyperspectral applications, plant leaf HSIs can be divided into multiple arbitrary shape region of interests and a background region in the spatial domain. This framework has the novelty that it can compress different arbitrary shape ROIs independently with different target rates and discard the background region. It can not only compression plant leaf HSIs, but also have the potential to be applied in other hyperspectral images with MAROIs. Fig. 1 shows the overview of HCSMAROI.

Different types of HSIs can be effectively divided with appropriate strategies. For plant leaf HSIs, SSDC can be used to obtain the optimal band and then different ROIs can be labeled. For other types of HSIs, such as HSIs of satellite remote sensing, many image terrain classification algorithms can be used to extract the ROIs.

The proposed framework can be summarized as follows:

Firstly, SSDC is used to obtain the key band of plant leaf HSIs; Secondly, the primary mask of the multiple arbitrary-shape ROIs could be extracted, and the final labeled mask of different ROIs can also be obtained in the key band; Thirdly, after discarding the useless background data for saving the memory storage, all arbitrary-shape ROIs of plant HSIs can be divided into blocks in the spatial domain and transformed into one-dimensional data one by one, then a random Gaussian matrix is selected to compress each arbitrary-shape ROI; Fourthly, StOMP is chosen to reconstruct each arbitrary-shape ROI; Finally, the reconstructed plant leaf HSI can be obtained by combining all reconstructed MAROIs.

Extraction the optimal band using SSDC

Gao et al. (2013) had compared the noise estimation performance of six main evaluation methods of homogeneous area (HA), geo-statistical (GS), local means and local standard deviations (LMLSD), SSDC, residual-scaled local standard deviation (RLSD), and homogeneous regions division and spectral de-correlation (HRDSDC), and obtained the conclusion that SSDC was the most reliable evaluation method for hyperspectral noise estimation performance among these six evaluation methods. Therefore, SSDC is chosen to evaluate the image quality of HSIs. After calculating the SSDC values of all bands, the band with the minimal SSDC is regarded as the optimal band.

ROI mask extraction and labeling

After obtaining the optimal band, k-means clustering algorithm is used to extract the mask for the arbitrary-shape ROIs of the HSIs. According to the characteristics of the optimal band, the number of classes, m, needs to be preset. The optimal band can be divided into m classes. But for the background region, the remaining m-1 classes can be combined to obtain the primary binary mask and construct the whole leaf regions.

Each region in the primary binary mask can represent a ROI in the optimal band. Each 8-connected region in the primary binary mask can be labeled with an index value to construct the labeled mask.

Blocking and expansion for arbitrary-shape ROIs

In order to obtain good reconstructed performance, hyperspectral data of ROIs were compressed and reconstructed while those of background were discarded. Because of stronger spectral correlation of HSIs, compared with the spatial correlation, all ROIs were divided into multiple sub blocks only in the spatial domain. In the light of our extensive experiments, spatial block size of 2 × 2 can be used to achieve good reconstructed performance.

Considering the high correlation of neighboring pixels in the spatial domain, each block of the same position of all bands is firstly expanded into one-dimensional data in the order of zigzag band by band, and then combined to construct the whole one-dimensional data. Figure 2 shows the expansion and inverse expansion scheme of a block.

CS and reconstruction for arbitrary-shape ROIs

The measurement matrix $\mathrm{\Phi }\in R^{M\times N}$ is used to sample each one-dimension expansion data of arbitrary-shape ROIs randomly; M is the signal length after sampling, N is the expansion length of a block, M << N; $\mathrm{\Phi }$ is the random Gauss measurement matrix.

The sampling rate can be defined as follow.

(1) $$R=\frac{M}{N}$$

The sampling process is as below.

(2) $$Y=\mathrm{\Phi }X$$where X is a one-dimension expansion data, and Y is the observed data. We use Discrete Cosine Transform (DCT) matrix to construct the sparse basis $\mathrm{\Psi }$, $\mathrm{\Psi }\in R^{N\times N}$, ${\mathrm{\Psi }}^T\times \mathrm{\Psi }=I$, $I$ is the identity matrix, which is an orthonormal matrix. Ψ is given by

(3) $$\mathrm{\Psi }=\frac{1}{\sqrt{N}}\left[\begin{array}{cccc}1\hfill & 1\hfill & ...\hfill & 1\hfill \\ \sqrt{2}\cos \frac{\pi }{2N}\hfill & \sqrt{2}\cos \frac{3\pi }{2N}\hfill & \dots \hfill & \sqrt{2}\cos \frac{\left(2N-1\right)\pi }{2N}\hfill \\ \dots \hfill & \dots \hfill & \dots \hfill & \dots \hfill \\ \sqrt{2}\cos \frac{\left(N-1\right)\pi }{2N}\hfill & \sqrt{2}\cos \frac{3\left(N-1\right)\pi }{2N}\hfill & \dots \hfill & \sqrt{2}\cos \frac{\left(2N-1\right)\left(N-1\right)\pi }{2N}\hfill \end{array}\right]$$

The sensing matrix A is constructed by Ψ and Φ and is shown by Eq. (4). To ensure the convergence of the reconstruction algorithm, the sensing matrix A must satisfy the principle of the restricted isometry property (RIP) (Candès, 2008).

(4) $$A=\mathrm{\Phi }\mathrm{\Psi }$$

Due to the high efficiency and high accuracy of StOMP, it is chosen as the reconstructed algorithm to reconstruct each one-dimension expansion data.

The inverse expansion process (Fig. 2B) is performed to reconstruct a block. After all blocks have been reconstructed, each ROI of plant leaf HSIs can be reconstructed. Finally, the reconstructed plant leaf HSI can be obtained by combining all reconstructed MAROIs according to the labeled mask.

HSIs of soybean leaves and tea leaves

In this study, a hyperspectral imaging system ImSpector V10E (ImSpector V10E, SPECIM, Finland) covering the spectral wavelengths of 400–1,000 nm was used to snap the soybean leaf HSIs. The system includes a CCD camera (DL-604M, Andor, Ireland), an imaging spectrograph, a lens, a light sources provided by 150 W quartz tungsten halogen lamp (2900-ER, Illumination, Santa Monica, CA, USA) and software (Isuzu Optics Corp, Zhubei, Taiwan) for the computer operating the spectral image system. The spectral resolution is 0.65 nm and the resolution of CCD array detector of the camera is 1,344 × 1,024. The system is driven by electronically controlled displacement platform (IRCP0076, Isuzu Optics Corp, Taiwan) to scan the samples line by line.

Two sets of soybean leaf HSIs are used in the experiment. The wavelength range of the first soybean leaf HSIs is from 476 nm to 939 nm, in which there are totally 596 bands. A single pixel is defined by a 16-bit unsigned integer and their resolution is 834 × 1,004, that is to say, the total pixel count of a band is 837,336. The wavelength range of the second soybean leaf HSIs is from 460 to 900 nm, in which there are totally 568 bands, and their resolution is 706 × 887. The bands of 666, 558 and 476 nm of these two sets of soybean leaf HSIs are selected as the red, green and blue channels to construct the false color composite RGB images as shown in Figs. 3A and 3B, respectively.

The tea leaf HSIs in our previous research (Xu et al., 2017) are used in this paper. A single pixel is defined by a 12-bit unsigned integer. The resolution of the first tea leaf HSIs is 435 × 642, in which there are totally 381 bands. The wavelength range of the second tea leaf HSIs is from 449 to 915 nm, in which there are totally 369 bands, and their resolution is 294 × 627. The image of 666, 555 and 454 nm of tea leaf HSIs are selected as the red, green and blue channels of the false color composite RGB images as shown in Figs. 4A and 4B.

Mask extraction

Primary mask extraction

Figure 5A indicates the SSDC curve of the first soybean leaf HSIs in which the 702 nm band shows the lowest SSDC value. Figure 5B shows the SSDC curve of the first tea leaf HSIs in which the 690 nm band obtains the lowest SSDC value. Therefore, the 702 nm band and the 690 nm band are chosen as the optimal bands which can be used to extract the mask of multiple arbitrary-shape ROIs of four pieces of soybean leaves and six pieces of tea leaves, respectively. Figures 6 and 7 show the 702 nm band image of soybean leaf HSIs and the 690 nm band image of tea leaf HSIs, respectively.

The k-means clustering algorithm is used to cluster these optimal bands. The number of classifications m is set as four and three for the first soybean leaf HSIs and the first tea leaf HSIs, respectively. Figures 8 and 9 show their primary binary masks by the k-means clustering algorithm.

Labeled mask extraction

The ROI regions of the primary mask are labeled from left to right and from top to bottom. In the labeled mask, the larger ROI label value, the higher its gray value.

Figures 10 and 11 show the labeled masks of multiple arbitrary-shape ROIs of soybean leaves and tea leaves. The total pixel area Area_ROIs of all ROIs in the labeled mask can be obtained as follows.

(7) $$Are{a}_{ROIs}=\sum _{\mathrm{i}=1}^{\mathrm{n}}Are{a}_{ROI(\mathrm{i})}$$where $Are{a}_{BOI(\mathrm{i})}$ is pixel area of the ith ROI in the labeled mask, n is the total number of ROIs.

Reconstruction performance analysis of multiple arbitrary-shape ROIs using the primary mask

In this case, multiple arbitrary-shape ROIs can be compressed and reconstructed using the primary binary mask at the whole target sampling rate. Single spectral compressive sensing (SSCS) and blocking compressive sensing (BCS) are chosen to make the performance comparison. The sampling rates of soybean leaf HSIs are 5%, 10%, 15% and 20%, and the sampling rates of tea leaf HSIs are 3%, 5%, 8%, 12%, 17% and 20%. That is to say, the actual sampling rates of different ROIs are respectively written as follows.

(8) $$Rati{o}_p=\frac{Are{a}_{whole}}{Are{a}_{ROIs}}\times p$$where $Are{a}_{whole}$ is the whole area of all ROIs in the primary mask, $p$ is the sampling rates of the HSIs, and $Rati{o}_p$ is the sampling rates of all ROIs.

Spatial domain reconstruction analysis

Figures 12–15 illustrate the experimental results of soybean leaf HSIs for SSCS, BCS and HCSMAROI at the sampling rates from 5% to 20%. Figures 16–21 illustrate the experimental results of tea leaf HSIs for SSCS, BCS and HCSMAROI at the sampling rates from 3% to 20%. Compared with SSCS, BCS preserves the spatial correlation to achieve better reconstructed image. HCSMAROI allocates all the bit rates to the leaf regions and can achieve significantly better reconstructed subjective quality for leaf regions than that of the other two algorithms at low sampling rates.

The reconstructed PSNRs performances of soybean leaf HSIs as shown in Table 2, the reconstructed PSNRs performances of tea leaf HSIs as shown in Table 3. The reconstructed PSNRs of HCSMAROI and BCS are higher than that of SSCS at different sampling rates. The PSNRs of reconstructed images of different algorithms are all improved with the increasing of the sampling rate. HCSMAROI achieves the highest PSNRs at all sampling rates. When the sampling rate is lower than 10%, the reconstructed PSNR values of HCSMAROI are significantly higher than that of BCS. When the sampling rate is greater than or equal to 20%, the reconstructed PSNR of HCSMAROI is close to that of BCS. It is shown that HCSMAROI can achieve quite good reconstructed PSNR in the spatial domain, especially at low sampling rate.

Table 2:

Comparisons of reconstructed PSNRs/dB for soybean leaf HSIs for different algorithms.

Algorithms		Sampling rates
		5%	10%	15%	20%
The first soybean leaf HSIs	SSCS	21.59 dB	31.70 dB	36.79 dB	38.51 dB
	BCS	33.25 dB	36.70 dB	38.19 dB	38.96 dB
	HCSMAROI	36.33 dB	38.88 dB	39.77 dB	40.24 dB
The second soybean leaf HSIs	SSCS	17.68 dB	30.91 dB	38.84 dB	40.60 dB
	BCS	34.40 dB	38.30 dB	39.78 dB	40.57 dB
	HCSMAROI	37.01 dB	39.80 dB	40.92 dB	41.48 dB

DOI: 10.7717/peerj-cs.802/table-2

Table 3:

Comparisons of reconstructed PSNRs/dB for tea leaf HSIs for different algorithms.

Algorithms		Sampling rates
		3%	5%	8%	12%	17%	20%
The first tea leaf HSIs	SSCS	7.78 dB	10.06 dB	16.35 dB	24.55 dB	33.12 dB	35.03 dB
	BCS	17.83 dB	27.56 dB	32.45 dB	34.24 dB	35.30 dB	35.68 dB
	HCSMAROI	30.74 dB	33.61 dB	35.35 dB	36.39 dB	37.01 dB	37.26 dB
The second tea leaf HSIs	SSCS	8.19 dB	9.86 dB	16.14 dB	25.76 dB	32.66 dB	34.60 dB
	BCS	18.37 dB	27.61 dB	32.53 dB	34.44 dB	35.55 dB	35.95 dB
	HCSMAROI	29.73 dB	33.30 dB	35.30 dB	36.47 dB	37.17 dB	37.43 dB

DOI: 10.7717/peerj-cs.802/table-3

Spectral domain reconstruction results analysis

Spectral analysis of plant HSIs plays an important role in monitoring of plant growth, disease and insect pest detection and the inversion of physiological parameters. Figure 22 shows the average reconstructed spectra of the multiple arbitrary-shape ROIs of SSCS, BCS and HCSMAROI at different sampling rates from 5% to 20% for the first soybean leaf HSIs. Figure 23 shows that the average reconstructed spectra of the multiple arbitrary-shape ROIs of SSCS, BCS and HCSMAROI at the sampling rates from 3% to 20% for the first tea leaf HSIs. Experimental results shows that the reconstructed spectra of BCS are obviously closer to the original one than that of SSCS at different sampling rates. Besides, especially at low sampling rates, the reconstructed spectra of HCSMAROI are closer to the original ones than those of the others.

Table 4 gives RMSE comparisons of spectral indices of TVI and DD of the first soybean leaf HSIs for different algorithms at different sampling rates. For spectral indices of TVI and DD at different sampling rates, RMSE of HCSMAROI is relatively less than that of the other algorithms. At the sampling rate of 10%, HCSMAROI has especially obvious advantage. And RMSE of SSCS shows the largest RMSE values at different sampling rates.

Table 4:

Comparisons of reconstructed RMSE of the spectral index TVI and DD for the first soybean leaf HSIs for different algorithms.

Spectral index	Algorithms	Sampling rates
		5%	10%	15%	20%
TVI	SSCS	0.5369	0.0617	0.0289	0.0290
	BCS	0.0193	0.0135	0.0071	0.0034
	HCSMAROI	0.0151	0.0047	0.0013	0.0000
DD	SSCS	0.6605	0.1126	0.1476	0.1810
	BCS	0.1149	0.1596	0.1173	0.0909
	HCSMAROI	0.1635	0.0980	0.0680	0.0559

DOI: 10.7717/peerj-cs.802/table-4

Table 5 shows RMSE comparisons of spectral indices of TVI and DD of tea leaf HSIs for different algorithms at different sampling rates. HCSMAROI can achieve obviously the best performance at low sampling rates of 3% and 5%.

Table 5:

Comparisons of reconstructed RMSE of the spectral index TVI and DD for the first tea leaf HSIs for different algorithms.

Spectral index	Algorithms	Sampling rates
		3%	5%	8%	12%	17%	20%
TVI	SSCS	0.9755	0.9172	0.6709	0.2066	0.0032	0.0060
	BCS	0.6236	0.1028	0.0096	0.0019	0.0005	0.0012
	HCSMAROI	0.0178	0.0015	0.0015	0.0023	0.0024	0.0021
DD	SSCS	0.9862	0.9646	0.8470	0.5427	0.2374	0.1618
	BCS	0.8329	0.4373	0.1740	0.0970	0.0671	0.0591
	HCSMAROI	0.2264	0.0999	0.0607	0.0452	0.0358	0.0320

DOI: 10.7717/peerj-cs.802/table-5

Reconstruction performance analysis for multiple arbitrary-shape ROIs using the labeled mask

In the “Reconstruction Performance Analysis of Multiple Arbitrary-shape ROIs using the Primary Mask”, the experimental results for multiple arbitrary-shape ROIs using the primary mask show that the proposed HCSMAROI can achieve significantly better reconstructed performance than that of the SSCS and BCS in the spatial and spectral domains. Therefore, HCSMAROI is chosen to perform the following experiments.

In this case, multiple arbitrary-shape ROIs can be compressed and reconstructed respectively using the labeled mask at different target sampling rates. This case can be regarded as a general one. Reconstruction of multiple arbitrary-shape ROIs using the primary mask is special situation of this case. For the first soybean leaf HSIs, the sampling rates of ROI1, ROI2, ROI3, and ROI4 are 5%, 10%, 15% and 20%, respectively. For the first tea leaf HSIs, the sampling rates of ROI1, ROI2, ROI3, ROI4, ROI5 and ROI6 are 3%, 5%, 8%, 12%, 17% and 20%, respectively. These ROIs can be compressed and reconstructed in turn according to their respective target rates. The whole sampling rate can be calculated as follow:

(9) $$R\mathrm{a}ti{o}_{whole}=\frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}Are{a}_{ROI(\mathrm{i})}\times p\left(i\right)}{Are{a}_{whole}}$$where $Are{a}_{BOI\left(\mathrm{i}\right)}$ and $p\left(i\right)$ are the area and sampling rate of the ith ROI respectively, $Are{a}_{whole}$ is the whole area of all ROIs in the primary mask, and $R\mathrm{a}ti{o}_{whole}$ is the whole sampling rate.

Figures 24 and 25 show the reconstructed images of the first soybean leaf HSIs and the first tea leaf HSIs, in which the ROI with higher sampling rate obtains better reconstructed quality, respectively. For soybean leaf HSIs, Table 6 also shows the same result that average reconstructed PSNR of ROI1 is the worst and that of ROI4 is the best. For tea leaf HSIs, Table 7 shows that average PSNR of reconstructed ROI1 is the worst and that of ROI6 is the best. These results shows that when the sampling rate of a certain ROI is greater than or equal to 10%, it can achieve quite good reconstructed quality in the spatial domain.

Figure 26 illustrates the reconstructed spectra of soybean leaf HSIs for different ROIs. For those ROIs whose sampling rates are larger than or equal to 10%, the reconstructed spectra can be quite closer to the original spectra. Figure 27 illustrates the reconstructed spectra of tea leaf HSIs for different ROIs. For those ROIs whose sampling rates are larger than or equal to 5%, the reconstructed spectra can be closer to the original spectra than the others. The higher the sampling rate of a ROI, the closer to the original spectra it is.

Table 6 and Table 7 show the RMSE of reconstructed spectral indices TVI and DD for the first soybean leaf HSIs and the first tea leaf HSIs, respectively. Similarly, the higher the sampling rate of a ROI, the smaller the reconstructed RMSE of reconstructed spectral indices TVI and DD. When the sampling rate of a certain ROI is greater than or equal to 15% for soybean leaf HSIs or 10% for tea leaf HSIs, the ROI can also achieve quite good reconstructed quality in the spectral domain.

Therefore, the proposed HCSMAROI can provide a flexible way to effectively compress and reconstruct different arbitrary-shape ROIs with different strategy meanwhile good reconstructed performance can be obtained in the spatial and spectral domains.