IRimage: open source software for processing images from infrared thermal cameras

Relationship between temperature and infrared radiation

Thermal cameras are based on the detection of infrared radiation emitted from objects by means of an array of sensors. Each of these sensors generates a digital signal (S), which is a function of radiance (L). Radiance is the radiant flux (i.e., amount of energy emitted, reflected, transmitted or received per unit time, usually measured in Watts, W) per unit surface and solid angle (in W sr⁻¹ m⁻²). The relationship between the signal (S) resulting from the voltage/current generated by the sensor and the associated electronics (usually quantified as Digital Numbers; DN) and L is usually linear, and gain (G) and offset (O) factors can be calibrated: (1)${\displaystyle S=G\cdot L+O.}$

Measuring radiance can be used to estimate temperature because the total amount of energy emitted by an object is a function of absolute temperature to the fourth power (according to the Stefan–Boltzmann law). The emission is, however, not equal at different wavelengths (even for a perfect emitter, i.e., a black body): according to Wien’s displacement law, the wavelength corresponding to the peak of emission also depends on temperature. For instance, the peak emission of the sun is around 500nm (in the visible portion of the spectrum), while that of a body at 25 °C is around 10µm (in the far infrared). Since detectors are only sensitive to part of the spectrum, it is necessary to take into account only the spectral radiance (L_λ) for a given wavelength (according to the Lambert’s cosine law and the Planck’s law) which is equal to: (2)${\displaystyle L_{\lambda }=\varepsilon \cdot \frac{2h{c}^2}{{\lambda }^5}\cdot \frac{1}{e^{\frac{hc}{\lambda kT}}-1},}$

where ɛ is the emissivity of the surface, h is the Planck constant, c is the speed of light in the medium, λ is the wavelength, k is the Boltzmann constant, and T is the absolute temperature of that surface (in kelvins). This equation needs to be integrated over the spectral band corresponding to the detector sensitivity (short-wavelength: 1.4–3 µm, mid-wavelength: 3–8 µm, or long-wavelength: 8–15 µm, depending on the type of sensor) or, for simplicity, be multiplied by the spectral sensitivity range (Gaussorgues, 1994). For a given camera (i.e., combination of electronics, sensors and lenses) this equation can be simplified as: (3)${\displaystyle L_{\lambda }=\varepsilon \cdot \frac{1}{R\cdot \left(e^{\frac{B}{T}}-1\right)},}$

where B and R are camera calibration parameters (together with G and O). In some cases, the constant 1 is also stored as calibration parameter F.

By combining Eqs. (1) and (3), it is possible to obtain an equation that represents the relationship between S and T for a given sensor, and can be used for its calibration: (4)${\displaystyle S=G\cdot \varepsilon \cdot \frac{1}{R\cdot \left(e^{\frac{B}{T}}-1\right)}+O.}$

Sources of radiation

The radiation received by the camera sensor is not equal to the radiation emitted by the object(s) in its field of view. Depending on the emissivity of the object’s surface, radiation reflected by the object’s surface can contribute significantly to the radiation received by the sensor. Furthermore, this radiation is then attenuated by the atmosphere (mainly by water molecules) even at short distances (Minkina & Klecha, 2016). Taking this into account, the signal detected by the sensor (S) can be considered to be composed of three terms: (5)${\displaystyle S=\tau \cdot S_{obj}+\tau \cdot S_{refl}+S_{atm},}$

where the first term is the equivalent digital signal originating from the target object (S_obj), attenuated by the atmosphere, which is represented by the atmospheric transmissivity factor tau (τ); the second term is the equivalent digital signal from the reflected radiation originating from the target object’s surroundings (S_refl), also attenuated by the atmosphere; and the last term is the equivalent digital signal originated from the atmosphere itself in the path between the object and the sensor (S_atm).

Estimation of atmospheric transmissivity

There are many different models available to estimate atmospheric transmissivity. For short distances, simple models that take into account the amount of water in the air can provide adequate estimates. For long distances (e.g., for infrared cameras used in satellites), more sophisticated models which take into account not only water but also carbon dioxide, ozone, and other molecules, and other atmospheric factors such as scattering are used (Gaussorgues, 1994; Zhang et al., 2016). In this article, the method used in FLIR Systems’ cameras was adopted (FLIR Systems, 2001), which estimates atmospheric transmissivity(τ) based on air water content (H), calculated from air temperature (t) and relative humidity (RH), and the distance between the object and the sensor (d): (6)${\displaystyle H=RH\cdot e^{\left(1.5587+6.939\cdot 10^{-2}\cdot t-2.7816\cdot 10^{-4}\cdot t^2+6.8455\cdot 10^{-7}\cdot t^3\right)},}$ (7)${\displaystyle \tau =X\cdot e^{\left[-\sqrt{d}\cdot \left({\alpha }_1+{\beta }_1\cdot \sqrt{H}\right)\right]}+\left(1-X\right)\cdot e^{\left[-\sqrt{d}\cdot \left({\alpha }_2+{\beta }_2\cdot \sqrt{H}\right)\right]}.}$

Estimation of digital signal values for different radiation sources

Assuming all temperatures and emissivities are known, the signal values originating from the different radiation sources, which contribute to the total signal produced by the sensor, can be estimated using Eq. (4). For the target object, the signal (S_obj) can be calculated based on the object temperature (T_obj) and its emissivity (ɛ): (8)${\displaystyle S_{obj}=G\cdot \varepsilon \cdot \frac{1}{R\cdot \left(e^{\frac{B}{T_{obj}}}-1\right)}+O.}$

The signal originated from the atmosphere between the object and the sensor (S_atm) can be calculated based on air temperature (T_atm) and its emissivity, which is equal to 1 − τ: (9)${\displaystyle S_{atm}=G\cdot \left(1-\tau \right)\cdot \frac{1}{R\cdot \left(e^{\frac{B}{T_{atm}}}-1\right)}+O.}$

For estimating the signal from radiation reflected by the target object (S_refl), one must take into account the reflectivity of the object, which is equal to 1 − ɛ. Also, it should be necessary to know the temperature of the surrounding objects (T_refl) and their emissivity (ɛ_refl): (10)${\displaystyle S_{refl}=G\cdot \left(1-\varepsilon \right)\cdot \left({\varepsilon }_{refl}\right)\cdot \frac{1}{R\cdot \left(e^{\frac{B}{T_{refl}}}-1\right)}+O.}$

Since in most cases it would be difficult to determine the temperature and emissivity of all the surrounding objects, the usual procedure is to estimate an aparent reflected temperature (T_app.refl), by measuring the apparent temperature of a reflective material with ɛ ≈ 0 (usually aluminium foil). Using this procedure, Eq. (10) would be replaced by: (11)${\displaystyle S_{refl}=G\cdot \left(1-\varepsilon \right)\cdot \frac{1}{R\cdot \left(e^{\frac{B}{T_{app.refl}}}-1\right)}+O.}$

Object temperature calculation

In order to calculate object temperature (T_obj), it is necessary to first obtain the signal originating from the object by solving Eq. (5) by S_obj, and usign the total signal S and the results from Eqs. 7, 9, and 11: (12)${\displaystyle S_{obj}=\frac{S}{\tau }-S_{refl}-\frac{S_{atm}}{\tau }.}$

Finally, by solving Eq. (8) by T_obj and using the result of Eq. (12) and the sensor’s gain, offset and calibration parameters (G, O, B, and R), it is possible to calculate the object temperature as follows: (13)${\displaystyle T_{obj}=\frac{B}{\text{log}\left(\frac{G\cdot \varepsilon }{R\cdot \left(S_{obj}-O\right)}+1\right)}.}$

Extraction of parameters from JPEG files.

The temperature calculation method relies on having access to the raw sensor data obtained by the camera. In the case of FLIR cameras, this data is stored in the radiometric JPG files (the standard file format for these cameras) as metadata tags in “EXIF” format. This metadata also includes camera-specific and user-set parameters which are also used to calculate temperature. All the parameters that are extracted from the JPG files, and the corresponding variables used in IRimage are detailed in Table 1. First, IRimage uses the ExifTool software (Harvey, 2003) to processes all images in JPG format within the user-selected folder in order to extract the raw sensor data, which is stored as a PNG image. Next, all camera-specific, atmospheric and user-set parameters are extracted.

Calculation of derived variables

The next step is the calculation of variables derived from these parameters (detailed in Table 2), including the calculation of atmospheric transmissivity (using Eqs. (6)–(7)) and the estimated signal from reflected objects and the atmosphere (using Eqs. (9)–(11)). The byte order (endianness) of the raw image is determined from the image type (PNG or TIFF). This works in almost all cases, but it has been found that this rule does not hold for some (at least three) camera models. In those cases, an exception to this rule is included in the code.

Depending on the option selected by the user, both the extraction of parameters and the calculation of these variables are either performed for each file or only for the first file in the folder (when the user wants to apply the same set of parameters to all files). In the latter case, the user can also modify the “user-selected” parameters.

Temperature calculation

The variables used for the final temperature calculation are detailed in Table 3. The PNG image containing the raw sensor data is opened, and each pixel containing the digital signal from the sensor is processed sequentially. First, the object signal is estimated using Eq. (12), and then the temperature value is calculated using Eq. (13).

Processing thermal images

IRimage was implemented and tested using FLIR brand cameras (FLIR Systems Inc., USA), which is one of the most widely used brands in research (Harrap et al., 2018), and is able to process FLIR’s radiometric JPG thermal image format.

The “Process” function of IRimage processes complete folders of thermal images, since this is usually the case in research uses. The radiometric JPG image format includes user-set parameters (i.e., emissivity, air temperature and humidity, reflected temperature, and object distance) within individual images, but it is also possible to use different parameter values to calculate temperature from raw data. IRimage can process the images using either of three options (Fig. 1): (1) process each file using the stored parameters (e.g., when the user has manually set these parameters in the camera according to specific conditions for each image); (2) use a set of global parameters for all images, which is useful when all images were captured under the same conditions, or if parameters need to be modified globally (in this case a dialog box is shown where parameters can be set, Fig. 1); or (3) use parameters stored in a text file, in which specific parameters can be defined for each image.

IRimage processes all images in JPG format within the user-selected folder. Raw data containing the digital signal from the sensor is extracted and temperature is estimated for each pixel using the algorithm detailed in the “Theoretical background for temperature calculations” section. After processing, three images are stored for each input file, corresponding to the raw data, the estimated temperature, and a false-color image. The estimated temperature pixel values are also stored as text, in a .csv (comma-separated values) file that can be opened in a spreadsheet or statistical software. Also, irrespective of the processing option selected, the parameters used are stored in a .csv text file which can be later used to reproduce the same results (with the “Use parameters from file” option, Fig. 1). Each of these output file types are stored in different subfolders within the user-selected folder.

Measuring the temperature of objects

A third function is included to perform reproducible measurements of the temperature of objects in the images. With the “Measure” function (Fig. 1), the user can select up to 255 different objects (rectangular, oval, or free-form areas, lines or points) and obtain temperature measurements (mean, minimum, maximum and standard deviation) for each of them, for each of the images in a folder. The selected objects are stored in a “mask” image, which can be later used to reproduce the same measurements. A mask image can also be modified or created using other methods and used in IRimage.

Creating customized false-color images and videos

The “Color” function allows the user to produce false-color images representing the temperature values, which are useful for visualizing and reporting (Fig. 1).

The default palette used in IRimage is “mpl-inferno”, one of the default palettes in the popular data visualization Python package Matplotlib (Van der Walt & Smith, 2015), which is also included in FIJI. It is a typical “thermal” palette that represents colder values in dark blue, and transitions through red and yellow for warmer values (emulating the color of hot liquid metal, or the radiation emitted by a black body). It was selected because it is both perceptually uniform and suitable for color-blind viewers. Alternatively, the classic greyscale palette, representing colder values as black and warmer values as white, is supported as well. Another important aspect is selecting the appropriate temperature scale, in order to efficiently represent the temperature values in the images using the full color palette. IRimage allows the user to select the contrast level, by automatically adjusting the minimum and maximum displayed values through a “histogram stretching” algorithm (Fisher et al., 2003). It operates by setting an amount of pixels with extreme values (the “tails” of the histogram) that are excluded (0, 0.3 and 3% for the low, normal and high contrast options, respectively). There are also two ways to calculate the temperature range for a set of images: either use the same scale for all of them (based on the range of temperatures in the full image set, which allows for a better comparison between images), or adjust the scale to the temperature range in each image (which allows for a better visualization of temperature differences within each image). It is also possible to add a scale bar to the images, showing the temperature scale and the color palette, with two different sizes for the scale bar and the font. Lastly, the user can choose whether to produce a video file for the complete set of images as a sequence.

Testing the temperature estimations against other software

The Test function provides a way of testing the algorithm, by comparing the results obtained with the Process function of IRimage against data exported using another software (e.g., that provided by the camera manufacturer). Since IRimage is an open-source software and therefore its modification and customization is possible (and encouraged), this function can be used to check if the calculations have not been altered by any change in the code made by the user (for that purpose, a test image is included with IRimage, along with the temperature data exported using FLIR Tools). Also, it can be used to check whether IRimage functions correctly for a given camera when it is used for the first time. After the function is run, a scatter plot is drawn for each image for which there is reference data available, comparing temperature values of all the pixels, and a text file indicating the mean and maximum temperature differences.