A computational framework for colour metrics and colour space transforms

An object-oriented computational framework for the transformation of colour data and colour metric tensors is presented. The main idea of the design is to represent the transforms between spaces as compositions of objects from a class hierarchy providing the methods for both the transforms themselves and the corresponding Jacobian matrices. In this way, new colour spaces can be implemented on the fly by transforming from any existing colour space, and colour data in various formats as well as colour metric tensors and colour difference data can easily be transformed between the colour spaces. This reduces what normally requires several days of coding to a few lines of code without introducing a significant computational overhead. The framework is implemented in the Python programming language. Abstract An object-oriented computational framework for the transformation of colour data and colour metric tensors is presented. The main idea of the design is to represent the transforms between spaces as compositions of objects from a class hierarchy providing the methods for both the transforms themselves and the corresponding Jacobian matrices. In this way, new colour spaces can be implemented on the ﬂy by transforming from any existing colour space, and colour data in various formats as well as colour metric tensors and colour difference data can easily be transformed between the colour spaces. This reduces what normally requires several days of coding to a few lines of code without introducing a signiﬁcant computational overhead. The framework is implemented in the Python programming language.


Introduction
Colour data such as measured colours, specified colours or pixels of colour images are most commonly described as sets of points in a three-dimensional space -a so-called colour space. Many different colour spaces are currently in use in various settings. For many applications, selecting the best colour space for processing the data can be crucial [1]. Converting between all the different colour spaces can be challenging. Different conventions for scaling and normalisation is used, and many of the colour spaces commonly in use are inaccurately defined. The complexity of conversion is particularly present for computations involving colour metric data, which, by nature, is tensorial [2], giving rise to the need for not only the direct transformations, but also the corresponding Jacobian matrices -a tedious and error-prone process [3]. So far, no common framework for such transformations of colour data and metrics including the automated computation of Jacobian matrices has been constructed.
From other fields of computational science, it is well established that objectoriented frameworks can be useful for simplifying such matters [4]. With the advent of modern high-level interpreted languages, the computational overhead is not nearly as high as before, and the ease of use has increased significantly [5]. Thus, in order to simplify the matters for colour science and engineering, an object-oriented framework for colour space construction, and conversion of colour data and colour metric tensor data is designed. The framework is currently limited to three-dimensional colour spaces.
Following the background material on the principles of transforming colour data and related tensorial data in the following section, the principles and ideas underlying the framework are presented. To demonstrate to which degree the framework simplifies the implementation of colour data and metric transformations, an implementation of the framework using the high-level programming language Python [6] is applied to some standard example problems.

Transformation of colour data
Transformations between different colour spaces can in general take the shape of a function,x =x(x), where x = (x 1 , x 2 , x 3 ) T represents a colour, i.e., a point in a colour space. Fortunately, most common colour space conversions are made up of a small set of relatively simple mathematical operations. The linear transformation is a very common ingredient in the transforms. Some colour spaces, such as, e.g., the CIECAT02 colour adaptation space [7], are even defined simply by a linear transformation from some other colour space: Where A is a 3 × 3 constant matrix. Combined with the so-called gamma correction, which is applied channel-wise, most RGB type colour spaces, and also, e.g., the IPT [8] colour space can be construced where γ > 0 is the constant exponent. For many perceptual colour spaces such as CIELAB, both Cartesian and cylindrical coordinates are commonly used for describing the chromatic plane. The transformation from Cartesian to polar is

Computer Science
with the corresponding inverse transform Chromaticities and luminances are often represented in projective spaces such as xyY,x 1 = Colour spaces used for colour metrics such as ∆E E [9] and the various DIN99 metrics [10] often include a logarithmic compression of some or all of the channels such as lightness and chroma (radius in polar coordinates): where a i and b i are the parameters of the transform. Recently, the Poincaré disk representation of the hyperbolic plane has been used for representing the chromatic plane [11,12]. The chroma-preserving mapping to the Poincaré disk can be written as a mapping of the radius in polar coordinates as where R > 0 is the radius of curvature. Besides these more generic transformations, various non-linear transformation functions specific to individual colour spaces are used in such cases as sRGB, CIELAB, CIELUV, the underlying colour space of the CIEDE2000 metric, etc.

Transformation of tensorial data
Most colour metrics can be represented in the form of a line element, or a differential quadratic form [13,Chapter 8.4], as Here, G is the metric tensor -a function of the coordinates. For metrics defined as Euclidean distances in a given colour space, the metric tensor is the identity tensor, I , in the given space. Some colour metrics, like, e.g., CIEDE2000, cannot be written in this form, but can be linearised -or Riemannised -to a good approximation [3]. Under a coordinate transformation,x =x(x), this metric transforms according to where ∂x/∂x is the Jacobian matrix of the coordinate transform with componentns ∂x i /∂x j . In other words, the metric tensor transforms according tō Under composition of several coordinate transformations,x =x(x) =x(x(x)), the process is nested, which can also be seen directly from the chain rule for the Jacobian matrices, All the points with unit distance from a given central point -a unit ballconstitute an ellipsoid ∆x T G ∆x = 1.
The cross section of this ellipsoid with a principal plane in a given coordinate is obtained by setting the corresponding ∆x i = 0, reducing the ellipsoid to an ellipsis [3], with angle θ and semi-axes a and b given by tan(2θ) = 2g 12

System architecture
Since the computation of generic colour space transforms and, in partiular the composition of their Jacobian matrices can be a tedious and error-prone process (see, e.g., [3]), an object-oriented framework for transforming colour and metric data between colour spaces has been implemented as a Python package colour. The package consists of six partially interdependent modules space, data, metric, tensor, statistics and misc. The relationship between the modules is shown in Figure 1. In the figure, the arrows indicate dependencies between the modules in the form of Python imports. Each of the modules contain functions, classes and predefined objects with the purpose of simplifying the implementation of new colour spaces and metrics.

Representing colour spaces
The core functionality of the colour space and colour metric transforms is found in the space module. The basic idea in designing the object oriented framework is to realise a colour space as an object, and to facilitate the construction of new such objects by providing classes for transforming new colour spaces from already existing ones. The class hierarchy which constistutes the core of the of the module, is shown in Figure 2. All boxes represent classes, and the arrows denote class inheritance. Italicized method names indicate methods that should be overridden in a subclass. Details about attributes and auxiliary methods etc. have been left out for readability.
All colour space objects must derive from the abstract Space class, and as such implement the methods to_XYZ and from_XYZ for converting colour data between the XYZ colour space and the colour space represented by the object, and the methods jacobian_XYZ and inv_jacobian_XYZ for computing the corresponding Jacobian matrix and its inverse. The two latter methods are implemented in Space as inverses of each other, so the subclasses only need to implement one of them -the other one can be inherited. The colour data is represented as N × 3 NumPy [14] ndarrays, and the Jacobian matrices as N × 3 × 3 ndarrays.
All transformations between colour spaces must go through XYZ, which thus serves a special role, and has a separate class of its own. Here, the transformations are simply the identity transform, and the Jacobian matrices are identity matrices. All other colour spaces will be built by transforming colour data from an already existing colour space, starting from XYZ. To facilitate making the specific transforms, an abstract class Transform is provided. During instantiation of a transformed colour space object, the base space of the transformation has to be set. The virtual methods to_base and from_base for converting colour data to and from the base base, and jacobian_base and inv_jacobian_base for computing the Jacobian matrix and its inverse between the current space and the base space must be provided in the derived classes. The methods to_XYZ, from_XYZ, jacobian_XYZ, and inv_jacobian_XYZ are implemented in the base class Transform using the transformation between the current space and the base space (provided by derived classes) and the corresponding transformations in the base class, see Equation (12). Hence, there is no need to reimplement these in the derived classes. Finally, the concrete colour space tranforms are implemented as classes TransformXXX derived from Transform. They must all implement the methods to_base, from_base and either jacobian_base or inv_jacobian_base. The remaining methods will be inferred by inheritance. In some cases, though, it is more efficient to provide more methods in order to reduce the computational cost. For example, in TransformLinear, both methods jacobian_base and inv_jacobian_base are provided in order to avoid inverting every single Jacobian matrix for large data sets.

Representing colour and metric data
The colour space objects constructed by the method described above, will convert colour data represented as N × 3 ndarrays (N colour data points). For reallife applications, colours some times come as single data points (3-vectors), some times as lists of colour data (N ×3 matrices), and some times as images (M ×N ×3 arrays). In order for the user not having to deal with converting back and forth between these formats, as well as remembering in which colour space all the data is given, a separate class Data for storing colour data has been implemented as part of the data module, cf. Figure 3. Again, the boxes denote classes. In this module, there are no inheritance relationship between the classes, but they are related by the TensorData class having an attribute of the Data type.
A colour Data object can be instantiated with single colour data, lists of colour data, or colour images in any implemented colour space. The Data object takes the colour space (object) of the data as an argument, and keeps a dictionary of the colour spaces in which the data in question has been computed. When the colour data in a given space is requested (using the get method), it first checks the dictionary whether it has already been computed. If not, it is computed, stored in the dictionary and returned. All the actual computations are taken care of by the hierarcy of colour space objects representing the transforms necessary for building the colour space.
A similar approach is taken for colour metric data in the form of colour tensors, see Figure 3. In this case, both the locations of the colour metrics (as colour Data), and the metrics themselves are represented in the class. Like for the colour data, a dictionary of the computed tensors is maintained. For the conversion between the different colour spaces, the Jacobian matrices are applied according to Equation (10). The nested tensor transforms, Equation (12), are implicitly taken care of by the colour space class hierarchy without the user having to interfere.

Colour metrics, tensors and statistics
The four remaining modules, metric, tensor, statistics, and misc contain separate functions (not part of the class hierarchy) for computing various properties of colour data, colour transforms, and sets of these. The metric module has functions for computing the most common colour metrics, such as the standard CIE ∆E ab and ∆E uv metrics, CIEDE2000 [15], the different versions of the DIN99 metric as described by Cui [10], the log-compressed OSA-UCS metric ∆E E by Oleari et al. [9], as well as a general Euclidean distance and the Poincare disk metric developed in Reference [12] in any colour space. All these functions take two colour data objects of the same size as arguments, and return an Nvector of colour differences.
The tensor module has functions for computing the metric tensors corresponding to the metrics in the metric package. The functions take one colour Data object as argument, and returns the corresponding TensorData object. The statistics module contains functions for calculating various statistics of colour metric data, such as the STRESS measure [16] and Pant's R-values [3,17].
The misc module contains miscellaneous supporting functions.

Computational complexity
The framework has been implemented using NumPy [14] ndarrays, and all operations for colour and metric conversion are vectorised. Thus, all the real computations take place using the highly optimised underlying libraries for matrices, such as LAPACK [18] etc. No loops over individual colour data are implemented in the high-level language. Thus, there are only two sources of computational overhead by using the framework.
First, there are the function calls associated with computing the transformations between a given colour space and its bases all the way back to the CIEXYZ colour space. These function calls will only take place once per transformation call. This can be a significant overhead when the data set consists of only one or very few data points. But then the computation is very quick anyway. For bigger colour data sets, such as images, this will represent only very few function calls (given by the number of steps in the transformation from CIEXYZ to the given space), and thus be negligible in comparison with the real computation, which takes place at the highly optimised low-level code.
Secondly, all colour and metric conversions go through the CIEXYZ colour space. When converting between two colour spaces based on a common basis other than CIEXYZ, an unneccesary conversion back and forth between this common basis and CIEXYZ will inevitably take place. It would, in principle, be possible to eliminate this by advanced optimisation techniques, but since the computations are already fast (fractions of a second even for quite large images), and the goal of the framework has been ease of implementation rather than computational efficiency, this has not been prioritized.
Not all of the operations in the statistics module are vectorised, althoug this would in principle also be possible. The reason for this is that they are mainly meant for colour research applications, and as such, they are not expected to be used in production. For the relevant use in research, data collection etc. will be much more time consuming than the actual computations, so computational efficiency has not been emphasized in this part of the framework.

Example application
In order to demonstrate the power of the proposed approach, a simple demo application is shown in Figure 4. In this short code (less than a page), (i) a new colour space is implemented, (ii) individual colours, lists of colours and a colour image is converted to the new colour space, (iii) the tensorial data corresponding to the MacAdam ellipses is converted by the help of the Jacobian matrices of the transformation to the new space, and (iv) the colour difference between two images is computed as a Eucildean distance in the newly constructed colour space. These operations would normally require days of programming, but with the use of the proposed framework it is all achieved by a few lines of code.
In lines 3-6, the library is imported. In a real application, one would normally only import the colour package (import colour), and refer to the elements as, e.g., colour.space.TransformLinear etc., but here the specific classes, objects and functions needed are imported specifically, simply in order to reduce the size and improve the readability of the remaining code. The transformation to the IPT colour space [8] is composed of a linear transform from XYZ followed by a gamma correction, followed by final linear transform. The code for constructing this colour space is given in lines 9-16 of  Figure 5A resulting from this (lines 27-29) are shown in Figure 6.
In order to demonstrate also the transformation of tensorial colour data, the code in lines 32-37 loads, transforms, and plots the MacAdam ellipses [19] in the PT-plane of the IPT space. The latter includes the tedious process of computing the transformation of the corresponding metric tensors according to Equation (10). The resulting plot is shown in Figure 7.
Similarly, the colour.metric module can compute colour differences of colour data in any format, including images. For example, the code in lines 41-43 of Figure 4 computes the difference maps of the two images shown in Figure 5 for ∆E ab , ∆E 00 and the Euclidean distance in the newly implemented IPT colour space. The results are shown in Figure 8.
Please note that the entire code used to generate Figures 5-8 is shown in Figure 4.

Conclusion
An object-oriented computational framework for colour metrics and colour has been designed and implemented in Python. The framework strongly simplifies the implementation of new colour spaces for transfroming colour data and as well as tensorial colour metric data between the various colour spaces without compromising too much on the computational complexity. The code is freely available at GitHub 1 . Future extensions could include ICC support, computation geodesics based on the colour metrics [20], computation and representation of colour gamuts [21], as well as gamut mapping algorithms [22] in any colour space, and under any colour metric.