Using cantor sets for error detection

Introduction

Computers and communication systems invariably have to deal with the ill effects of noise, which can lead to errors in computation and information processing. In a landmark paper published in 1950, Richard Hamming addressed this problem by introducing mathematical techniques for error detection and correction (Hamming, 1950). Since then, coding theory has burgeoned to be a field of its own right, boasting of important research and developments in the art and science of error detection and correction (Lin & Costello, 1983). Error detection/correction techniques have therefore been a fundamental part of most computing systems and communication networks, typically applied on the input data after data compression (lossy/lossless) and encryption (Bose, 2008).

Shannon’s separation theorem (Shannon, 1959) states that under certain assumptions, data compression (source coding) and error protection (channel coding) can be performed separately and independently (sequentially) while still maintaining optimality. However, in several practical scenarios, these assumptions are not met and hence there is a need for joint source channel coding. In image/video browsing and web-based applications, it is highly desirable to have the property of progressive transmission¹  (Said & Pearlman, 1996). However, even the slightest error can wreak havoc on a progressive bitstream. Hence, there is a need for incorporating error detection when decoding a compressed progressive bitstream.

Lossless data compression refers to the removal of redundancy in the data—a pre-requisite step in most storage and communication systems before the application of an encryption algorithm and error control coding (for error detection/correction). Consider an i.i.d. binary message source S which emits ‘0’ with probability p (0 < p < 1). A binary message M of length N from such a source needs to be losslessly compressed. Shannon (Shannon, 1948) showed that such a message can at best be losslessly compressed to ≥H(S)⋅N bits on an average, where H(S) is the Shannon’s entropy of the source. For an individual binary message M from such an i.i.d. source, we can compute H(⋅) as follows: (1)${\displaystyle H\left(M\right)=-p{log}_2\left(p\right)-\left(1-p\right){log}_2\left(1-p\right)bits∕symbol,}$ where $p=\frac{NumberofzerosinM}{N}$. There are several lossless compression algorithms in literature - Shannon-Fano coding, Huffman coding, Arithmetic coding, Lempel–Ziv coding and others (Cover & Thomas, 2006; Sayood, 2000). Among these, Arithmetic coding (Rissanen & Langdon, 1979) achieves the Shannon entropy limit for increasing message lengths. Arithmetic coding is used extensively in several practical applications owing to its speed, efficiency and progressive bitstream property. In fact, it is used in JPEG2000 (Taubman & Marcellin, 2002), the international standard for still image compression, replacing the popular Huffman coding which was used in the earlier JPEG (Wallace, 1992) standard.

In 2009 (Nagaraj, Vaidya & Bhat, 2009), it was shown that Arithmetic coding is closely related to a 1D non-linear chaotic dynamical system known as Generalized Luröth Series (GLS). Specifically, it was shown that lossless compression or encoding of the binary message M can be performed as follows. First, the message M is treated as a symbolic sequence on an appropriately chosen chaotic GLS. The initial value on the GLS corresponding to this symbolic sequence is computed by iterating backwards on the map. This initial value (written in binary) serves as the compressed file. For decompression, we start with this initial value (the compressed file) and iterate forwards on the (same) GLS, and record the symbolic sequence. This symbolic sequence is the decoded M. Such a simple lossless compression algorithm (known as GLS-coding) was proved to achieve Shannon’s entropy limit (Nagaraj, Vaidya & Bhat, 2009). Arithmetic coding turns out to be a special case of GLS-coding (Nagaraj, Vaidya & Bhat, 2009).

Unfortunately, it turns out that GLS-coding (hence also arithmetic coding), having the progressive bitstream property, is very sensitive to errors. Even a single bit error in the compressed file can lead to catastrophic decoding errors. This has been well documented in the data compression literature (Boyd et al., 1997) and researchers have since been trying to enhance Arithmetic coding with error detection and correction properties (Anand, Ramchandran & Kozintsev, 2001; Pettijohn, Hoffman & Sayood, 2001). In this work, our aim is to incorporate error detection into GLS-coding by using Cantor set while not sacrificing much on the lossless compression ratio performance of GLS-coding. As we shall demonstrate, Cantor set has desirable properties that enable efficient error detection with GLS-decoding.

The paper is organized as follows. In ‘GLS-Coding and Decoding’, we briefly describe GLS-coding of a binary message and its sensitivity to errors. In ‘Error Detection Using Cantor Set’, we explore Cantor set and describe its wonderful properties that enable error detection. Particularly, it is shown that Repetition codes belong to a Cantor set. Inspired by this, we incorporate error detection into GLS-coding using a Cantor set in ‘Incorporating Error Detection into GLS-Coding Using a Cantor Set’ and show simulation results of the proposed method. We conclude with open challenging problems in ‘Conclusions and Future Work’.

GLS-Coding and Decoding

In this section, we shall briefly describe GLS-coding first proposed by Nagaraj, Vaidya & Bhat (2009). We are given a message M, of length L, from a binary i.i.d. source S. Our aim is to losslessly compress M. To this end, we first determine the probability of zeros in M, denoted by p. We then construct the GLS map T_p as follows: ${\displaystyle x_{n+1}=T_p\left(x_n\right)=\left\{\begin{array}{cc}\frac{x_n}{p},\hfill & \text{if}0\le x_n<p,\hfill \\ \frac{1-x_n}{1-p},\hfill & \text{if}p\le x_n<1.\hfill \end{array}\right.}$ The GLS-map, also known as the skew-tent map², T_p, has two intervals: [0, p) and [p, 1) which are tagged with the symbols ‘0’ and ‘1’ respectively. Starting with any initial value x₀ ∈ [0, 1), the above map T_p can be applied iteratively to yield the real-valued time series {x_i}. The symbolic sequence of this time series, denoted by {S_i}, is simply obtained by: ${\displaystyle S_i=\left\{\begin{array}{cc}0,\hfill & \text{if}0\le x_i<p,\hfill \\ 1,\hfill & \text{if}p\le x_i<1.\hfill \end{array}\right.}$ Given the symbolic sequence {S_i}, the inverse of T_p is given by: ${\displaystyle T_p^{-1}\left(y_j\right)=\left\{\begin{array}{cc}p{y}_j,\hfill & \text{if}{S}_j=0,\hfill \\ 1-y_j\left(1-p\right),\hfill & \text{if}{S}_j=1,\hfill \end{array}\right.}$ For GLS-coding, we begin by treating the given message M as a symbolic sequence ${\left\{S_i\right\}}_{i=0}^{i=L-1}$ of length L on the GLS T_p. The goal is to obtain the initial value x₀ ∈ [0, 1) which when iterated forwards on T_p produces the symbolic sequence {S_i} (= message M). To obtain x₀, we begin by initializing START₀ = 0.0 and END₀ = 1.0. We then determine the inverse images of START₀ and END₀ by iterating backwards on T_p, by using $T_p^{-1}$ and starting with the last symbol S_L−1. At the end of the first back-iteration, we obtain START₁ and END₁. We determine the inverse images of START₁ and END₁ (given S_L−2) to determine the new interval [START₂, END₂). This is repeated L times to yield the final interval [START_L−1, END_L−1). All points within this final interval have the same symbolic sequence (=M = {S_i}). We take the mid-point $x_0=\frac{STAR{T}_{L-1}+EN{D}_{L-1}}{2}$ as the compressed file. Since x₀ is a real number (between 0 and 1), we write its binary representation to the file. The number of bits needed to represent the compressed file x₀ is ⌈ − log₂(END_L−1 − START_L−1)⌉ bits. This is proved to be Shannon optimal³ in Nagaraj, Vaidya & Bhat (2009) and Arithmetic coding is shown to be a special case of GLS-coding⁴.

GLS-decoding is straightforward. At the decoder, given the value of p, we construct the GLS (skew-tent map) T_p as described earlier. Given that we know x₀ (the compressed file), all that we need to do is iterate forwards on the map T_p for L (= length of message M) iterations and output the symbolic sequence {S_i}. This is the decoded message and in the absence of any noise, this is exactly the same as M which was input to the encoder.

As an example, consider M = 0010110100 (L = 10). In this case, $p=\frac{6}{10}=0.6$ and T_0.6 is shown in Fig. 1. With START₀ = 0.0 and END₀ = 1.0, we obtain [START₁, END₁) = [0.0, 0.6), [START₂, END₂) = [0.0, 0.36), [START₃, END₃) = [0.8560, 1.0) and so on.

Effect of single-bit error on GLS-decoding: sensitive dependence on initial value of the map

Thus far, we have discussed lossless coding in the absence of any kind of noise. However, in practical applications, noise is unavoidable. If the data is in a compressed form, then it is quite likely that the decoder would be unable to decode or would decode incorrectly. Even detecting whether an error has occurred helps in several communication protocols since a repeat request could be initiated.

In GLS-coding, the compressed file is the initial value of the symbolic sequence (the message M) on the appropriate GLS. Since GLS is a chaotic map, it exhibits sensitive dependence on initial values, the hallmark of deterministic chaos (Alligood, Sauer & Yorke, 1996). A small perturbation in the initial value will result in a symbolic sequence which is uncorrelated to the original symbolic sequence after a few iterations. This means that with a very high probability, even a slight amount of noise that is added to the initial value (compressed file) will result in a wrongly decoded message (symbolic sequence), which will be very different from the actual intended message. This is demonstrated in Fig. 2. The first bit of the compressed file is flipped and GLS-decoding is performed. The difference in the decoded message from the original message is shown in Fig. 2A. As it can be seen, the decoded message is very different from the original message. On the other hand, if the middle bit of the compressed file is flipped then the decoded image is accurate up to 5,000 bits and the remaining 5,000 bits are wrongly decoded (Fig. 2B). The error affects only those bits which are subsequently decoded.

Error Detection Using Cantor Set

In GLS-coding, every real number on the interval [0, 1) represents an initial value (compressed file). Thus, any error in the initial value will result in another real number which is also an initial value, but for an entirely different symbolic sequence (message). It represents a valid compressed file which decodes to a different message. Therefore in order to detect errors, we necessarily require that when noise gets added to the initial value while transmission on the communication channel, it should result in a value that is not a valid compressed file, so that at the decoder it can be flagged for error. This necessarily implies that not all real numbers in the interval [0, 1) can be valid compressed files. We need to restrict the set of valid compressed files to a smaller subset of [0, 1). This subset should be uncountable and dense since it should be able to decode all possible (infinite length) messages. At the same time, it should have negligible measure (zero measure) so that when noise is added, the probability that it falls outside the set is 1. Cantor sets provide the perfect solution.

Repetition codes ${\mathcal{R}}_n$ lie on a cantor set

Repetition codes are the oldest and most basic error detection and correction codes in coding theory. They are frequently used in applications where the cost and complexity of encoding and decoding are a primary concern. Loskot & Beaulieu (2004) provide a long list of practical applications of repetition codes. Repetition codes are robust against impulsive noise and used in retransmission protocols, spread spectrum systems, multicarrier systems, infrared communications, transmit delay diversity, BLAST signaling, rate-matching in cellular systems, and synchronization of ultrawideband systems⁷. Thus, repetition codes are very useful in communications.

They are described as follows. Consider a message from a binary alphabet {0, 1}. A repetition code ${\mathcal{R}}_n$ (n > 1, odd integer) is a block code which assigns: ${\displaystyle 0\mapsto \underset{n}{\underbrace{0\dots 0}}}$ ${\displaystyle 1\mapsto \underset{n}{\underbrace{1\dots 1}}.}$ ${\mathcal{R}}_n$ can correct up to $\frac{n-1}{2}$ bit errors since the minimum hamming distance of ${\mathcal{R}}_n$ is n. Since n is chosen to be an odd positive integer (>1), a majority count in every block of n symbols acts as a very simple but efficient decoding algorithm. The repetition code ${\mathcal{R}}_n$ is a linear block code with a rate $=\frac{1}{n}$.

We shall provide a new interpretation of ${\mathcal{R}}_n$, inspired by Cantor set. Start with the real line segment (0, 1]. Remove the middle (1 − 2⁻ⁿ⁺¹) fraction of the set (0, 1]. In the remaining two intervals, remove the same fraction and repeat this process in a recursive fashion (refer to Fig. 3). When this process is carried over an infinite number of times, the set that remains is a Cantor set. Furthermore, the binary representation of every element of the Cantor set forms the codewords of ${\mathcal{R}}_n$.

In order to see this, consider n = 3. Figure 3 shows how ${\mathcal{R}}_3$ is recursively constructed. If the above step is terminated at iteration k, then there remains a set of intervals whose binary expansion (of length nk) contains the codewords for all possible binary messages of length k. For example, at k = 2 for ${\mathcal{R}}_3$, we can see that there are four intervals which contains real numbers with binary expansions starting from 000000, 000111, 111000 and 111111. These are the codewords for the messages 00, 01, 10 and 11 respectively. In the limit of this process, the set results in a Cantor set of measure zero which contains codewords for all binary messages which are infinitely long.

Incorporating Error Detection into Gls-Coding Using a Cantor Set

Having established that one of the oldest error detection/correction methods, namely repetition codes, belong to a Cantor set, we shall extend this idea of using a Cantor set for error detection to GLS-coding. First, we establish a connection between repetition codes and GLS-coding.

Conclusions and Future Work

In this work, we have provided a novel application of Cantor sets for incorporating error detection into a lossless data compression algorithm (GLS-coding). Cantor sets have paradoxical properties that enable error detection and correction. Repetition codes are an example of codewords on a self-similar Cantor set which can detect and correct errors. By reserving a forbidden symbol on the interval [0, 1), we can ensure that the codewords for GLS-coding lie on a Cantor set and thereby detect errors while simultaneously GLS-decoding (thus preserving progressive transmission property), and without significantly increasing the compressed file size. This approach can be applied to any mode of GLS and generalizable to larger alphabets. However, we do not know whether other efficient error control codes can be similarly designed using such Cantor sets (or other fractals in higher dimensions) and whether we can exploit the structure of the Cantor set to perform efficient error correction. These are important research directions worth exploring in the future.

Supplemental Information