An analysis of the locality of binary representations in genetic and evolutionary algorithms

The point locality p_r for a representation r is $p_r\stackrel{\mathrm{def}}{=}\overline{|r\left({\hat{s}}_i\right)-r\left(s\right)|}\forall s\in {\left\{0,1\right\}}^{\ell },\forall i\in \left[\right.0,\ell \left.\right)$. Explicitly, (3)${\displaystyle p_r\stackrel{\mathrm{def}}{=}\frac{\sum _s\sum _i|r\left({\hat{s}}_i\right)-r\left(s\right)|}{2^{\ell }\cdot \ell }.}$

Theorem 10: Expected Point Locality

$\mathbb{E}\left[p_r\right]=\frac{2^{\ell }+1}{3}$ for nonredundant binary-integer representations.

We prove this by direct computation, using a few identities, starting with the definition of p_r: ${\displaystyle p_r\stackrel{\mathrm{def}}{=}\frac{\sum _s\sum _i|r\left({\hat{s}}_i\right)-r\left(s\right)|}{\ell 2^{\ell }}}$ (4)${\displaystyle ⟹\mathbb{E}\left[p_r\right]=\frac{1}{\ell 2^{\ell }}\mathbb{E}\left[\sum _s\sum _i|r\left({\hat{s}}_i\right)-r\left(s\right)|\right].}$ Let L = {0, …, 2^ℓ − 1}. Computing the expectation in Eq. (4) is equivalent to assigning numbers in L to vertices of the ℓ-cube uniformly at random, and then summing the absolute differences between every adjacent pair. Let k denote the value assigned to vertex s, and let π_k denote a size ℓ random subset of L∖{k}.

 Due to linearity of expectation, the sum of absolute differences for each vertex s can be examined independently even though they are dependent variables. Then, s has ℓ neighbors where the ith neighbor is assigned the ith element in π_k, which we denote π_k(i). This gives us the following relation (letting n = 2^ℓ for simplicity): (5)${\displaystyle \mathbb{E}\left[\sum _s\sum _i|r\left({\hat{s}}_i\right)-r\left(s\right)|\right]=\mathbb{E}\left[\sum _{k=0}^{n-1}\sum _{i=1}^{\ell }|{\pi }_k\left(i\right)-k|\right].}$ Since every number in L∖{k} has equal probability of appearing as the ith element of π_k, 𝔼[π_k(i)] does not depend on i at all. Therefore, once we apply linearity of expectation on the right side of Eq. (5), we can replace π_k(i) with a discrete uniform random variable sampled from L∖{k}. Denote this random variable as X^k. We then have: ${\displaystyle =\ell \sum _{k=0}^{n-1}\mathbb{E}\left[|X^k-k|\right]=\ell \sum _{k=0}^{n-1}\sum _{x\in L\setminus \left\{k\right\}}|x-k|\cdot \text{P}\left(X^k=x\right)=\frac{\ell }{n-1}\sum _{k=0}^{n-1}\sum _{x\in L\setminus \left\{k\right\}}|x-k|=\frac{\ell }{n-1}\sum _{k=0}^{n-1}\left(S_{1,k}+S_{1,n-k-1}\right),}$ where we let $S_{i,j}={\sum }_{m=i}^{j}m$. Continuing with the fact that $S_{1,m}=\frac{m\left(m+1\right)}{2}$: ${\displaystyle =\frac{\ell }{2\left(n-1\right)}\sum _{k=0}^{n-1}\left(\left(n-k-1\right)\left(n-k\right)+k\left(k+1\right)\right)=\frac{\ell }{2\left(n-1\right)}\left(\sum _{k=0}^{n-1}{\left(n-k\right)}^2-\sum _{k=0}^{n-1}\left(n-k\right)+\sum _{k=0}^{n-1}{k}^2+\sum _{k=0}^{n-1}k\right).}$ Let $Q_{i,j}={\sum }_{m=i}^j{m}^2$. Then ${\displaystyle =\frac{\ell }{2\left(n-1\right)}\left(Q_{1,n}-S_{1,n}+Q_{1,n-1}+S_{1,n-1}\right)=\frac{\ell }{2\left(n-1\right)}\left(2Q_{1,n-1}+n^2-n\right).}$ Using the fact that $Q_{1,m}=\frac{m\left(m+1\right)\left(2m+1\right)}{6}$ and simplifying: ${\displaystyle =\frac{\ell }{2\left(n-1\right)}\cdot \frac{2n\left(n-1\right)\left(n+1\right)}{3}=\frac{\ell 2^{\ell }\left(2^{\ell }+1\right)}{3}.}$ Substituting back into Eq. (4) yields: ${\displaystyle \mathbb{E}\left[p_r\right]=\frac{1}{\ell 2^{\ell }}\cdot \frac{\ell 2^{\ell }\left(2^{\ell }+1\right)}{3}=\frac{2^{\ell }+1}{3}.}$

The general locality for a representation r is (6)${\displaystyle g_r\stackrel{\mathrm{def}}{=}\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\sum _{\left(s_1,s_2\right)\in S_{\ell }}|d^p\left(s_1,s_2\right)-d^g\left(s_1,s_2\right)|,}$ where S_ℓ is the set of all unordered pairs from {0, 1}^ℓ (so $\left|S_{\ell }\right|=\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)$), d^p(s₁, s₂) is the phenotypic distance |r(s₁) − r(s₂)|, and d^g(s₁, s₂) is the genotypic (Hamming) distance between s₁ and s₂. Note that our definition of general locality mirrors Chiam’s remoteness preservation and is also equivalent to Rothlauf’s d_c metric, since ${\sum }_{i=0}^{n_p}{\sum }_{j=i+1}^{n_p}$ is identical to ∑_{(s1,s2)∈Sℓ}.

(Lower bound) $g_r\ge \frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(\frac{1}{6}\left(2^{\ell }-1\right)\left(2^{\ell }\right)\left(2^{\ell }+1\right)-\ell 2^{2\left(\ell -1\right)}\right)$ for nonredundant binary-integer representations.

By definition, $g_r=\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}{\sum }_{\left(s_1,s_2\right)\in S_{\ell }}|d^p\left(s_1,s_2\right)-d^g\left(s_1,s_2\right)|$. By the triangle inequality, we have ${\displaystyle g_r\ge \frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\sum _{\left(s_1,s_2\right)\in S_{\ell }}\left(d^p\left(s_1,s_2\right)-d^g\left(s_1,s_2\right)\right)=\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(\sum _{\left(s_1,s_2\right)\in S_{\ell }}{d}^p\left(s_1,s_2\right)-\sum _{\left(s_1,s_2\right)\in S_{\ell }}{d}^g\left(s_1,s_2\right)\right)=\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(P-G\right),}$ where we let P = ∑_{(s1,s2)∈Sℓ}d^p(s₁, s₂) and G = ∑_{(s1,s2)∈Sℓ}d^g(s₁, s₂). Since P only deals with phenotypes in ℕ, it is equivalent to ${\displaystyle P=\sum _{i=0}^{2^{\ell }-1}\sum _{j=i+1}^{2^{\ell }-1}\left(j-i\right).}$ Let the outer sum fix i. The inner sum computes the sum of numbers from 1 to 2^ℓ − 1 − i. This reduces P to ${\displaystyle =\sum _{i=1}^{n}i+\sum _{i=1}^{n-1}i+\cdots +\sum _{i=1}^{1}i.}$ where we let n = 2^ℓ − 1 for simplicity. Using the facts that the sum of the first m natural numbers is $\frac{1}{2}m\left(m+1\right)$ and the sum of the first m squares is $\frac{1}{6}m\left(m+1\right)\left(2m+1\right)$, we have ${\displaystyle =\frac{1}{2}\sum _{i=1}^{n}i\left(i+1\right)=\frac{1}{2}\left(\sum _{i=1}^n{i}^2+\sum _{i=1}^{n}i\right)=\frac{1}{2}\left(\frac{1}{6}n\left(n+1\right)\left(2n+1\right)+\frac{1}{2}n\left(n+1\right)\right)=\frac{1}{6}n\left(n+1\right)\left(n+2\right).}$ Substituting n = 2^ℓ − 1 back into the equation yields: ${\displaystyle =\frac{1}{6}\left(2^{\ell }-1\right)\left(2^{\ell }\right)\left(2^{\ell }+1\right).}$

Now, since G is the sum of the Hamming distances between all unique pairs of bitstrings, it is equivalent to ${\displaystyle G=\frac{1}{2}{2}^{\ell }\sum _{i=1}^{\ell }i\left(\genfrac{}{}{0.0pt}{}{\ell }{i}\right),}$

because for each of the 2^ℓ bitstrings, a bitstring has $\left(\genfrac{}{}{0.0pt}{}{\ell }{i}\right)$ other bitstrings with Hamming distance i (choose i of the ℓ bits to be flipped). We divide by two because we count each pair twice. Simplifying G gives us ${\displaystyle =2^{\ell -1}\sum _{i=1}^{\ell }i\cdot \frac{\ell }{i}\left(\genfrac{}{}{0.0pt}{}{\ell -1}{i-1}\right)=\ell 2^{\ell -1}\sum _{i=1}^{\ell }\left(\genfrac{}{}{0.0pt}{}{\ell -1}{i-1}\right)=\ell 2^{\ell -1}{2}^{\ell -1}=\ell 2^{2\left(\ell -1\right)}.}$ Substituting P and G back into g_r yields ${\displaystyle g_r\ge \frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(P-G\right)=\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(\frac{1}{6}\left(2^{\ell }-1\right)\left(2^{\ell }\right)\left(2^{\ell }+1\right)-\ell 2^{2\left(\ell -1\right)}\right).}$

$g_r\sim \frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(\frac{1}{6}\left(2^{\ell }-1\right)\left(2^{\ell }\right)\left(2^{\ell }+1\right)-\ell 2^{2\left(\ell -1\right)}\right)$ for any nonredundant binary-integer representation r on ℓ bits. That is, as ℓ grows, the value of g_r is independent of the actual representation.

The key intuition behind this proof is that for nonredundant binary-integer representations, as ℓ grows, the phenotypic distances grow at an asymptotically greater rate than the genotypic distances. We can separate the phenotypic distances from the genotypic distances by partitioning S_ℓ into two sets $S_{\ell }=S_{\ell }^p\bigsqcup S_{\ell }^g$, where ⊔ denotes disjoint union: ${\displaystyle S_{\ell }^p=\left\{\left(s_1,s_2\right)\in S_{\ell }|d^p\left(s_1,s_2\right)>d^g\left(s_1,s_2\right)\right\},}$ ${\displaystyle S_{\ell }^g=\left\{\left(s_1,s_2\right)\in S_{\ell }|d^p\left(s_1,s_2\right)\le d^g\left(s_1,s_2\right)\right\}.}$ In other words, $S_{\ell }^p$ contains all the pairs in S_ℓ where the two bitstrings have greater phenotypic (Euclidean) distance than genotypic (Hamming) distance, and $S_{\ell }^g$ contains all pairs where the two bitstrings have greater or equal genotypic distance than phenotypic distance. We can rewrite g_r as (letting $C=1∕\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)$) ${\displaystyle g_r=\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\sum _{\left(s_1,s_2\right)\in S_{\ell }}|d^p\left(s_1,s_2\right)-d^g\left(s_1,s_2\right)|=C\left(P\left(\ell \right)+G\left(\ell \right)\right),}$ where we let (7)${\displaystyle P\left(\ell \right)=\sum _{\left(s_1,s_2\right)\in S_{\ell }^p}\left(d^p\left(s_1,s_2\right)-d^g\left(s_1,s_2\right)\right),}$ (8)${\displaystyle G\left(\ell \right)=\sum _{\left(s_1,s_2\right)\in S_{\ell }^g}\left(d^g\left(s_1,s_2\right)-d^p\left(s_1,s_2\right)\right).}$ Note that $\left|S_{\ell }^g\right|\le \frac{1}{2}\left(2\ell -1\right)2^{\ell }$ since each of the 2^ℓ bitstrings can have at most 2ℓ − 1 bitstrings for which their genotypic distance is greater or equal to their phenotypic distance. This is because for an integer i ≥ ℓ, the integers between i − ℓ and i + ℓ can be represented by bitstrings with genotypic distances greater than phenotypic distances j ≤ ℓ, where we subtract 1 due to the fact that each bitstring only has one other bitstring with Hamming distance ℓ. We then divide by two since we count each pair twice. Thus $\left|S_{\ell }^g\right|=\mathcal{O}\left(\ell 2^{\ell }\right)$. We can now say $\left|S_{\ell }^p\right|=\left|S_{\ell }\right|-\left|S_{\ell }^g\right|\ge \left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)-\frac{1}{2}\left(2\ell -1\right)2^{\ell }=\frac{2^{\ell }\left(2^{\ell }-2\ell \right)}{2}$, so $\left|S_{\ell }^p\right|=\Omega \left(2^{2\ell }\right)$. Consider ${\displaystyle \underset{\ell \to \infty }{lim}\frac{\left|S_{\ell }\right|}{\left|S_{\ell }^p\right|}=\underset{\ell \to \infty }{lim}\frac{\left|S_{\ell }^p\right|+\left|S_{\ell }^g\right|}{\left|S_{\ell }^p\right|}=\underset{\ell \to \infty }{lim}\left(1+\frac{\left|S_{\ell }^g\right|}{\left|S_{\ell }^p\right|}\right)=1+0=1,}$ since $\left|S_{\ell }^p\right|=\Omega \left(2^{2\ell }\right)$ grows faster than $\left|S_{\ell }^g\right|=\mathcal{O}\left(\ell 2^{\ell }\right)$. Thus $\left|S_{\ell }^p\right|$ dominates $\left|S_{\ell }^g\right|$, and $\left|S_{\ell }\right|\sim \left|S_{\ell }^p\right|$.

We can now perform a similar analysis for P(ℓ) and P(ℓ) + G(ℓ). Note that $G\left(\ell \right)\le \left|S_{\ell }^g\right|\left(\ell -1\right)\le \frac{1}{2}{\ell }^2{2}^{\ell }-\frac{1}{2}\ell 2^{\ell }$ since any pair in $S_{\ell }^g$ can have a maximum d^g − d^p of ℓ − 1. Thus $G\left(\ell \right)=\mathcal{O}\left({\ell }^2{2}^{\ell }\right)$. Also note that $P\left(\ell \right)\ge \left|S_{\ell }^p\right|\ge \frac{2^{\ell }\left(2^{\ell }-2\ell \right)}{2}$ since any pair in $S_{\ell }^p$ can have a minimum d^p − d^g of 1. Thus P(ℓ) = Ω(2^2ℓ). Consider ${\displaystyle \underset{\ell \to \infty }{lim}\frac{P\left(\ell \right)+G\left(\ell \right)}{P\left(\ell \right)}=\underset{\ell \to \infty }{lim}1+\frac{G\left(\ell \right)}{P\left(\ell \right)}=1+0=1,}$ since P(ℓ) = Ω(2^2ℓ) grows faster than $G\left(\ell \right)=\mathcal{O}\left({\ell }^2{2}^{\ell }\right)$, and so P(ℓ) + G(ℓ) ∼ P(ℓ). Now we can make a statement about g_r. We have ${\displaystyle \frac{1}{C}{g}_r=P\left(\ell \right)+G\left(\ell \right)}$ ${\displaystyle \underset{\ell \to \infty }{lim}\frac{g_r}{CP\left(\ell \right)}=\underset{\ell \to \infty }{lim}\frac{P\left(\ell \right)+G\left(\ell \right)}{P\left(\ell \right)}}$ ${\displaystyle \underset{\ell \to \infty }{lim}\frac{g_r}{CP\left(\ell \right)}=1.}$ Thus $\frac{g_r}{C}\sim P\left(\ell \right)$. Since $\left|S_{\ell }^p\right|\sim \left|S_{\ell }\right|$ and CP(ℓ) ∼ g_r, we replace $S_{\ell }^p$ with S_ℓ in Eq. (7) to obtain (recall $C=1∕\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)$) ${\displaystyle g_r\sim \frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\sum _{\left(s_1,s_2\right)\in S_{\ell }}\left(d^p\left(s_1,s_2\right)-d^g\left(s_1,s_2\right)\right)=\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(\frac{1}{6}\left(2^{\ell }-1\right)\left(2^{\ell }\right)\left(2^{\ell }+1\right)-\ell 2^{2\left(\ell -1\right)}\right),}$ which was found in the proof of Theorem 11. Thus g_r and $\frac{1}{\left(\genfrac{}{}{0.0pt}{}{2^{\ell }}{2}\right)}\left(\frac{1}{6}\left(2^{\ell }-1\right)\left(2^{\ell }\right)\left(2^{\ell }+1\right)-\ell 2^{2\left(\ell -1\right)}\right)$ are asymptotically equal for any representation r.