Qualitative reachability for open interval Markov chains

Well-formed IMCs

In the sequel, we will consider only IMCs for which there exists at least one assignment for each state. This restriction can be captured by the following syntactic conditions on the transition function. Let s ∈ S be a state. The well-formedness conditions for s are defined as follows:

1. ∑_s′∈S left(δ(s, s′)) ≤ 1,

2. ∑_s′∈S left(δ(s, s′)) = 1 implies that δ(s, s′) is left-closed for all s′ ∈ S,

3. ∑_s′∈S right(δ(s, s′)) ≥ 1, and

4. ∑_s′∈S right(δ(s, s′)) = 1 implies that δ(s, s′) is right-closed for all s′ ∈ S.

The following proposition establishes the correspondence between the well-formedness conditions and the existence of an assignment, and generalises to open IMCs similar assertions for IMCs with closed intervals only (for example, in Section 4 of Haddad & Monmege (2018)).

Proposition 1 Let (S, δ) be an IMC and s ∈ S be a state. Then the well-formedness conditions for s are satisfied if and only if there exists an assignment for s.                  □

Henceforth we assume that the IMCs that we consider satisfy the well-formedness conditions for each of their states.

Edges of IMCs

In the following, we refer to edges as those state pairs to which the transition function can associate positive probability. There are two situations in which the transition function enforces the association of probability 0 to a state pair (s, s′) ∈ S × S: first, when the right endpoint of δ(s, s′) is equal to 0 (and hence δ(s, s′) is the interval [0, 0]); second, when the sum of the left endpoints of transitions from s that do not have s′ as target state is equal to 1 (thereby preventing the assignment of positive probability to (s, s′)). Hence we define formally the set of edges E of O as the smallest set such that (s, s′) ∈ E if (1) right(δ(s, s′)) > 0 and (2) ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) < 1.²

We now formalise the intuition that edges are those state pairs which can be assigned positive probability by relating the notions of edges and assignments.

Proposition 2 Let (S, δ) be an IMC and s, s′ ∈ S be states. Then (s, s′) ∈ E if and only if there exists an assignment a for s such that a(s′) > 0.                                     □

We use edges to define the notion of path for IMCs: a path of an IMC O = (S, δ) is a sequence s₀s₁⋯ such that (s_i, s_i+1) ∈ E for all i ≥ 0. Given a path ρ = s₀s₁⋯ and i ≥ 0, we let state[ρ](i) = s_i be the (i + 1)-th state along ρ. We use ${\mathit{Paths}}_{\infty }^O$ to denote the set of paths of O, ${\mathit{Paths}}_{\ast }^O$ to denote the set of finite paths of O, and ${\mathit{Paths}}_{\infty }^O\left(s\right)$ and ${\mathit{Paths}}_{\ast }^O\left(s\right)$ to denote the sets of paths and finite paths starting in state s ∈ S. We refer to (S, E) as the graph of O.

We define the size of an IMC O = (S, δ) as the size of the representation of δ, which is the sum over all states s, s′ ∈ S of the binary representation of the endpoints of δ(s, s′), where rational numbers are encoded as the quotient of integers written in binary.

IMCs are presented typically with regard to two semantics, which we consider in turn. Given an IMC O = (S, δ), the uncertain Markov chain (UMC) semantics of O, denoted by [O]_U, is the smallest set of DTMCs such that (S, P) ∈ [O]_U if, for each state s ∈ S, the distribution P(s, ⋅) is an assignment for s. The interval Markov decision process (IMDP) semantics of O, denoted by [O]_I, is the MDP (S, Δ) where, for each state s ∈ S, we let Δ(s) be the set of assignments for s.

Let T⊆S be a set of states of IMC O = (S, δ). We define $\mathsf{Reach}\left(T\right)\subseteq {\mathit{Paths}}_{\infty }^O$ to be the set of paths of O that reach at least one state in T. Formally, $\mathsf{Reach}\left(T\right)=\left\{\rho \in {\mathit{Paths}}_{\infty }^O\mid \exists i\in \mathbb{N}.\mathsf{state}\left[\rho \right]\left(i\right)\in T\right\}$. In the following we assume without loss of generality that states in T are absorbing in all the IMCs that we consider, i.e., δ(s, s) = [1, 1] for all states s ∈ T.

Valid edge sets

Let O = (S, δ) be an IMC with edge set E, and let s ∈ S be a state of O. Edge (s′, s^′′) ∈ E has source s if s = s′ and target s if s = s^′′. Let E(s) = {(s, s′) ∈ E∣s′ ∈ S} be the set of edges of O with source s. Given ⋆ ∈ {[⋅, ⋅], (⋅, ⋅], [⋅, ⋅), (⋅, ⋅), 〈 + , ⋅〉, [0, ⋅〉, (0, ⋅〉, 〈0, ⋅〉, 〈⋅, ⋅]}, let E^⋆ = {(s, s′) ∈ E∣δ(s, s′) ∈ ℐ^⋆}. Given X⊆S, and given s and ⋆ as defined above, let E(s, X) = {(s, s′) ∈ E(s)∣s′ ∈ X} be the set of edges with source s and target in X, and let E^⋆(s, X) = E(s, X)∩E^⋆.

In the sequel, we will be interested in identifying the sets of edges with source state s ∈ S and target states that correspond exactly to the set of states assigned positive probability by assignments for s. This will allow us to reason about sets of edges with certain characteristics rather than reasoning directly about assignments. The characteristics of sets of edges with the same source state that we consider are the following: the first condition, called largeness, requires that the sum of the upper bounds of the set’s edges’ intervals is at least 1; the second condition, called realisability, requires that the edges that are not included in the set can be assigned probability 0. The formal definition of these characteristics now follows.

The following lemma specifies that a valid set of edges with source state s characterises exactly the support sets of at least one assignment for s. In its statement and proof we use the following notation. Given s ∈ S and B⊆E(s), we partition S into the following sets:

• T_B = {s′∣(s, s′) ∈ B} denotes the target states of edges in B,

• T_E(s)∖B = {s′∣(s, s′) ∈ E(s)∖B} denotes the target states of edges with source s that are not in B, and

• S_¬E(s) = S∖{s′∣(s, s′) ∈ E(s)} denotes states that are not target states of the edges that have source state s.

First we identify conditions that are analogues of well-formedness conditions restricted to the set B of edges. Then, using these conditions, we proceed as in the (⇒) direction of Proposition 1 to define an assignment that assigns positive probability only to target states of edges in B.

The well-formedness conditions (1) and (2), together with the fact that B⊆E(s), establishes the following conditions ($\tilde{1}$) and ($\tilde{2}$), whereas the largeness of B establishes the following conditions ($\tilde{3}$) and ($\tilde{4}$):

1. ∑_e∈Bleft(δ(e)) ≤ 1,

2. ∑_e∈Bleft(δ(e)) = 1 implies that δ(e) is left-closed for all e ∈ B,

3. ∑_e∈Bright(δ(e)) ≥ 1, and

4. ∑_e∈Bright(δ(e)) = 1 implies that δ(e) is right-closed for all e ∈ B.

It will be useful to consider the following strengthening of condition ($\tilde{2}$):

1. ∑_e∈Bleft(δ(e)) = 1 implies that δ(s, s′) is left-closed for all s′ ∈ S.

Condition (${\tilde{2}}^{\prime }$) has the following justification. Assume that ∑_e∈Bleft(δ(e)) = 1. From this fact, and from the combination of ∑_s′∈Sleft(δ(s, s′)) ≤ 1 (that is, condition (1) of well-formedness) and the fact that B⊆{s} × S, we have that ∑_s′∈Sleft(δ(s, s′)) = 1. Then, by condition (2) of well-formedness, we have that δ(s, s′) is left-closed for all s′ ∈ S. We note that an analogous strengthening of condition ($\tilde{4}$) cannot in general be obtained (because it is possible that there exists at least one pair (s, s′) ∈ ({s} × S)∖B such that right(δ(s, s′)) > 0 without contradicting well-formedness condition (3)).

Next, we define an assignment in a similar manner to that featured in the (⇒) direction of Proposition 1, taking care to guarantee that the assignment we define assigns positive probability only to target states of edges in B. As in the (⇒) direction of Proposition 1, we define a function f:S → [0, 1] such that:

1. left(δ(s, s′)) + f(s′) ∈ δ(s, s′) for all s′ ∈ S and

2. ∑_s′∈Sleft(δ(s, s′)) + f(s′) = 1.

We define a:S → [0, 1] by setting a(s′) = left(δ(s, s′)) + f(s′) for all s′ ∈ S. As in the proof of Proposition 1, the function a is an assignment, because condition (a) in the definition of f establishes that a(s′) ∈ δ(s, s′) for each state s′ ∈ S, and condition (b) establishes that a ∈ Dist(S).

In order to define an appropriate f, we consider the following cases.

Case ∑_e∈Bleft(δ(e)) = 1. As in analogous cases of the proof of Proposition 1, we let f(s′) = 0 for all s′ ∈ S. Condition (a) in the definition of f is satisfied for the following reason. Given that ∑_e∈Bleft(δ(e)) = 1, the antecedent of condition (${\tilde{2}}^{\prime }$) is satisfied, and hence δ(s, s′) is left-closed (i.e., left(δ(s, s′)) ∈ δ(s, s′)) for all s′ ∈ S; then the choice of f(s′) = 0 for all s′ ∈ S means that left(δ(s, s′)) + f(s′) ∈ δ(s, s′) is satisfied for all s′ ∈ S, hence establishing condition (a). We now establish the satisfaction of condition (b) in the definition of f. Given that f(s′) = 0 for all s′ ∈ S, showing condition (b) reduces to showing that ∑_s′∈Sleft(δ(s, s′)) = 1. Given that ∑_e∈Bleft(δ(e)) = 1, our task reduces in turn to showing that left(δ(s, s′)) = 0 for all s′ ∈ S∖T_B. Recall that S∖T_B = T_E(s)∖B∪S_¬E(s) (because T_B, T_E(s)∖B and S_¬E(s) form a partition of S). We proceed first by showing that left(δ(s, s′)) = 0 for all s′ ∈ T_E(s)∖B. Given that E(s)∖B⊆E^[0,⋅〉 from the realisability of B, and that left(δ(e)) = 0 for all e ∈ E^[0,⋅〉 by definition, we have that left(δ(s, s′)) = 0 for all s′ ∈ T_E(s)∖B. Next, we show that left(δ(s, s′)) = 0 for all s′ ∈ S_¬E(s). From the definition of edges, for s′ ∈ S_¬E(s), either right(δ(s, s′)) = 0 or ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) ≥ 1. In the case of right(δ(s, s′)) = 0, from left(δ(s, s′)) ≤ right(δ(s, s′)) (by definition of δ), we must have left(δ(s, s′)) = 0. In the case of ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) ≥ 1, given that well-formedness condition (1) specifies that ∑_s^′′∈Sleft(δ(s, s^′′)) ≤ 1, we must have left(δ(s, s′)) = 0. Hence we have established condition (b) in the definition of f. Next we establish that support(a) = T_B. First we show that support(a)⊆T_B. From the reasoning in the previous paragraph, we have left(δ(s, s′)) = 0 for all s′ ∈ S∖T_B. Given that f(s′) = 0 for all s′ ∈ S, we have a(s′) = left(δ(s, s′)) + f(s′) = 0 for all s′ ∈ S∖T_B. Hence support(a)⊆T_B. Second we show that support(a)⊇T_B. Note that ∑_e∈Bleft(δ(e)) = 1 holds in the definition of this case, and that ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) < 1 holds for all (s, s′) ∈ E from the definition of edges. Combining these two facts allows us to conclude that left(δ(e)) > 0 for all e ∈ B. Then a(s′) = left(δ(s, s′)) + f(s′) > 0 for all s′ ∈ T_B, establishing that support(a)⊇T_B.

Case ∑_e∈Bleft(δ(e)) < 1 and ∑_e∈Bright(δ(e)) = 1. We let f(s′) = right(δ(s, s′)) − left(δ(s, s′)) for all s′ ∈ T_B, and let f(s′) = 0 for all s′ ∈ S∖T_B. Hence left(δ(s, s′)) + f(s′) = right(δ(s, s′)) for all s′ ∈ T_B, and left(δ(s, s′)) + f(s′) = left(δ(s, s′)) for all s′ ∈ S∖T_B. First we establish that the choice of f above yields an assignment by showing that conditions (a) and (b) in the definition of f are satisfied. First consider condition (a). The antecedent of condition ($\tilde{4}$) is satisfied for this case, hence δ(e) is right-closed for all e ∈ B, i.e., right(δ(s, s′)) ∈ δ(s, s′) for all s′ ∈ T_B. For s′ ∈ S∖T_B, because f(s′) = 0, we need to show that left(δ(s, s′)) ∈ δ(s, s′), i.e., that δ(s, s′) is left-closed. Recalling that T_E(s)∖B and S_¬E(s) form a partition of S∖T_B, we have two sub-cases based on whether s′ belongs to T_E(s)∖B or to S_¬E(s). For s′ ∈ T_E(s)∖B, given that E(s)∖B⊆E^[0,⋅〉 from the realisability of B, we have that δ(s, s′) is left-closed. For s′ ∈ S_¬E(s), from the definition of edges, we either have right(δ(s, s′)) = 0 or ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) ≥ 1. For right(δ(s, s′)) = 0, it must be the case that δ(s, s′) = [0, 0], which is left-closed. For ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) ≥ 1, given that well-formedness condition (1) specifies that ∑_s^′′∈Sleft(δ(s, s^′′)) ≤ 1, we must have ∑_s^′′∈Sleft(δ(s, s^′′)) = 1. Then well-formedness condition (2) specifies that δ(s, s′) is left-closed. Hence we have established condition (a) in the definition of f. Next we turn our attention to condition (b) in the definition of f. Recall that T_B, T_E(s)∖B and S_¬E(s) form a partition of S. First consider T_B. Noting that ∑_e∈Bright(δ(e)) = 1, we have ∑_s′∈TBright(δ(s, s′)) = 1. Then, from the choice of f, we have ∑_s′∈TBleft(δ(s, s′)) + f(s′) = 1. Now consider T_E(s)∖B. Recall that, for s′ ∈ T_E(s)∖B, we have chosen f(s′) = 0 and established that left(δ(s, s′)) = 0 (because δ(s, s′) = [0, 0]) above, and hence ∑_{s′∈TE(s)∖B}left(δ(s, s′)) + f(s′) = 0. Finally, for s′ ∈ S_¬E(s), we either have right(δ(s, s′)) = 0, in which case left(δ(s, s′)) = 0, or ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) ≥ 1, in which case the well-formedness condition (1) specifying that ∑_s^′′∈Sleft(δ(s, s^′′)) ≤ 1 establishes that left(δ(s, s′)) = 0. Given that f(s′) = 0 for all s′ ∈ S∖T_B and S_¬E(s)⊆S∖T_B, we have ∑_{s′∈S¬E(s)}left(δ(s, s′)) + f(s′) = 0. From the fact that T_B, T_E(s)∖B and S_¬E(s) form a partition of S, we then conclude that: ${\displaystyle \sum _{s^{\prime }\in S}\mathsf{left}\left(\delta \left(s,s^{\prime }\right)\right)+f\left(s^{\prime }\right)=\sum _{s^{\prime }\in T_B}\mathsf{right}\left(\delta \left(s,s^{\prime }\right)\right)=1.}$ Hence we have established condition (b) in the definition of f, and therefore the choice of f yields an assignment.

It remains to show that support(a) = T_B. From the reasoning in the previous two paragraphs, we have established that left(δ(s, s′)) + f(s′) = right(δ(s, s′)) for all s′ ∈ T_B, and left(δ(s, s′)) + f(s′) = 0 for all s′ ∈ S∖T_B. Hence support(a)⊆T_B. From the fact that right(δ(s, s′)) > 0 for s′ ∈ S such that (s, s′) ∈ E(s), and from B⊆E(s), we have that right(δ(s, s′)) > 0 for all s′ ∈ T_B. Hence support(a)⊇T_B. We then conclude that support(a) = T_B.

Case ∑_e∈Bleft(δ(e)) < 1 < ∑_e∈Bright(δ(e)). For each s′ ∈ T_B, we let: ${\displaystyle f\left(s^{\prime }\right)=\left(1-\sum _{s^{\prime \prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime \prime }\right)\right)\right)\cdot \frac{\mathsf{width}\left(\delta \left(s,s^{\prime }\right)\right)}{\sum _{s^{\prime \prime }\in T_B}\mathsf{width}\left(\delta \left(s,s^{\prime \prime }\right)\right)},}$ and let f(s′) = 0 for each s′ ∈ S∖T_B. First we show that this definition of f satisfies condition (a), i.e., left(δ(s, s′)) + f(s′) ∈ δ(s, s′) for all s′ ∈ S. In the sub-case in which left(δ(s, s′)) = right(δ(s, s′)) (i.e., δ(s, s′) is the degenerate interval [left(δ(s, s′)), right(δ(s, s′))]), we have f(s′) = 0, because either s′ ∈ S∖T_B, or s′ ∈ T_B and f(s′) = 0 is a consequence of the fact that width(δ(s, s′)) = 0. As in the proof of Proposition 1, we then conclude that left(δ(s, s′)) + f(s′) ∈ δ(s, s′). Now consider the sub-case in which left(δ(s, s′)) < right(δ(s, s′)). Recall again that T_B, T_E(s)∖B and S_¬E(s) form a partition of S. If s′ ∈ T_E(s)∖B or s′ ∈ S_¬E(s), we conclude, following identical reasoning used in the previous case, that left(δ(s, s′)) + f(s′) ∈ δ(s, s′). On the other hand, if s′ ∈ T_B, we proceed in a similar manner to that of the analogous case of Proposition 1. In order to show left(δ(s, s′)) + f(s′) ∈ δ(s, s′), we will establish that 0 < f(s′) < width(δ(s, s′)). The fact that f(s′) > 0 is a consequence of the following three facts: 1 − ∑_e∈Bleft(δ(e)) > 0 (from the definition of the case), width(δ(s, s′)) > 0 (given that we are considering the sub-case of left(δ(s, s′)) < right(δ(s, s′))), and ∑_{s^′′∈TB}width(δ(s, s^′′)) > 0 (from the previous fact and from the fact that width(δ(s, s^′′)) ≥ 0 for all s^′′ ∈ S). We now show that f(s′) < width(δ(s, s′)). From the definition of f, this requires showing that $\frac{1-{\sum }_{s^{\prime \prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime \prime }\right)\right)}{{\sum }_{s^{\prime \prime }\in T_B}\mathsf{width}\left(\delta \left(s,s^{\prime \prime }\right)\right)}<1$. Given that we are considering the case in which 1 < ∑_e∈Bright(δ(e)), i.e., 1 < ∑_{s^′′∈TB}left(δ(s, s^′′)) + width(δ(s, s^′′)), we can conclude that 1 − ∑_{s^′′∈TB}left(δ(s, s^′′)) < ∑_{s^′′∈TB}width(δ(s, s^′′)) and hence $\frac{1-{\sum }_{s^{\prime \prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime \prime }\right)\right)}{{\sum }_{s^{\prime \prime }\in T_B}\mathsf{width}\left(\delta \left(s,s^{\prime \prime }\right)\right)}<1$. Thus we have established that the choice of f(s′) satisfies left(δ(s, s′)) + f(s′) ∈ δ(s, s′) for s′ ∈ T_B. We now proceed to show that f satisfies condition (b), i.e., ∑_s′∈Sleft(δ(s, s′)) + f(s′) = 1. Recall that T_B, T_E(s)∖B and S_¬E(s) form a partition of S, and that f(s′) = 0 for each s′ ∈ S∖T_B. Following similar reasoning to that of the previous case and of the proof of Proposition 1, we show that left(δ(s, s′)) = 0, and hence left(δ(s, s′)) + f(s′) = 0, for each s′ ∈ S∖T_B. For s′ ∈ T_E(s)∖B, we have left(δ(s, s′)) = 0 from the fact that E(s)∖B⊆E^[0,⋅〉 because B is realisable. For s′ ∈ S_¬E(s), we have left(δ(s, s′)) = 0 by the definition of edges: either right(δ(s, s′)) = 0 and hence left(δ(s, s′)) = 0, or the combination of ∑_{s^′′∈S∖{s′}}left(δ(s, s^′′)) ≥ 1 and ∑_s^′′∈Sleft(δ(s, s^′′)) ≤ 1 (well-formedness condition (1)) establishes that left(δ(s, s′)) = 0. Given that left(δ(s, s′)) + f(s′) = 0, for each s′ ∈ S∖T_B, our aim reduces to showing that ∑_s′∈TBleft(δ(s, s′)) + f(s′) = 1. We proceed as for the analogous case in Proposition 1: ${\displaystyle \sum _{s^{\prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime }\right)\right)+f\left(s^{\prime }\right)=\sum _{s^{\prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime }\right)\right)+\sum _{s^{\prime }\in T_B}f\left(s^{\prime }\right)=\sum _{s^{\prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime }\right)\right)+\sum _{s^{\prime }\in T_B}\left(1-\sum _{s^{\prime \prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime \prime }\right)\right)\right)\cdot \frac{\mathsf{width}\left(\delta \left(s,s^{\prime }\right)\right)}{\sum _{s^{\prime \prime }\in T_B}\mathsf{width}\left(\delta \left(s,s^{\prime \prime }\right)\right)}=\sum _{s^{\prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime }\right)\right)+\left(1-\sum _{s^{\prime \prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime \prime }\right)\right)\right)\cdot \frac{\sum _{s^{\prime }\in T_B}\mathsf{width}\left(\delta \left(s,s^{\prime }\right)\right)}{\sum _{s^{\prime \prime }\in T_B}\mathsf{width}\left(\delta \left(s,s^{\prime \prime }\right)\right)}=\sum _{s^{\prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime }\right)\right)+\left(1-\sum _{s^{\prime \prime }\in T_B}\mathsf{left}\left(\delta \left(s,s^{\prime \prime }\right)\right)\right)=1.}$ Next, we show that support(a) = T_B. The fact that support(a)⊆T_B follows from left(δ(s, s′)) + f(s′) = 0, for each s′ ∈ S∖T_B, which was established in the previous paragraph. In order to establish that support(a)⊇T_B, we observe the following facts. For s′ ∈ T_B, we have (s, s′) ∈ B⊆E(s), and hence right(δ(s, s′)) > 0 from the definition of edges. Then if width(δ(s, s′)) = 0, from left(δ(s, s′)) = right(δ(s, s′)) we have left(δ(s, s′)) > 0. Instead, if width(δ(s, s′)) > 0, we have f(s′) > 0 from the choice of f. Hence, for each s′ ∈ T_B, we have left(δ(s, s′)) + f(s′) > 0, from which we obtain support(a)⊇T_B. We conclude that support(a) = T_B.

This concludes the (⇒) direction of the proof.

(⇐) Let a be an assignment for state s such that support(a) = T_B. Our aim is to show that B is valid, i.e., that B is large and realisable. From the definition of assignments, we have a(s′) ∈ δ(s, s′), and hence a(s′) ≤ right(δ(s, s′)) for each state s′ ∈ S. From this fact, and given that a is a distribution (i.e., sums to 1), we have: ${\displaystyle \sum _{\left(s,s^{\prime }\right)\in B}\mathsf{right}\left(\delta \left(s,s^{\prime }\right)\right)=\sum _{s^{\prime }\in \mathsf{support}\left(\mathbf{a}\right)}\mathsf{right}\left(\delta \left(s,s^{\prime }\right)\right)\ge \sum _{s^{\prime }\in \mathsf{support}\left(\mathbf{a}\right)}\mathbf{a}\left(s^{\prime }\right)=1.}$ In the case of ∑_(s,s′)∈Bright(δ(s, s′)) > 1, the fact that B is large follows immediately from the definition of largeness. In the case of ∑_(s,s′)∈Bright(δ(s, s′)) = 1, then a(s′) = right(δ(s, s′)) (i.e., a(s′) must be equal to the right endpoint of δ(s, s′)) for each (s, s′) ∈ B. Hence B⊆E^〈⋅,⋅], and as a consequence B is large. The fact that {(s, s′)∣s′ ∈ support(a)} = B implies that a(s′) = 0 for all (s, s′) ∈ E(s)∖B. Hence E(s)∖B⊆E^[0,⋅〉, and therefore B is realisable. Because B is large and realisable, B is valid.                         ■

A consequence of Lemma 1, together with Proposition 1 and the fact that we consider only well-formed IMCs, is that there exists at least one valid subset of outgoing edges from each state.

For each state s ∈ S, we let V alid(s) = {B⊆E(s)∣B is valid}. Note that |V alid(s)| = 2^|E(s)| − 1 in the worst case (when all edges in E(s) are associated with intervals [0, 1]). Given a valid set B ∈ V alid(s), we let V alidAssign(B) be the set of assignments a for s that witness Lemma 1, i.e., all assignments a for s such that {(s, s′)∣s′ ∈ support(a)} = B. Let V alid = ⋃_s∈SV alid(s) be the set of valid sets of the IMC. A witness assignment function w:V alid → Dist(S) assigns to each valid set B ∈ V alid an assignment from V alidAssign(B).

The qualitative MDP abstraction of O with respect to witness assignment function w is the MDP [O]_w = (S, Δ_w), where Δ_w is defined by Δ_w(s) = {w(B)∣B ∈ V alid(s)} for each state s ∈ S.

Computation of $S_{\forall }^{0,\mathrm{U}}$

The case for $S_{\forall }^{0,\mathrm{U}}$ is straightforward. We compute the complement of $S_{\forall }^{0,\mathrm{U}}$, namely the set of states for which there exists a DTMC in the UMC semantics such that T is reached with positive probability, written formally as $S\setminus S_{\forall }^{0,\mathrm{U}}=\left\{s\in S\mid \exists D\in {\left[O\right]}_{\mathrm{U}}.{\mathrm{Pr}}_s^D\left(\mathsf{Reach}\left(T\right)\right)>0\right\}$. The computation of this set reduces to reachability on the graph of the IMC according to the following lemma.

Hence the set $S_{\forall }^{0,\mathrm{U}}$ is equal to the complement of the set of states from which there exists a path reaching T in the graph of the IMC. Given that the graph of the IMC and the latter set of states can be computed in polynomial time, we conclude that $S_{\forall }^{0,\mathrm{U}}$ can be computed in polynomial time.

Computation of $S_{\exists }^{0,\mathrm{U}}$

We show that $S_{\exists }^{0,\mathrm{U}}$ can be obtained by computing, in the qualitative MDP abstraction [O]_w = (S, Δ_w) of O with respect to some (arbitrary) witness assignment function w, the set of states for which there exists a scheduler such that T is reached with probability 0.

To establish the correctness of this approach, we show a more general result: for λ ∈ {0, 1}, the set of states of [O]_w for which there exists a scheduler such that T is reached with probability λ is equal to the set of states of O for which there exists a DTMC in [O]_U such that T is reached with probability λ. The case in which λ = 0 will be used subsequently for the computation of $S_{\exists }^{0,\mathrm{U}}$, whereas the case in which λ = 1 will be used later in the article for the computation of $S_{\exists }^{1,\mathrm{U}}$.

In particular, Lemma 3 allows us to reduce the problem of computing $S_{\exists }^{0,\mathrm{U}}$ to that of computing the set $S_{\exists }^{0,w}=\left\{s\in S\mid \exists \sigma \in {\Sigma }^{{\left[O\right]}_w}.{\mathrm{Pr}}_s^{\sigma }\left(\mathsf{Reach}\left(T\right)\right)=0\right\}$ on [O]_w for an arbitrary witness assignment function w. As in the case of standard finite MDP techniques (see Forejt et al. (2011)), we proceed by computing the complement of this set, i.e., we compute the set $S\setminus S_{\exists }^{0,w}=\left\{s\in S\mid \forall \sigma \in {\Sigma }^{{\left[O\right]}_w}.{\mathrm{Pr}}_s^{\sigma }\left(\mathsf{Reach}\left(T\right)\right)>0\right\}$. For a state set X⊆S, let CPre(X) be the set of states for which there exists a distribution such that all states assigned positive probability by the distribution are in X, defined formally as CPre(X) = {s ∈ S∣∃μ ∈ Δ_w(s).support(μ)⊆X}. Furthermore, let $\overline{\mathsf{CPre}}\left(X\right)$ be set of states such that all available distributions make a transition to X with positive probability, defined formally as $\overline{\mathsf{CPre}}\left(X\right)=\left\{s\in S\mid \forall \mu \in {\Delta }_w\left(s\right).\mathsf{support}\left(\mu \right)\cap X\ne 0̸\right\}$. Note that $\overline{\mathsf{CPre}}$ is the dual of the CPre operator, i.e., $\overline{\mathsf{CPre}}\left(X\right)=S\setminus \mathsf{CPre}\left(S\setminus X\right)$. The standard algorithm for computing the set of states of a finite MDP for which all schedulers result in reaching a set T of target states with probability strictly greater than 0 operates in the following way: starting from X₀ = T, we let $X_{i+1}=X_i\cup \overline{\mathsf{CPre}}\left(X_i\right)$ for progressively larger values of i ≥ 0, until we reach a fixpoint (that is, until we obtain X_i^∗+1 = X_i^∗ for some i^∗). However, a direct application of this algorithm to [O]_w results in an exponential-time algorithm, given that the size of the transition function Δ_w of [O]_w is in general exponential in the size of O. For this reason, we propose an algorithm that operates directly on the IMC O, without requiring the explicit construction of [O]_w. We proceed by establishing that CPre can be implemented in polynomial time in the size of O.

Based on Lemma 4, instead of constructing and analysing the qualitative MDP abstraction [O]_w, we propose computing directly on O the sets X₀ = T and $X_{i+1}=X_i\cup \overline{\mathsf{CPre}}\left(X_i\right)$ for increasing indices i until a fixpoint is reached. Given that a fixpoint must be reached within |S| steps, and the fact that Lemma 4 specifies that CPre(S∖X_i), and hence $\overline{\mathsf{CPre}}\left(X_i\right)$, can be done in polynomial time in the size of O, we have that the set $\left\{s\in S\mid \forall \sigma \in {\Sigma }^{{\left[O\right]}_w}.{\mathrm{Pr}}_s^{\sigma }\left(\mathsf{Reach}\left(T\right)\right)>0\right\}$ can be computed in polynomial time in the size of O. The complement of this set is equal to $S_{\exists }^{0,\mathrm{U}}$, as established by Lemma 3, and hence we can compute $S_{\exists }^{0,\mathrm{U}}$ in polynomial time in the size of O.

Computation of $S_{\exists }^{1,\mathrm{U}}$

We proceed in a manner analogous to that for the case of $S_{\exists }^{0,\mathrm{U}}$. First note that, by Lemma 3, the set $S_{\exists }^{1,\mathrm{U}}$ is equal to the set of states of [O]_w for which there exists a scheduler that results in T being reached with probability 1, i.e., $S_{\exists }^{1,\mathrm{U}}=\left\{s\in S\mid \exists \sigma \in {\Sigma }^{{\left[O\right]}_w}.{\mathrm{Pr}}_s^{\sigma }\left(\mathsf{Reach}\left(T\right)\right)=1\right\}$. Hence our aim is to compute $\left\{s\in S\mid \exists \sigma \in {\Sigma }^{{\left[O\right]}_w}.{\mathrm{Pr}}_s^{\sigma }\left(\mathsf{Reach}\left(T\right)\right)=1\right\}$ on [O]_w. We recall the standard algorithm for the computation of this set on finite MDPs (de Alfaro, 1997; de Alfaro, 1999). Given state sets X, Y⊆S, we let ${\displaystyle \mathsf{APre}\left(Y,X\right)=\left\{s\in S\mid \exists \mu \in {\Delta }_w\left(s\right).\mathsf{support}\left(\mu \right)\subseteq Y\wedge \mathsf{support}\left(\mu \right)\cap X\ne 0̸\right\}.}$ That is, APre(Y, X) is the set of states for which there exists a distribution such that (a) all states assigned positive probability by the distribution are in Y and (b) there exists a state in X that is assigned positive probability by the distribution. The standard algorithm for computing the set of states for which there exists a scheduler that results in T being reached with probability 1 proceeds as follows. First we set Y₀ = S and $X_0^0=T$. Then sequence $X_0^0,X_1^0,\dots$ is computed by letting $X_{i_0+1}^0=X_{i_0}^0\cup \mathsf{APre}\left(Y_0,X_{i_0}^0\right)$ for progressively larger indices i₀ ≥ 0 until a fixpoint is obtained, that is, until we obtain $X_{i_0^{\ast }+1}^0=X_{i_0^{\ast }}^0$ for some $i_0^{\ast }$. Next we let $Y_1=X_{i_0^{\ast }}^0$, $X_0^1=T$ and compute $X_{i_1+1}^1=X_{i_1}^1\cup \mathsf{APre}\left(Y_1,X_{i_1}^1\right)$ for larger i₁ ≥ 0 until a fixpoint $X_{i_1^{\ast }}^1$ is obtained. Then we let $Y_2=X_{i_1^{\ast }}^1$ and $X_0^2=T$, and repeat the process. We terminate the algorithm when a fixpoint is reached in the sequence Y₀, Y₁, ….³ The algorithm requires at most |S|² calls to APre. In an analogous manner to CPre in the case of $S_{\exists }^{0,\mathrm{U}}$, we show that APre can be characterised by efficiently checkable conditions on O.