Formal modeling of a causal consistent distributed system and verification of its history via model checking using colored Petri net

Abstract model of DS

This article considers a simple abstract model of a DS and models it using the colored Petri net. Let us assume that our DS consists of a few systems, and a process runs on each. From now on, we use the term process to represent a system and the process that runs on it. Let us assume that DS comprises a group g of independent processes P_i (i = 1, 2…, N), and a communication channel exists between each pair of processes, as shown in Fig. 2 (Tanenbaum & Steen, 2006; Coulouris et al., 2011). The communication channel is a usual abstract concept in DSs and is assumed unidirectional. Channel Ch_ij is used to send messages of the process P_i to process P_j. Based on the definition of FIFO-ordered deliver property for communication channels (also known as FIFO-ordered channels), If process P_i sends two messages m₁ and then m₂ to process P_j (via channel Ch_ij), message m₁ will be delivered before message m₂.

Assumption 1: Let us assume that all communication channels have FIFO-ordered delivery property.

Let us assume that communication channels are reliable and no message will be lost, but the message delivery delay is not deterministic. Some algorithms for implementing reliable multicast have been developed (Coulouris et al., 2011). In a DS, when a process P_i writes value v on a replicated variable x, the write message is multicasted to replication group g of processes. If process P_i reads the value of the replicated variable x, no message multicasting is required. In this article, only a single group of processes is considered.

In a DS, each process has its local copy of shared variables. When a process P_i in a group g of processes writes value r on data item x, it multicasts this via sending messages to the group members in DS to notify them. $Multicas{t}_{P_i}\left(g,m\right)$ denotes the multicast of message m to the member of the group g by process P_i. When the sender process is clear, for simplicity, we denote $Multicast\left(g,m\right)$.

Concept of causal consistency

This part presents the concept and formal definition of CC. First, the concept of the happens-before relationship between two events is presented. Then, the concept of Fideg’s clocks and their properties are introduced. Then, the concept of CC and its formal definition are presented.

Algorithms for implementing CC in DS

Many researchers have studied the implementation of CC in DS. Birman & Joseph (1987) introduced the concept of causal ordering of messages. They presented the algorithm for implementing causal ordering of messages in DS at Cornell University in the ISIS system (Birman, Joseph & Schmuck, 1987). Their proposed method has the overhead of sending many messages. Schiper, Eggli & Sandoz (1989) presented an algorithm for implementing CC in DS that needs the transmission of fewer messages than the previous method. Their proposed method needs the attachment of the vector timestamp of the message’s sending event along with the content of the message. Their proposed method needs each process to keep a buffer named the ORD-Buff and send it via message to other processes. Raynal, Schiper & Toueg (1991) presented an algorithm for implementing CC requiring sending a two-dimensional matrix with each message. Each process has a vector and a two-dimensional matrix as local variables used by their proposed algorithm. Kshemkalyani & Singhal (1996, 1998) presented their proposed optimal space and time algorithm.

Birman, Schiper & Stephenson (1991) presented an algorithm for implementing the CC in DS taught in the textbook by Coulouris et al. (2011). We modeled this algorithm in the current article. This algorithm requires Fidge’s vector clock for logical time synchronization of processes and needs the concept of hold-back and delivery queues. The algorithm for implementing CC in DS is presented in Algorithm 1.

When a process P_i multicasts a message m to all other members of group g, it updates its vector clocks as V_i[i] = V_i[i] + 1 and then multicast message <V_i, m>. When a process P_j delivers a multicast message from the sender process P_i, the message is placed in the hold-back queue. Every message in the hold-back queue that meets the following conditions will be moved to the delivery queue.

$$\mathrm{\forall }k\in \left\{1,..,N\right\},k\ne i,V_j\left[k\right]\ge V_i\left[k\right].$$

Note: the condition $V_i^g\left[i\right]=V_j^g\left[i\right]+1$ in line 9 of the algorithm is not required if the communication channels support the FIFO-ordered delivery. This article assumes all channels have FIFO-delivery property via implementing one of the known algorithms for this abstraction.

Figure 3 shows delivery and hold-back queues. When a message enters the delivery queue, vector clocks of process P_j will be updated as V_j[j] = V_j[j]+1, which can be considered as the delivery of a message. Process P_j completely updates its vector clocks based on the Fidge algorithm when processing the next message from the delivery queue (Coulouris et al., 2011).

Figure 4 shows a DH of a CCDS. When the issued message m′ of process P₁ receives to process P₃, it will be inserted in the hold-back queue of process P₃ and stay there until it meets the required condition for transferring to the delivery queue of process P₃. This condition will be met when the issued message m by process P₂ is delivered to process P₃. When message m′ is received by process P₃, the vector clock of the message represents that some event happened in process P₂, but process P₃ is unaware of this event happening in P₂. Therefore, process P₃ postpones the delivery of message m′ issued by process P₁ until it delivers message m from process P₂.

In this article, a formal colored Petri net model of a CCDS that is composed of three processes is presented. The proposed hierarchical model was designed in two abstraction layers. A complete description of the proposed model is presented in the following sections. Let us assume that time synchronization of all replicas is performed by logical Fidge clocks (vector clocks). The proposed model’s input is a sample DH of a CCDS.

Case study 1. Table 1 shows a sample history of operations in a CCDS with three processes. The global sequence number represents the partial order of operations in DS. This article assumes that all state variables’ initial value is zero. The heading of the table’s rows represents the names of processes, and rows contain the operations of each process. At the first row of the table, W(x):1 denotes that process P₁ writes value one on the replicated data-item x at global sequence order 1.

The write operation of process P₂ (W(y):2) has a causal relationship with the write operation of process P₁ (W(x):1). Let us consider the worst-case scenario in which process P₂ has instruction y = x * 2. In this case, the written value of the variable y has been influenced by the read value of the variable x. Process P₃ does not understand this relationship correctly. The first read of P₃ (R(y):2) sees value two, and the second read of this process (R(x):0) sees value zero, then we are faced with a DS that does not support CC.

Petri net is a formal method that is designed to model concurrent systems. Various extensions of the classical Petri net have been proposed. One of the most successful is the colored Petri net, which can model synchronous and asynchronous communications (Jensen & Kristensen, 2009). The modeling power of colored Petri net is extensively extended by (1) using data type (color set) for tokens and (2) using machine learning (ML) artificial language that is a formal language based on the Lambda calculus that preserved the formal property of Petri net (Paulson, 1996). One of the best tools for modeling by colored Petri net is the CPN tool used in this article (Beaudouin-Lafon et al., 2001).

Color sets of the model

The proposed colored Petri net model in this article is parametric. Two important parameters are the number of constituting processes in the distributed system’s model and the number of data items that processes update and read them. The following two constants determine the number of processes and the number of variables in the model.

val NoProcs = 3;

val NoVars = 2;

Definitions of color sets that are used in the model and represent data types of places and tokens are as follows:

colset PROCESS = index proc with 1..NoProcs;

colset CLOCKSVECTOR = list INT;

colset STATEVARIABLE = record varName: STRING * value: INT;

colset LOCALSTATE = list STATEVARIABLE;

colset OPERATIONTYPE = with R | W;

colset OPERATION = record process: STRING * opr: OPERATIONTYPE * varName: STRING * value:INT * gSeqNo:INT;

colset OPERATIONS = list OPERATION;

colset MESSAGE = record varName: STRING * value:INT * vector: CLOCKSVECTOR * sender: PROCESS;

colset MESSAGELIST = list MESSAGE;

colset SENDMESSAGE = record process: PROCESS * msgs: MESSAGELIST;

Color set PROCESS is of type index and is defined to represent a process identity. Color set CLOCKSVECTOR is a list of integer numbers used to represent a process’s Fidge vector clocks. Color set STATEVARIBLE is a record with fields: (1) varName that is of type string to represent the name of a variable and (2) value that is of type int and represents the value of a variable. Color set LOCALSTATE is a list of type STATEVARIABLE and is defined for representing values of variables of a process that may be replicated between processes. Color set OPERATIONTYPE is an enumerated type defined to denote read and write operations on variables. Color set OPERATION is a record with five fields. The first field with the name process represents the name of the process that will execute this operation. The second field, opr, represents the type of operation (read/write). The third field, named varName, represents the variable name on which the operation will be performed. The fourth field with name value represents the variable’s value in case of read operation or value that must be stored in the variable in case of write operation. The last field, named gSeqNo, represents this operation’s global sequence number. Color set OPERATIONS is a list of type OPERATION used to represent process operations.

Operations on the state variables of a process will be stored according to the ascending value of the global state number of operations. Color set MESSAGE is a record with four fields: the first field, named varName, represents the variable’s name. The second field, named value, represents the variable’s value, the third field, named vector, represents the Fidge vector clocks of the message issuer process, and the last field, named sender, represents the message issuer process. Color set MESSAGELIST is a list of types of MESSAGE. Color set SENDMESSAGE is a record with two fields. The first field, named process represents the message issuer process, and the second field, named msgs is the list of issued messages by this process.

Proposed colored Petri net model of system

This article’s proposed CCDS model is hierarchical in two abstraction layers. The top-level model of the proposed system model with initial markings of case study 2 is shown in Fig. 5.

Operations of process P_i is stored in place OperationsPi with color set OPERATIONS and values of its state variables is stored in place LocalMemoryPi with color set LOCALSTATE and process identity is stored in place Pri with color set PROCESS. Place GlobalStep with color set int represents the global sequence number of the system with initial value one. Place outbox with color set SENDMESSAGE represents issued messages of processes and place inBox with color set INCOMEMESSAGE represents incoming messages of all processes that will be stored in their hold-back queues. Place inBox in the top-level model of Fig. 5 plays the rule of the hold-back queue for all processes. The marking of this place is a list in which each element represents a list of messages stored in the hold-back queue of each process in CCDS.

Transition StepUpdate is responsible for increasing the global sequence number of the model. The transition network plays the role of communication networks and channels in DS. Substitution transition Ri represents operations of process Pi, an instance of the Replica module. Colored Petri net model of the Replica module of substitution transition of process P₂ is shown in Fig. 6. Place LocalMemory is an I/O port related to the place LocalMemoryPi of process P_i in the top-level model and is related to socket OperationsPi of process P_i.

I/O port OutBox is related to socket outBox, I/O port InBox is related to socket inBox, and I/O port StepNumber is related to socket GlobalStep in the top-level model of Fig. 5. I/O port ProcessNumber is related to the place Pri of process P_i in the top-level model of the system. Place Clocks with color set CLOCKSVECTOR is an internal place and represents Fidge vector clocks of a process. Transition ExecuteOperation picks up an operation from the beginning of the operations list of a process on each fire and executes it. Transition PickupMessage picks up a message from one of the channels in each firing. Initial markings of Figs. 5 and 6 are based on the DH of Table 1. Operations of process P₂ are defined using constant P2Operations based on Table 1, and they are used as initial marking of place OperationsP2 in the model of Fig. 5.

val P2Operations = [{process="p2", opr=R, varName="x", value=1, gSeqNo=2}, {process="p2", opr=W, varName ="x", value=2, gSeqNo=3}]

The proposed model has two important parameters, NoVars and NoProcs. NoVars determines the number of variables (data items) of the input DH and the model. NoProcs determines the number of processes in the input DH and the model. These variables are used in the model’s functions to increase scalability.

Functions of the proposed model

This part of the article presents the functions developed and used in the proposed model. Figure 7 displays the structure chart of the proposed model’s functions. The system’s top-level model uses nine functions displayed in the figure’s top left corner.

The signature and description of all nine functions used in the proposed model’s top-level module are presented in Table 2. Row 5 to 9 in Table 2 provide the fold technique that permits the variable number of input parameters of functions that standard ML and CPN ML do not support.

Function appendMessage is used in the output arc of the transition network in the top-level model of the proposed system. The function stepUpEn is used as a guard condition for transition StepUpdate. Global sequence number zero represents the unpermitted state of the system. It denotes a true deadlock of the model and disables all transitions. Value zero will be generated when a process executes a read operation based on the history, and the value of the local memory of a process is not the same as the expected value that is indicated in history. It means that the history of DS is not reachable by CCDS, proving that the history is not valid. Therefore, after the proof is completed, the execution of the model will be stopped to overcome the state space explosion problem.

The proposed model assumed that the maximum number of operations of each process is limited to up to 100 operations, and operations of each process are stored based on the ascending order of their gSeqNo field of operations that represents the global sequence number of each operation. If the value of the global sequence number is zero or all operations of all processes have been executed (therefore, the list of their operations is empty), function stepUpEn returns the false value as a result. As far as a process has an unexecuted operation in that its sequence number is equal to the global sequence number of the system, the global sequence number of the model will not increase. It means that as long as a process has an unexecuted operation with a low sequence number, an operation with a higher sequence number of no process can execute. The initial value of the global sequence number of the model is one, and operations of all processes start with a sequence number one.

The system’s second-level module of the model (replica module) uses 13 functions displayed on the right side of Fig. 7. The signature and description of all 13 functions used in the proposed model’s top-level module are presented in Table 3. The arrow from one function to another represents it calls the function located in the destination of the arrow. Two functions in the low right corner of Fig. 7 are not directly used in the module replica.

The function checkRead is used as a guard condition of transition ExecuteOperation in the low level of the proposed model in Fig. 6. Based on the distributed protocol of implementing CC, if items of vector clocks of the issuer process of the message (except the item that represents the index of issuer process) are greater than (be fresher than) items of vector clocks of delivering process, it means that information of delivering process about other processes beyond the issuer one is not up-to-date. It means there is at least a message that the delivery process is unaware of. In this case, the function returns the false Boolean value. The isPickupEnabled function is used as a guard condition of transition pickupMessage in the low-level model of Fig. 6. This function searches the hold-back queue of a process and, if it finds any message that meets the delivery condition, returns (1) the true Boolean value, (2) the index of the message in the delivery queue, and (3) that message as a result. The function pickupMsg is used on the output arc of transition pickupMessage that is connected to place InBox in the low-level model of Fig. 6. After enabling this transition by calling the guard function isPickupEnabeled.

The function isPickupEnabeled returns the first message of the delivery queue and its index. This index is used as the third input parameter in calling function pickupMsg. The first input parameter of this function is the list of messages of the hold-back queue, and the second input parameter is the identity of the process that picks up a message for processing. Each time this function is called, it picks up a message from the head of the current process delivery queue and returns the updated list of messages after this pick-up. Each process sends its updated vector clock and a message to the receiver. The process that delivers the message updates its vector clock by information of the vector clock of the incoming message based on the Fidge algorithm. This work is accomplished by calling the function updateClockByMsg.

The function check is used as a guard condition of transition ExecuteOperation in the low-level model of Fig. 5. This guard condition means that until any operation exists with a low sequential number, the model will never execute an operation with a greater sequential number of any other process. The function sendMessage is used in the inscription of the output arc of transition ExecuteOperation. If the operation is read, this transition does not add any message to the end of the list of issued messages of a process held in place OutBox and returns an unchanged list to this place. If the operation is a write operation, a new message is added to the end of the issued messages list of place OutBox.

The next section of the article discusses the analysis of the state space graph of the proposed method. The model-checking approach, which is presented in the next section, also can be considered a minor part of the materials and method. The proposed hierarchical model of the CCDS is designed to yield intended results with simple model-checking, and the heaviest part of the work has been accomplished in the modeling part.

Case studies and analysis of the state space graphs

Report 1 shows the summary report of state space generation of the proposed model of CCDS by the input DH of case study 1, which is presented in Table 1. The CPN tool automatically generates the model’s SSG. The status of the state space graph is full, and the CPN tool has completed the generation of the state space graph in less than a second. The state space and strongly connected component (SCC) graphs have 46 nodes and 72 edges. The state space graph has eight dead markings; some presented in the report.

Report 1: Summary report of state space of proposed model based on the input history of case study 1.

This article presents two customized proposed functions for analyzing the state space graph. SSG’s final (dead) states can be divided into two groups: (1) Dead states’ markings in which the global sequence counter’s value is negative. These states are not possible by the casually consistent distributed system. The absolute value of this number denotes the global step number of input DH that is not valid with CCDS. (2) Dead states that the global sequence counter place marking is bigger than zero in them. These states are reachable in causally consistent distributed systems.

Table 4 briefly describes the four developed functions for customized model checking (analysis) of the state space graph.

The first function is hasAnswer, and its signature is as follows: fun hasAnswer(): BOOL. This function does not have an input parameter and returns a Boolean result. It calls a predefined state space analysis function of CPN tools to get the list of all dead states of SSG. Then, it searches the states in the list that the global sequence number of them is non-zero. It does so by looking at the marking of the place GlobalStep if the top-level module of the proposed system model is shown in Fig. 5. If any such state exists, it returns true value. Figure 8 shows the results of running codes for analysis of the proposed model after the generation of the SSG by considering the input DH of case study 1. Calling the CPN tools’s built-in function ListDeadMarkings() returns the list of dead states of the SSG. Calling the function List.length(DeadMarkings) returns the number of dead states in the SSG. Our proposed function hasAnswer() returns the value false as a result and shows that this input DH of case study 1, which is shown in Table 1, is not a valid image of a CCDS. The function getErrorStep() returns the global step number of invalid input DH, step 5 in case study 1. Calling the function proposeCorrection() shows the first required modification such that the input DH of case study 1 becomes a valid history of a CCDS. Input DH may require more corrections, and the function proposeCorrection only proposes the first modification. This function proposes that the result of the read operation of process three at global step number 5 on the variable must return one instead of 0 in Table 1.

Case study 2. Let us assume that the operations of processes in a DH are as shown in Table 5.

Report 2 shows the summary report of the state space graph of the case study 2. The status of the state space graph generation is full, has 27,434 nodes, and has 60,915 arcs. It was generated in 43 s. Case study 2 has a bigger state space graph than case study 1. Figure 9 shows the results of analyzing the state space graph. The call of function hasAnswer returns the true value, which means that the given DH is a valid history of a CCDS.

Our second developed function for state space analysis is the function shortestAnswer. Prototype of it is: fun shortestAnswer(): INT * INT. This function will be called if the result of the calling function hasAnswer be true. This function investigates only terminal (dead) nodes of the SSG that global sequence number of the model is not zero for them. This function returns (1) a node number from investigated nodes whose distance from the starting node of SSG (node no. 1) is less than the other nodes and (2) the length of this path. This function finds the shortest proof scenario for a given distributed snapshot by CCDS.

The result of calling function shortestAnswer for the distributed snapshot of case study 2 is shown in Fig. 9. The path’s sequence of nodes’ numbers yields by calling this function. The sequence of events can be extracted by investigating the edges of this path. Analysis shows that the shortest proof scenario for generating the DH of case study 2 comprises 31 nodes and requires 30 transition firings.

Report 2: Summary report of state space of proposed model based on the input history of case study 2.

Passing through 31 nodes starting from the system’s initial state makes it possible to reach a dead state that completely yields the DH of case study 2. The nodes’ numbers of this path are obtained by calling the code: if isCCDH = true then NodesInPath(1, #1 (ShortestPath)) else []. Thirty edges of this path represent the sequence of events that produce DH of case study 2. Figures 10 and 11 show part of the SSG of case study 2 that presents the nodes of the automatically extracted shortest path. Some parts of the arcs’ information, which is crucial for visualization of the proof scenario, are shown in these figures.

Figure 12 shows the manually generated proof scenario from extracted information of the state space graphs’ analysis. We visualized the extracted proof scenario for the validity of case study 2 input DH via computation of the shortest path, as is shown in Fig. 12. Values of the Fidge’s clocks of the events are shown in this figure.

Case study 3. Let us assume the operations of processes in a DS is as shown in Table 6.

Case study 3 comprises four processes compared to case studies 1 and 2, which contain only three processes. This case study comprises four variables compared to case Study 1, which contains only two variables, and case study 2, which contains three. This case study demonstrates and evaluates the proposed model’s scalability. The proposed model is hierarchical, and increasing the number of processes requires increasing the instance of module “replica” in the top-level module and adding required arcs. The state space summary report of the model based on the input history of case study 3 is shown in report 3. The SSG of this case study has 23,081 nodes and 62,140 arcs. This graph is much bigger than the SSG of previous case studies. The graph was generated in 77 s, and the model has 5,062 dead markings.

Report 3: Summary report of state space of proposed model based on the input history of case study 3.

Figure 13 shows the result of the state space analysis. Call of function hasAnswer on the state space graph of case study 3 returns false, representing that the DH of case study 3 is invalid under CCDS. The result of the calling function proposeCorrection proposes modifying the value in DH. State space analysis shows that at step 10, the result of the read operation of process four on variable w must be 6, but in Table 6, the value zero is denoted.

Case study 4. Let us assume that the operations of processes in a DH are as shown in Table 7.

DH of case study 4 is composed of five processes and five variables. Report 4 shows the summary report of the state space graph of the case study 4. The state space graph generation status is full, has been completed in 15 s, and has 17,867 nodes and 43,886 arcs. Case study 4 demonstrates the scalability of the proposed model.

Figure 14 shows the results of analyzing the state space graph. The call of function hasAnswer returns the true value, which means that the given DH is a valid history of a CCDS. Analysis of the state space graph by calling the function shortestAnswer shows that the shortest proof scenario for generating the DH of case study 4 comprises 55 nodes and requires 54 transition firings. Figure 14 displays the sequence of state space nodes that generate the proof scenario of input DH in case study 4.

Report 4: Summary report of state space of proposed model based on the input history of case study 4.

Scalability of the proposed model

One important issue regarding the proposed model is its scalability. How much scalable is the proposed model? The scalability of the model can be studied from the following viewpoints: (1) scalability regarding the number of variables (data items) of the input DH, (2) scalability regarding the number of the constituting processes, and (3) scalability of model checking the state space graph of the model.

The constant model parameter NoVars defines the number of variables (data items) used by the input DH and, accordingly, in the model. In case studies one up to 4, the value of this parameter was 2, 3, 4 and 5, respectively. The proposed model has been investigated regarding the value of this parameter. Results showed that the proposed model is designed such that the value of this parameter does not affect the size of the model’s state space graph. This means that the proposed model is highly scalable in terms of the number of variables in input DH. Increasing the number of variables increased the state space generation time a little. It increases the generation of the state space graph because of the increase in the size of the color sets, such as the list that we used to store values of local copies of the variables in each process and linear searches in these lists. The time of generation of the state space graph has an O(1) relationship regarding the number of variables in the input DH and the model. The model parameter NoVars is used in the functions findVariableIndex, checkRead, updateLocalState, and applyIncomeMsg and initializeClocks used in the Replica module, as shown in Figs. 6 and 7.

The constant model parameter NoProcs defines the number of constituting processes used by the input DH and, accordingly, in the model. The proposed model is designed to be scalable in terms of the number of constituting processes. Although the scalability regarding the number of processes is not only via changing the initial marking of the proposed model, a minor modification in the top-level module is also required. The proposed model has a hierarchical design, which helps to increase the number of constituting processes by increasing the substitution transition of the Replica module. Case studies 2 and 3 are designed with three processes, case study 3 with four, and case study 4 with five processes. Figure 5 shows the top-level module of case study 1, which contains three processes. The guard condition of the transition StepUpdate in Fig. 5 requires a variable number of arguments regarding the scalability issue of the proposed model that is not supported in standard ML and CPN ML. Fold structure constituting two functions fold and step0, along with two functions $ and a have been defined to support the variable number of parameters for calling the function minHdStep. All of these functions are visible in the structure chart of Fig. 7. This technique increases the proposed model’s scalability regarding the number of constituting processes. Increasing the number of processes increases the size of the model’s state space graph and its generation time.

Write operations will be groupcasted to other model processes, and considering the variable delay of message transmissions increases the size of the model’s state space graph. Read operations only increase the size of the state space graph by a small amount. By increasing the size of the state space graph, it is evident that the processing time of some of the model checkings (analysis) of this graph will be increased. The processing order of some model checkings is in quadratic order, based on the number of nodes and/or arcs of the state space graph. Therefore, increasing the state space graph’s size will increase the time of model-checking.

Different approaches can be used to alleviate this limitation. The first approach is considering reasonable assumptions in the model. We can consider a limit to the delay in message transmission. This limiting assumption decreases the size of the state space graph of the model. The second approach is changing the CPN tools software so that we can combine the generation of the state space graph and do model chacking concurrently. Model checking is usually done after the generation of the state space graph of the proposed model. Doing model checking simultaneously with generating the state space graph significantly decreases the analysis time. When we find that this input DH is not valid during the generation of the state space graph, we can stop the generation of the remaining part and report the evidence.

Some analyses do not require that we backtrack the traversed path of the state space graph. As the third approach, if the current node of the state space graph is valid, we are not required to keep up with the previous nodes. This approach decreases the processing time and memory usage. However, this approach is not applicable in all types of analysis but in some applications considering the appropriate revision of the proposed model to support it. Applying the second and third approaches requires access to the CPN tools’ source code and changing it to support them.

Limitations of the proposed model

The proposed model is designed to be as simple as possible. The main reason is avoidance of the state space explosion problem. The most important limitations of the proposed model are as follows: (1) the read/write operations of the input DH have a simple style and must be divided into separate steps to make its process easier. (2) The user needs to initialize the model’s parameters NoVars and NoProcs, according to the input DH. (3) Increasing the number of processes in the model requires adding substitution transition of module “Replica,” adding the required arcs, and revising the guard condition of the transition StepUpdate in the top-level module shown in Fig. 5. The authors used the fold techniques to support variable input parameters to function minHdStep in the guard condition of this transition. (4) By increasing the number of processes and the write operations in the input DH, the size of the state space graph will increase extensively. This phenomenon is expected and is the inherent behavior of the formal methods based on generating the state space graph. One of the best approaches to deal with this problem is combining the process of generating the state space graph and concurrently doing our customized model checking. (5) The analysis of the state space graph generates the proof scenario if the input DH is valid under CCDS. The developed function extracts the proof scenario automatically via customized model checking, but it has a lot of detailed information that requires deep familiarity with color sets and marking of the state space nodes. Automatic generation of a message sequence chart helps immensely in understanding the proof scenario in a shorter time without the requirement of understanding the details of the model design. (6) It is possible to design the model such that limitation three, mentioned before, has been eliminated. It causes more scalability of the model, such that to increase the number of processes, the model’s top-level module will not be required to be modified. Increasing the number of processes can be accomplished only by modification of the input markings related to the input DH. However, developing such a model has some issues: (1) the complexity of the model’s color sets increases extensively and understanding the state space nodes’ marking becomes more complicated. (2) All model functions and state space analysis must be revised accordingly. (3) The processing time of the state space generation increases. Nevertheless, it causes a decrease in the number of palaces in the model and, therefore, decreases the size of the state space graph.