FoBSim: an extensible open-source simulation tool for integrated fog-blockchain systems

FC simulation tools

Recently, our research group has started to investigate the state-of-the-art related to cloud, IoT and fog simulation tools in Markus & Kertesz (2020). Within this study, several simulation tools were classified, compared, and analyzed, such as the DockerSim tool (Nikdel, Gao & Neville, 2017), FogNetSim++ (Qayyum et al., 2018), and EdgeCloudSim (Sonmez, Ozgovde & Ersoy, 2018). Furthermore, technical details, advantages, vulnerabilities, and software quality issues were also discussed.

Rahman et al. (2019) surveyed 15 simulation tools for cloud and data centers networks scenarios. The tools were discussed and compared according to several criteria, such as the Graphical User Interface (GUI) availability, the language with which the simulator was implemented, and the communications model. Consequently, they proposed the Nutshell tool which addresses some drawbacks that were ignored by most of the surveyed simulators. For example, most surveyed simulators had abstract network implementation and low-level details were missing. Further, non of the studied tools provided an addressing scheme, a congestion control mechanism, or a traffic pattern recognition mechanism. Out of those 15 presented simulation tools, seven were defined as extensions of the CloudSim toolkit (Calheiros et al., 2011).

Yousefpour et al. (2019) presented a complete survey about FC, referencing 450 publications specifically concerned with FC development and applications. Within their extended survey, some FC simulation tools, such as iFogSim (Gupta et al., 2017; Naas et al., 2018), Emufog (Mayer et al., 2017), Fogbed (Coutinho et al., 2018), and MyiFogSim (Lopes et al., 2017) were discussed. As iFogSim was conceptually built using the CloudSim communications model, it inherited some of its properties, such as the ability to co-execute multiple tasks at the same time and the availability of plugable resource management policies.

Generally speaking, any cloud simulation tool can be extended to be a fog-enabled simulation tool. This is because of the fundamental property of the fog layer acting as a bridge between end-users and the cloud. In other words, adding a fog module to a cloud simulation tool, describing communications, roles, services, and parameters of fog nodes, is sufficient to claim that the tool is a fog-enhanced cloud simulation tool. Additionally, in a project that targets a Fog-BC integration applications, many researchers used a reliable, general-purpose fog simulator and implemented the BC as if it was an application case, such as in Kumar et al. (2020). The results of such simulation approach can be trusted valid for limited cases, such as providing a proof of concept of the proposal. However, critical issues, such as scalability and heterogeneity in huge networks, need to be simulated in a more specialized simulation environments. To mention one critical case, the BC protocols deployed in different CAs require more precise and accurate deployment of the BC entities and inter-operation in different layers of a Fog-enhanced IoT-Cloud paradigm. Consequently, as some simulation scenarios need an event-driven implementation, while others need a data-driven implementation, a scenario’s outputs may differ when simulated using different simulation environments. Such possibility of fluctuated simulation outputs should normally lead to unreliable simulation results.

BC simulation tools

As we have previously investigated how a Fog-Blockchain integration is envisioned, we started the implementation of FoBSim with a simple BC simulation tool described in Baniata (2020). Consequently, we discuss the state of the art regarding BC simulation tools available in the literature. In later sections, we describe how FoBSim serves as a reliable tool to mimic an FC-BC integration scenario.

Anilkumar et al. (2019) have compared different available simulation platforms specifically mimicking the Ethereum BC, namely Remix Ethereum (Ethereum, 2020), Truffle Suite (Truffle Blockchain Group, 2020), Mist (Bahga & Madisetti, 2017), and Geth (Bruno, 2018). The comparison includes some guidelines and properties such as the initialization and the ease of deployment. The authors concluded that Truffle Suite is ideal for testing and development, Remix is ideal for compilation and error detection and correction, while Mist and Geth are relatively easy to deploy. Alharby & Van Moorsel (2019) and Faria & Correia (2019) proposed a somewhat limited simulation tool, namely BlockSim, implemented in Python, which specifically deploys the PoW algorithm to mimic the Bitcoin and Ethereum systems. Similarly, Wang et al. (2018) proposed a simulation model to evaluate what is named Quality of Blockchain (QoB). The proposed model targets only the PoW-based systems aiming to evaluate the effect on changing different parameters of the simulated scenarios on the QoB. For example, average block size, number of TXs per block/day, the size of the memPool, etc. affecting the latency measurements. Furthermore, the authors identified five main characteristics that must be available in any BC simulation tool, namely the ability to scale through time, broadcast and multi-cast messages through the network, be Event-Driven, so that miners can act on received messages while working on other BC-related tasks, process messages in parallel, and handle concurrency issues.

Gervais et al. (2016) analyzed some of the probable attacks and vulnerabilities of PoW-based BCs through emulating the conditions in such systems. Sub-consequently, they categorized the parameters affecting the emulation into consensus-related, such as block distribution time, mining power, and the distribution of the miners, and network-related parameters, such as the block size distribution, the number of reachable network nodes, and the distribution of those nodes. They basically presented a quantitative framework to objectively compare PoW-based BCs rather than providing a general-purpose simulation tool.

Memon et al. (2018) simulated the mining process in PoW-based BCs using the Queuing Theory, aiming to provide statistics on those, and similar systems. Zhao, Guo & Chan (2020) simulated a BC system for specifically validating their proposed Proof-of-Generation (PoG) algorithm. Hence, the implementation objective was comparing the PoG with other CAs such as PoW and PoS. Another limited BC implementation was proposed by Piriou & Dumas (2018), where only the blocks appending and broadcasting aspects are considered. The tool was implemented using Python, and it aimed at performing Monte Carlo simulations to obtain probabilistic results on consistency and the ability to discard double-spending attacks of BC protocols. In Deshpande, Nasirifard & Jacobsen (2018), the eVIBES simulation was presented, which is a configurable simulation framework for gaining empirical insights into the dynamic properties of PoW-based Ethereum BCs. However, the PoW computations are excluded in eVIBES, and the last updates on the code were committed in 2018.

To highlight the comparison between the mentioned BC simulation tools and our proposed FoBSim tool, we gathered the differences in Table 2. PL, PoW, PoS, PoA, SC, DM, PM, IDM, and F are abbreviations for Programming Language, Proof-of-Work, Proof-of-Stake, Proof-of-Authority, Smart Contracts, Data Management, Payment Management, Identity Management, and Fog-enhanced, respectively. As shown in the table, none of the previously proposed BC simulation tools makes the PoA algorithm available for simulation scenarios, provides a suitable simulation environment for identity management applications, or, most importantly, facilitates the integration of FC in a BC application.

Many other references can be found in the literature, in which a part of a BC system, or a specific mechanism is implemented. The simulated “part” is only used to analyze a specific property in strict conditions, or to validate a proposed technique or mechanism under named and biased circumstances, such as in Wang et al. (2020) and Raman et al. (2019). It is also worth mentioning here that some open-source BC projects are available and can be used to simulate BC scenarios. For example, the HyperLedger (The Linux Foundation, 2020) projects administered by the Linux Foundation are highly sophisticated and well-implemented BC systems. One can locally clone any project that suits the application needs and construct a local network. However, those projects are not targeting the simulation purposes as much as providing realized BC services for the industrial projects. Additionally, most of these projects, such as Indy, are hard to re-configure and, if re-configured, very sensitive to small changes in their code. Indy, for example, uses specifically a modified version of PBFT CA, namely Plenum, while Fabric uses RAFT.

Consensus algorithms

Several approaches were proposed as a solution for the aforementioned needs, among which are the most famous PoW CA. PoW was deployed in 2009 in the first BC system, i.e., Bitcoin (Nakamoto, 2019), and is currently used in other robust BC systems; such as Ethereum (Vujičic, Jagodić & Ranđić, 2018). Although PoW methods have proven strong security and support to BC systems, they have some drawbacks, such as high energy consumption and high latency, that encouraged the R&D communities to search for other trusted methods.

The PoS algorithm (King & Nadal, 2012) was proposed a couple of years later in order to solve the high energy consumption problem implied by PoW. PoS is currently being optimized to provide similar advantages as PoW. Ethereum, for example, is planning to substitute PoW with PoS in the very near future. However, some drawbacks of PoS need to be solved before its official deployment, such as The Monopoly Problem (Larimer, 2013), The Bribe Attack (Bentov, Gabizon & Mizrahi, 2016; Deirmentzoglou, Papakyriakopoulos & Patsakis, 2019), and relatively low reliability (Zhang & Chan, 2020).

In PoW-based BCs, a BC node proves the validity of its generated block of data by coupling a puzzle solution within the block. The puzzle solution is generally characterized by hardship to be obtained while it can easily be validated once found. Generally, the puzzle is a mathematical problem that requires high computational power to be obtained. In PoS-based BCs, the BC node that is allowed to generate the next block is chosen randomly by the system. To encourage the system to pick a specific BC node, staking more digital coins in deposit shall increase the probability of being chosen. This provides high trust measures as faulty generated blocks are not tolerated by the system, and the staked coins of the malicious/faulty BC node would be burned as a penalty.

Other approaches were proposed that provide trust in BCs. Examples include the PoET (Buntinx, 2017), and the PoA (Avasthi & Saxena, 2018). PoET-based BCs generate randomly selected times for BC nodes. The one node whose randomly picked time elapses first, is the one who is granted the opportunity to generate the next block. PoA, on the other hand, implies that only blocks signed by authorized members are validated and confirmed by the BC network. Those authorized nodes must be known trusted participants that can be tracked and penalized in case of faulty behavior. Both of these CAs share the property of being suitable for private and permissioned BCs, while PoW and PoS are known for being suitable for public and permissionless BCs.

FoBSim allows to choose the suitable CA according to the simulated scenario. While there are many versions of each CA mentioned, we currently provide the simplest version of each so that modifications can be performed with no complexities. To obtain more information about them, more details can be found at Sheikh (2018), Singh et al. (2019), and Chen et al. (2017).

Transactions

In a very simple scenario, an end-user sends a request to the BC network, which consists of BC nodes, to perform a defined TX. As stated at the beginning of this section, TXs may be data to be stored (i.e., payment data, reputation data, identity data, etc.), or can be SCs whose results can be either saved in a centralized (in the case of Cloud) or distributed manner (in the cases of fog or BC). Once the TX is performed, it should be agreed on by the majority of BC nodes if to be saved on the distributed ledger and, sub-consequently, be added to the chain saved in all BC nodes.

On the other hand, if the fog layer is controlling and automating the communications between the end-user layer and the BC network, as in Baniata & Kertész (2020), the TXs are sent from end-users to the fog. After that, some communication takes place between the fog layer and the BC network in order to successfully perform the tasks requested by end-users. In such system model, we assume that the BC network lays in a different layer than the fog layer. The case where the BC network is placed in the fog layer is covered in “Functionality of the BC Deployment”. Nevertheless, a feedback with the appropriate result of each TX should be easily achievable by end-users.

Distributed ledger

In the case were data needs to be stored in a decentralized manner, no Trusted Third Party (TTP) needs to be included in the storing process. The entity considered as a TTP in regular fog-enhanced cloud systems is the cloud, where data is stored. However, computations can take place in the fog layer to enhance the QoS.

Within DLT-enabled systems, such as BC, groups of data are accumulated in blocks, and coupled with a proof of validity, as explained in “Consensus Algorithms”. Once a new block of TXs is generated, and the proof is coupled with them, the new block is broadcast among all BC nodes. Nodes who receive the new blocks verify the proof and the data within each TX, and if everything is confirmed valid, the new block is added to the local chain. With each BC node behaving this way, the new block is added to the chain in a distributed manner. That is, a copy of the same chain, with the same exact order of blocks, exists in each BC node. Further, a hash of the previous block is added to the new block, so that any alteration attack of this block in the future will be impractical, and hence almost impossible.

Functionality of the BC deployment

As a BC-assisted FC system can provide computational and storage services, the BC placement within the FC architecture may differ. That is, BC can be placed in the fog layer, the end-user layer, or the cloud layer. In FoBSim, however, we consider only the first two mentioned placement cases.

When the BC is deployed in the fog layer, storage and computational services are performed by the fog nodes themselves. In other words, fog nodes wear a second hat, which is a BC network hat. Thus, when storage to be provided by the fog while fog nodes are also BC nodes, data is stored in all fog nodes in the fog layer. A simple system model is demonstrated in Fig. 3A, where only one chain is constructed in the lower fog layer and one fog control point in the upper layer monitors the BC functionality. However, such a model is not practical and more complexities appear in a real-life scenario, including heterogeneous fog nodes, multiple BC deployments, different CAs, and different service models. In such complex systems, FoBSim can be easily extended by adding the needed classes and modules and, hence, cover necessary proposed scenario entities. A note is worth underlining here is the importance of differentiating between the services provided by fog nodes which are BC nodes, and the services provided by fog nodes which are not BC nodes. The first type gets incentivized by end-users for providing both fog services and BC services, while the second type gets incentivized by end-users for providing only fog services. Such critical issues need to be taken care of, when simulating Fog-BC scenarios, to maximize the reliability of the obtained results.

In a system model where the BC is deployed in the end-user layer, we can distinguish two types of end-users; namely task requester and BC node. In a fog-enhanced BC system, the fog controls the communication between the two types of end-users. Specifically, BC nodes perform the tasks that were sent to the BC network by the fog, which originally were requested by task requester end-users. Further, the fog can control the privacy preservation of data and incentivize BC nodes in the form of digital currency, as in Baniata, Anaqreh & Kertesz (2021). To be specific, BC nodes can be further sub-categorized according to the scenario to be simulated. Adding other types of BC nodes is up to the developers and the system model. For example, the Bitcoin system is modeled in a simpler way, where BC is directly connected to task requester end-users, and it only provides a payment ledger service. Ethereum, on the other hand, provides computational and data management services. This makes Ethereum surpass Bitcoin because it can provide more services to end-users. FoBSim improves both system models by optionally adding the fog layer. The system model provided by FoBSim when the BC is deployed in the end-user layer is demonstrated in Fig. 3B.

FoBSim modules

To facilitate the understanding of FoBSim, we demonstrate the methods within each FoBSim module in Fig. 5. Further, we show the classes and methods of FoBSim modules in tables of the Supplemental Material for this paper. Some notes to be taken care of need to be underlined as well:

1. There is a big opportunity for developers to implement new methods in the fog layer. For example, the fog nodes can be extensible to provide privacy-preserving mechanisms (such as described in Baniata, Almobaideen & Kertesz (2020)), computational services (such as described in Fröhlich, Gelenbe & Nowak (2020)), or reputation and trust management services (such as described in Debe et al. (2019)).

2. memPool.py: In this module, the mempool, where TXs are accumulated, is a Python multiprocessing queue that allows different processes to synchronously add() and get() TXs.

3. There are other minor methods from other modules also called by FoBSim entities that mint a new Block, or receive a new TX/Block, in order to synchronously and smoothly apply each different CA’s policies, as declared in its simple version.

4. After each simulation run, some temporary files can be found in the temporary folder of FoBSim. These files are originally initiated by the main module, the BC module, or the miner module. The temporary files are used synchronously by different FoBSim entities, mimicking the real-world interaction between BC entities. The current version of FoBSim generates some or all of the following files depending on the simulated scenario:

• Miners’ local chains.

• Miners’ local records of users’ wallets.

• Log of blocks confirmed by the majority of miners.

• Log of final amounts in miners’ wallets (initial values − staked values + awards).

• Log of coin amounts which were staked by miners.

• The longest confirmed chain.

• Forking log

Genesis block generation

The first block added to the chain in each simulation run is the most important block of the chain. Different scenarios imply different formats of this block, and different methods to broadcast it among, and be accepted by, miner nodes. In the current version of FoBSim, however, a genesis block is initiated with a list of TXs containing only the string “genesis_block” and the labels of the miners available when this block was generated. The block number is 0, the nonce is 0, the generator_id is “The Network”, the previous hash is 0, and the hash is generated using the hashing_function in the blockchain.py module. The timestamp of the genesis block indicates when the chain was launched, hence all blocks shall have bigger timestamp values than the genesis timestamp. Figure 1 of the Supplemental Material of this paper shows a standard FoBSim genesis block, generated in a BC network that consists of two miner nodes.

FoBSim consensus algorithms

Currently, there are three available CAs ready to be used in different simulation scenarios. Next, we describe each one individually as to facilitate any modifications by developers. However, we need to indicate that the three included CAs are in their simplest versions and may require some individual modification in case of the need of more sophisticated ones. Before delving into the CAs, however, we need to discuss the Gossip protocol in FoBSim, as it is deployed regardless of what CA is chosen.

Transaction/block validation in FoBSim

Here, we need to underline some differences between the terms verification, validation, and confirmation, and we need to see how FoBSim differentiates between those terms in different scenarios. As we have touched on these differences in Baniata & Kertész (2020), we need to accurately define each of these terms in order to correctly describe how FoBSim works.

Validation is the process when a miner (either a minter or receiver) checks the correctness of a claim. That is, in the case of a minter miner, the puzzle solution (or nonce) provided with the minted block needs to be correct before the block is broadcast. If the nonce was valid, the block is broadcast, otherwise, a new solution is searched for. While in the case of a receiver miner, the nonce is checked once. If in this later case the solution was valid, the block is accepted, otherwise, the block is rejected.

In the case of payment functionality, the validity of TXs fetched from the mempool is tested. This means that the amount of coins in the wallet of the sender of each TX, in the payment functionality, is compared to the amount to be transferred. If the wallet contains less than the transferred amount, the TX is withdrawn from the block. Later when the new block is received by a miner, the same hash validation and TXs validation take place, except if one of the TXs were invalid, the whole block is rejected. In the case of a block rejection, the minter miner is usually reported in a reputation-aware context. If all the contents of a newly received block are valid (i.e., the hash, the TXs, the wallets, the block number, and the nonce) the block is added to the locally saved chain. Here, we can say that TXs are confirmed, because the block is added to the chain (i.e. the block is confirmed).

The verification, on the other hand, is the process of verifying the identity of an entity. For example, in the case of PoA, only authorized miners are allowed to mint new blocks. Similarly, in the case of PoS, a received block should be generated by a miner that all other miners expect to receive the new block from. Additionally, public information about end-users’ wallets need to be accessible by miners to validate their TXs. Thus, a received block, with some TXs generated by end-users who do not have wallets, or whose wallets contents are not readable by miners, can not be validated and confirmed. Failing to confirm a TX is not necessarily caused by end-users not having sufficient coins to transfer, but may also happen for end-users who can not be verified.

All of these critical principles are, by default, taken care of in FoBSim. All miners are informed about the end-users public identities and wallets’ contents. After that, transferred coins are updated locally in each miner. Consequently, a new TX from the same end-user will be compared to the updated amount of coins in its wallet. Invalid TXs are not included in the block being minted, while invalid TXs cause the rejection of the whole received block. Once a block’s contents are validated, and the TXs/block generators are verified, the TXs are confirmed, the locally saved wallets amounts are updated, and the block is locally confirmed and added to the chain. The most interesting thing is that the very small probability of a double spend attack (Karame, Androulaki & Capkun, 2012) which can appear in PoW-based scenarios, can be easily simulated in FoBSim. All processes are actually happening during each simulation run, rather than substituting them with a small delay as in most BC simulation tools we checked. Hence, validation, verification, and confirmation processes can be modified according to the scenario to be simulated. Nevertheless, Bitcoin decreases the double spend attack probability by regularly raising the difficulty of the puzzle, which is a property that can be modified in FoBSim as well. To facilitate the simulation of such critical scenarios, we deployed two broadcasting approaches for newly minted blocks. The first allows the broadcast process using a simple FOR loop, where miners sequentially validate and confirm new blocks. The second allows the broadcast process using the multiprocessing package, which allows all miners to receive and process new blocks at the same time. Relatively, developers need to be cautious when using the second approach, because of some critical challenges similar to those mentioned in “The Proof of Work”.

Awarding winning miners

Generally speaking, BC miners get rewarded by two system entities for providing the BC service (i.e., BC functionality). The first is the end-user who generated the TX, who pays a small fee once the TX is confirmed (e.g., Gas in Ethereum). The second is the BC network itself (i.e., all miner nodes), which updates the winning miner’s wallet once a new block (minted by the winning miner) is confirmed. We can notice here how important it is to clarify the difference between validation, verification, and confirmation. That is, a miner is verified by its public label and public wallet key/address (ID). Then, a miner being authorized to mint a new block is validated (claim). Finally, a miner is awarded for minting a conformable block (miner’s wallet is updated).

In FoBSim, we implemented the second mechanism, where miners get rewarded for their services by the network. We assume this part is hard because it, also, needs to be agreed on by the majority of BC miners (i.e., at least 51%), and it requires the condition that they confirm the block. The default implementation of FoBSim does that. For the first incentivization mechanism, we thought that it is not applicable in many different scenarios, hence we left it for the developers to add it if needed. For example, to allow end-users to provide fees for getting tasks in the BC, one field can be added to generated TXs, containing the amount of fees the end-user is willing to pay for the service. Once a miner picks a TX (mostly, TXs with higher fees are faster to be picked and processed by miners) and the block containing the TX is confirmed, all miners add the TX fees to the winning miner’s wallet. Figure 2A of the Supplemental Material presents a screenshot of FoBSim output, concluding that a new block was received from Miner_2 by Miner_3, and that the BC module just obtained the needed confirmations to consider the new block confirmed by the whole BC network. Thus, the minter is awarded. Later, the receiver miner presents its updated local chain according to the successful network confirmation. On the other hand, Figure 2B of the Supplemental Material presents a screenshot of the miner_wallets_log after a simulation run, where the PoA CA was used and all miners, except for Miner_5, were authorized to mint new blocks (initial wallet value was 1,000).

Strategies in FoBSim

As had been discussed so far, there are some default strategies used by FoBSim entities throughout each simulation run. To mention some, TXs are picked by miners with no preference, e.g., the highest Gas or priority. Also, a default chain is a single linear chain and new blocks are added to the top of this chain. Some applications, however, have multiple chains or multi-dimentional chains, e.g., Directed Acyclic Graph (DAG) based chains. Additionally, if two blocks appear in the network, the block that was accepted by the majority of miners is confirmed rather than, in some BC systems, the older one is confirmed even if it was confirmed by the minority. Further, a valid block is immediately announced, once found, into the FoBSim network, while in some applications, there might be a conditional delay. For instance, if a selfish mining attack scenario is to be simulated, miners would prefer to keep their newly found blocks secret, hoping they will find the next block as well (Negy, Rizun & Sirer, 2020).

The current version of FoBSim supposes that the data flows from end-users to the fog, and from the fog to the BC network. However, there are other possible data flow schemes that can be simulated, as depicted in Fig. 6. For example, the BC in the current version provides DLT services to end-users, which are communicating with the BC through the fog layer, while services might be provided by the fog layer to the BC network or from the BC network to the fogs in some applications. Further, an application where end-users may need to request data directly from the BC might be possible, which implies different data flow scheme as well. FoBSim facilitates the modification of the data flow in the simulated application, and presents an extra Cloud module that can add more possibilities to the application.

Network connectivity characteristics are a major and critical concern in any BC system. To facilitate network architects job, FoBSim allows to define the number of nodes in each layer, the number of neighbors of each BC node, and the general topology of the network. Additionally, all BC nodes are connected into one giant component by default, whether they were deployed in the fog layer or end-user layer. Accordingly, the effect of manipulating the topology of simulated networks can be easily captured.

FoBSim constraints

Some properties have not been implemented in the current version of FoBSim, such as MT, Digital Signatures and Mining Pools. Additionally, FoBSim source code can be run on a PC with Microsoft Windows or Linux OS, but it may need some modifications if to be run on a PC with a MAC OS (some functions require access to OS operations such as deleting or modifying files located at the secondary memory). Finally, the default limit of recursion in Python may restrict the number of miners to 1,500, which may raise some error regarding the maximum allowed memory use by the interpreter. To solve this, one can modify the maximum limit using the sys.setrecursionlimit in the main function.

Case 1: Comparing time consumption of PoW, PoS, and PoA

When we compare PoW, PoS and PoA in terms of average time consumed for block confirmation, PoW is expected to present the highest time consumption. This is because of the mathematical puzzle that each minter needs to solve in order to prove its illegibility to mint the next block. In PoS, on the other hand, the network algorithm randomly chooses the next minter, while it (slightly) prefers a miner with a higher amount of staked coins. Once a minter is chosen, all miners are informed about the generator of the next block and, thus, the minter needs to perform no tasks other than accumulating TXs in a new standard block. Other miners then accept the new block if it was generated by the minter they were informed about, hence the verification process takes nearly no time (assuming that the transmission delay between miners is set to 0). In simple versions of those two algorithms, all miners have the same source code, thus all miners may be minters, verifiers, and chain maintainers.

The PoA algorithm is the tricky one though. This is because all authorized miners mint new blocks, verify newly minted blocks, and maintain the chain locally. Meanwhile, other BC nodes verify new blocks and maintain the chain, but do not mint new blocks (De Angelis et al., 2018). Consequently, every BC node has a list of authorized entities, including the methods to verify their newly minted blocks. This implies that the more authorized entities, the more complex the verification can be on the receiver side. Accordingly, it is advised that a small number of entities be given authorization for decreasing the complexity of verification (Binance Academy, 2020). Meanwhile, the more maintainers in a PoA-based BC, the higher the overall security level of the system.

In this case study, we run FoBSim several times, with which we deploy different CAs under similar conditions. The simulation runs targeted specifically the measurement of the average time consumed by each CA, from the moment where a miner is triggered to mint a new block, until the minted block by this miner is confirmed by, at least, 51% of other BC miners. To accurately measure this average, we added some variables holding the starting time and the elapsed time, exactly before calling the build_block() function and right after a block is confirmed by reaching the required number of confirmations.

As described in Table 3, we changed the difficulty of the puzzle during the PoW-based BC simulation runs from an easy level (5), to a harder level (10), and finally to very hard levels (15) and (20). During the runs where PoA was used, we changed the number of authorized miners from 2/5 (2 authorized out of a total of 5 miners), 5/10, 10/20, and 25 authorized miners for the rest of runs.

As we wanted to abstractly measure the average confirmation time, we avoided the computational services and payment functionality, because both imply extra time consumption for performing the computational tasks, and validating the payments, respectively. We also avoided the identity management functionality because the number of TXs per end-user is limited by the number of ID attributes required to be saved on the chain. Hence, our best choice was the data management functionality. We kept the total number of TXs delivered to the mempool unchanged, which gives equivalent input for all simulation runs. However, we changed the number of TXs generated by each user as to be equal to the number of miners in each run. More precisely, as the total number of TXs is determined using Eq. (2), where $a$, $b$ and $c$ are the number of fog nodes, the number of end-users, and the number of TXs per end-user, respectively, the values of those variables fluctuated in each run. Concerning the runs where a PoS is deployed, miner nodes were initiated with a wallet that has 1,000 coins, allowing miners to stake random amounts of coins. Additionally, winning miners were awarded 5 coins for each confirmed block they had minted.

(2) $$|TXs|=a\times b\times c$$

We deployed the FoBSim environment on Google Cloud Platform, using a C2-standard-16 (up to 3.8 GHz, 16 vCPUs, 64 GB memory), with Debian OS. We have chosen to place the BC in the end-user layer for all runs, not for any reason other than testing the reliability and stability, of FoBSim components and results, in such complex inter-operable (Belchior et al., 2020) Edge-Fog-BC scenarios. Table 4 presents the exact results we obtained, which are depicted in Figs. 7A and 7B.

According to the results obtained from the simulation runs, one can notice that PoW-based BCs consume much more time to confirm a block, than PoA- and PoS-based BCs, which is inline with the theoretical and experimental results of most previous research. Additionally, the average block confirmation time, in PoW-based and PoA-based BCs, seems to be directly proportional to the BC network size, which complies with the results recently presented in Misic, Misic & Chang (2020). Comparatively, an average block confirmation time in a PoS-based BC seems unaffected by the network size, which complies with the expectations recently presented in Cao et al. (2020).

Case 2: Capturing the effect of using the gossip protocol

In this case, we compare the number of chain forks at the end of several simulation runs, where we interchangeably activate and deactivate the gossiping property in a PoW-based BC. Accordingly, one can notice the effect of gossiping on ledger finality under different conditions, namely the puzzle difficulty and the transmission delay between miners. As it was mentioned in “Gossip Protocol”, gossiping is a continuous process during the life time of the network, which implies that miners would mostly have different chain versions at any given moment. In this case, we detect the number of chain versions at the end of simulation runs, which can be decreased to one version under strictly designed parameters, such as medium network size, high puzzle difficulty, low transmission delay, low number of neighbors per miner, etc. Nevertheless, our goal in this case is to demonstrate how the activation of the gossiping property during a simulation run on FoBSim can decrease the number of chain versions and, thus, it can positively contribute to the consistency of the distributed ledger. For this case, we also deployed the FoBSim environment on the Google Cloud Platform, using a C2-standard-16 VM (up to 3.8 GHz, 16 vCPUs, 64 GB memory), with Ubuntu OS.

Table 5 presents the initial configuration in each simulation scenario, while Tables 6 and 7 present the results we obtained by running the described scenarios, which are depicted in Figs. 8A and 8B. As can be noted from the results, the default gossip protocol in FoBSim could decrease the number of chain versions at the end of each simulation run. Although the number of chain versions did not reach the optimum value (i.e., one chain version), it is obvious that activating the gossiping property decreases the number of chain versions at each simulation run and, thus, enhances the distributed ledger consistency.

Case 3: Comparing deployment efficiency of BC in the fog layer vs. end-user layer

In this case, we compare BC deployment efficiency in the fog layer and end-user layer. The efficiency we are seeking is determined by both the total time needed to perform all requested BC services and total storage cost. That is, less time and storage needed to perform all tasks (e.g., confirm all newly minted blocks or run the generated SCs) indicates higher efficiency of the BC system. To fairly compare the BC efficiency when deployed in those two layers, we stabilize all BC parameters that are configurable in FoBSim, except for the number of miner nodes to deduce the trend in total time consumption when the network dynamically allows for new nodes to join the network. We deployed the FoBSim tool on the Google Cloud Platform, using a C2-standard-16 VM (up to 3.8 GHz, 16 vCPUs, 64 GB memory), with Ubuntu OS. The detailed parameter configuration while running the described scenarios is presented in Table 8.

Recalling the results presented in Bi, Yang & Zheng (2018) and Li et al. (2017), average transmission delay between miners in the fog layer can be estimated by 12 ms., while it can be estimated between miners in the end-user layer to 1,000 ms. (higher transmission delays were reported in well known BC networks, such as Bitcoin, in Sallal (2018)). We simulated the data management BC service and PoW consensus with gossiping activated. According to Eq. (2), the number of requested tasks was automatically modified due to the continuous change in the number of fog nodes (since we oscillated the number of fog nodes to deduce the trend of total time consumption). The total average time for performing requested BC services, in similar simulation sittings, while the BC is deployed in end-user and fog layers, is compared in Fig. 9A.

To accurately measure the storage cost during the simulation run, we implemented an independent Python code, available in the FoBSim repository, namely storage_cost_analysis.py. As described in “FoBSim Modules”, the output analysis files, ledgers, wallets, etc. of running a given simulation scenario using FoBSim, are automatically saved in a folder titled “temporary” within the same repository. Thus, our implemented storage analyzer aims at regularly (i.e., every one second as a default sitting) measuring the size of this temporary folder while the simulation is running. The measured sizes are then saved into an Excel sheet to facilitate performing the analysis we are seeking. To exemplify this, the total storage used by the BC network is compared in Fig. 9B, where similar simulation sittings were configured (detailed in Table 8), except for the layer where the BC is deployed.

It can be noted from the results presented in the third case that deploying the BC network in the fog layer may enhance its efficiency in terms of total time consumed to perform similar tasks in similar configuration, and in terms of total storage cost by the BC network to maintain the same distributed ledger (same number of confirmed blocks by the end of the simulation run).