Frameworks for developing an Agro-Prosumer Community Group platform

Phase 1 Prosumer clustering

The first phase includes clustering of the prosumers’ profiles and detecting any outliers using a hierarchical clustering method. Prosumers’ seasonal profiles for two seasons (summer and winter) are taken as an input for this phase. There are three steps in this phase: creation of regional groups, building clusters, and outlier detection.

Step I Creating regional groups

In this phase, agro-prosumer’s postcode is taken as an input. The prosumers are partitioned into groups based on their postcodes within a certain region. This will mean that the delivery of prosumer produce can be done without the need for long-distance transportation. The output of this step will provide GL-clusters (geographical location based-clusters) based on postcodes and the neighborhood zone.

Step II Outlier detection

In order to deal with outliers, a threshold is set: after calculating the distance between existing clusters, if the shortest distance is not further than the threshold, we assign the dataset to its closest cluster. If the shortest distance exceeds the threshold, this means that the cluster could belong to a minor group. The objects in the minor group are those that did not belong to any major groups. Objects in minor groups are data points, not outliers as they do not belong to any major groups. Further clustering of objects in minor groups can be done for future analysis.

Step III Building clusters

For each GL-cluster obtained from step one and after removing outliers, the corresponding agro-prosumer profiles are considered in the next step, and clusters are formed based upon prosumers’ production history. The hierarchical clustering method is used to decide the number of clusters. Initially, each prosumer profile is placed in a unique cluster. For each pair of clusters, some value of dissimilarity or distance is computed. In this case, minimum variance, i.e., Ward’s criterion, is used to minimize the total within-cluster variance and find the pair of clusters that leads to minimum increase in total within-cluster variance after merging. In every step, the clusters with the minimum variance in the current clustering are merged until the whole dataset forms a single cluster. Hierarchical clustering helps in identifying groups in the dataset. Thus, the output from this step will be number of prosumer clusters based on their production similarity.

Phase 2 Prosumer cluster optimization and forming pre-requisites:

This phase involves the optimization of prosumer groups based on the number of prosumers in each group and their production amount. Firstly the clusters are optimized and pre-requisites for each cluster-group is formed. The optimization steps and pre-requisites are further illustrated in this section.

Step I Optimization of prosumer clusters

Agro-prosumer cluster-groups created by using hierarchical clustering are optimized to produce sufficient number of clusters that will then represent different agro-prosumer community groups. The number of clusters produced by optimization, depends on the variation of production quantity. If the variation is large, too many clusters could be formed, which are not feasible to manage. Thus, this stage involves optimizing the clusters into a feasible number of APCGs by merging small clusters into one or splitting large clusters into smaller ones to obtain a feasible number of APCGs to satisfy market requirements. In order to determine the ideal number of clusters, firstly, suburb requirements are analyzed and the expectations of relevant APCGs are derived.

To optimize APCGs:

Let X be the population of suburb ABC and C is the per capita consumption of lemons. Assume that the APCG framework targets a minimum 1% of lemon market for a suburb ABC. Then the requirement (R_expected) of lemons for suburb ABC using APCGs can be calculated with

R_expected = X*C*0.01.

Let L be the number of clusters formed using the clustering method, and R_L represents every APCG’s goal.

R_L = R_expected/L.

After determining the suburb’s requirements, next step optimizes the clusters by evaluating number of agro-prosumers present in each APCG (as shown in Fig. 3). Let say P_l and P_h respectively be the lowest and highest number of prosumers expected in each group. Let R_L be the minimum amount of production expected from each APCG. Prosumers count (P_num) and the production quantity (R_obtained) from a specific prosumer cluster is shown in Eqs. (1) and (2). (1)${\displaystyle P_{\mathrm{l}}<P_{\mathrm{num}}<P_{\mathrm{h}}}$ (2)${\displaystyle R_{\mathrm{obtained}}>R_{\mathrm{L}}}$ If the production is less than the expected amount (R_obtained<R_L), or the number of prosumers is lower than the ideal number of prosumers (P_l>P_num), that agro-prosumer cluster is merged with the closest prosumer cluster, and the same process continues until the prosumer cluster can meet the total production requirement (R_obtained>R_L) and the number of prosumers (P_l<P_num<P _h) defined for the APCG.

Now, if too many prosumers form an agro-prosumer clusters, the clusters are further break down into small size clusters consisting of an most favourable number of prosumers and meeting the production goals i.e., R_obtained>R_L and P_l<P_num<P_h.

The final output of optimization will result in the optimised prosumer clusters, which are then represented as APCGs. Now these APCGs are analysed to identify the unique production characteristics or pattern of each group which will be denoted as the pre-requisite of the APCGs.

Pre-requisite formation

Introduction to APCGs includes formation of unique entry requirements for each group. The two key input, as discussed in the previous sections, to determine the prosumers’ adherence are the “lower threshold” (Lt) and the “upper threshold” (Ut). The defined inputs used as pre-requisites of each APCGs will be:

• Lower threshold: L_t

• Upper threshold: U_t

Framework 2: agro-prosumers recruitment framework

A new recruitment framework is designed to evaluate real time behavior of new agro-prosumers and allocate them in specific APCG groups. The reason for designing this approach is to encourage participation of non-farmers and new gardeners, which not only help them to estimate production details, outline incentive benefits etc., but will also ease off the management of APCG in the long run. Additionally this framework requirement is new and won’t be justified to use CSA methods which basically works on partnership basis and useful for a large piece of land (Ahluwalia & Miller, 2014; Cone & Myhre, 2000; Blattel-Mink et al., 2017). An overview of the framework is shown in Fig. 4.

New agro-prosumers who are interested in joining the APCGs, and their real-time behavior profiles, are collected as input for this framework. We term these agro-prosumers “prospect agro-prosumers” who are assumed to be new to the community sharing network; thus, because there are no previous production profiles, real-time production needs to be determined. The final outcome of this framework is the recruitment of prospect agro-prosumer to suitable APCGs. This stage is further divided into four components, which are explained below.

The framework has four components:

1. An approach to evaluate agro-prosumers’ production performance;

2. Agro-prosumers’ transaction assessment during the evaluation period

3. An approach to analyse agro-prosumers’ stability; and

4. Agro-prosumers recruitment to a specific APCG after the evaluation period.

The varying nature of agro-prosumers’ production behaviour is evaluated using the above approaches, and allocates them to a temporary “variable APCG”. Later on, the prosumers’ overall behaviour is stored and evaluated prior to recruitment to a specific APCG, i.e., to one of the final APCGs. The requirements for the proposed solution are covered via four components (listed above) discussed in detail below.

Approach to evaluate prosumers’ production performance

Finding an approach to estimate agro-prosumers’ performance is the first component of the evaluation technique, which helps in understanding the evaluation period activities and evaluation inputs.

Agro-prosumer evaluation measures

As discussed previously, the “evaluation period” is an established period of consecutive seasons during which the production behavior of new agro-prosumers who are interested in joining an APCG, is evaluated. The evaluation period is divided into two seasons per year in Australia: winter (i.e., March–August) and summer (i.e., September–February). These winter and summer seasons show non-overlapping, mutually exclusive time periods and are assigned with a production transaction between agro-prosumer and the APCG module using production value.

Agro-prosumers’ production data is generated using the Australian national average. Production data such as family size, farming methods (organic, inorganic), lemon variety (three major lemon variety has been used) and number of trees (1–10 has been randomly used) and their respective ages (age of a tree is assumed from 5–100 years), are collected as input to evaluate their consumption pattern and production performance for two season or annually. Agro-prosumers’ surplus production is considered as the final value for one season/year. Thus, prosumers’ performance is estimated using that final value, and is evaluated for each season. Next section explains the approach used to determine the prosumers’ performance for each season during the evaluation period.

The proposed approach

This approach requires two inputs: the input from the agro-prosumer and the input from the APCG module as shown in Fig. 5. Inputs from the agro-prosumer include production summary for a season and the prosumer’s preferred APCG. The APCG module’s input comprises the pre-requisites of the available APCGs.

A probabilistic approach is used here to evaluate agro-prosumers’ production performance based on the pre-condition criteria of their preferred APCGs.

Results of this approach are the “performance indices” and variable APCG of the agro-prosumer for each season. Performance indices are used to anticipate the level of success and/or failure of an agro-prosumer in meeting the pre-condition criteria of his/her preferred APCGs. To utilize it, different levels of success and failure are represented using a four-point scale as shown in Table 1. In fact, each performance index shows different value or success rate of performance in the production behavior.

The performance scale ranges from 0 to 3, where 3 represents the complete success or match, and the minimum success rate is 80% for meeting the pre-condition criteria. If the success rate is less than 79%, it will be considered as a “failure”.

The performance scale used here has single-integer values. It is difficult to use extreme values, i.e., only high or low, to measure prosumer behavior. Hence, in order to determine and model the performance of prosumers more accurately, various levels of performance should be identified first. Moreover, to accurately determine prosumer performance, the various levels of performance must be identified. A performance score with a value from 0 to 3 will help to indicate prosumers’ performance for APCGs development.

• Complete success: The highest point on the performance Score is 3, which indicates “Complete success”. This score suggests 100% success rate in interacting with the prosumers’ production-sharing process. This level of performance according to the PS suggests that the prosumer is strongly suited to his preferred APCG and meets the desired pre-condition criteria.

• Intermediate success: This level denotes 90–99% of success rate in interacting with prosumers’ production behavior. Performance Score 2 shows that it is the “medium success” level. This score suggests that in meeting the prosumers’ preferred APCG requirements, prosumers’ performance reliability is good.

• Entry success: Performance score 1 indicates “Entry success”. This score suggests 80–89% success rate while satisfying the pre-requirements of the preferred APCG’s. This performance index score suggests that the prosumer is slightly reliable in meeting the desired pre-condition criteria of his/her preferred APCG.

• Failure: 0 reflects the lowest score in performance, indicating “failure”. This level depicts 0–79% rate of success in fulfilling the pre-requirements. Thus, this level shows that the prosumer’s performance is not reliable enough to meet the pre-condition criteria for the APCG. Hence, the prosumer with this index could be matched with other APCG rather than the preferred one.

The mathematical expression of performance indices is given in Eq. (3).

For a season (j) of the evaluation period, the rate of success of the prosumer (P_ij) being allocated to prefer variable APCG (C_p): (3)${\displaystyle Rate\left(P_{ij}\in C_p\right)\left\{\begin{array}{c}\hfill 100\text{\%}:if{E}_{ij}\ge L_p\hfill \\ \hfill \frac{E_{ij}}{L_p}:if{E}_{ij}<L_p\hfill \end{array}\right\}}$ where P_ij is an i^th agro-prosumer’s performance in the j^th season, C_p is the preferred APCG, E_ij is the real time production commitment of i^th agro-prosumer and L_p is the production threshold of agro-prosumers preferred APCG.

Agro-prosumers’ transaction assessment during the evaluation period

For ongoing assessment during the evaluation period, agro-prosumer is aimed to assign into his chosen APCG for each season. The evaluation process is shown in Fig. 6. The key steps of the process are as follows: the prospect prosumer is asked to submit records of production in real time for “n” seasons during the evaluation period. For each season, dynamic production amount is compared with the minimum threshold (E_th), which is the minimum requirement of any APCG. If the prosumers’ production is equal or greater than the E_th, the prosumer is viewed to be an eligible prosumer.

Next, if a prospect agro-prosumer receives ‘eligible prosumer’ status during his/her first season, she/he will be promoted to the next season and then to following seasons. However, if she/he fails to meet the eligible agro-prosumer requirement, in the first production season, the evaluation period will be extended with more seasons.

However, if the new agro-prosumer is not able to match the minimum threshold (E_th), then the prosumer’s evaluation period is extended by another season and the prosumer remains under evaluation until succeeded. On the completion of the evaluation period, prospect agro-prosumers’ stability will be analyzed using stability index, which is discussed next.

An approach developed to analyze agro-prosumer stability

The stability of an agro-prosumers’ reliability is estimated for his/her preferred APCG, as well as for those assigned throughout the evaluation period. Figure 7 shows a process to obtain prosumers’ stability for agro-prosumers’ chosen APCG.

During evaluation period, for each season, agro-prosumers’ performance index values are taken as an input along with their temporary APCGs. Equations (4) and (5) formulate a mathematical equation for the approach. SI represents the stability index which is used to determine the feasibility that prosumers will remain in their preferred APCG. The output for I index is between 0 and 3, and a higher I shows high chances of prosumers remaining in their preferred APCG: (4)${\displaystyle I_{pi}=\frac{\sum _{j=1}^{ns}P{X}_{ij}}{ns}.}$ Above I_pi is the stability index of the i^th prosumer with respect to chosen APCG (C_p), PX _ij is an i^th prosumers’ performance index in the j^th season and ns is the number of seasons where the prosumer is assigned to his/her chosen APCG. To determine most suitable APCG for an agro-prosumer, rate of engagement to a specific APCG is calculated using Eq. (5). For example, if the agro-prosumers’ rate is higher for APCG1 than other APCGs, than the chosen APCG1 is seen as the most favorable APCG for that prosumer’s engagement. (5)${\displaystyle Rate\left(P_i\in APC{G}_{Fr}\right)=\frac{countof\left(APC{G}_{Fr}\right)}{ns}}$ where P_i is the i^th prosumer, APCG_Fr is the r^th temporary APCG, count of (APCG_Fr) shows the total number of times the prosumer is selected to r^th temporary APCG during the evaluation period and ns is the number of seasons.

The next section discusses agro-prosumer engagement to the permanent APCG based on the previously-described method.

Agro-prosumers engagement to the permanent APCG after the valuation period

Agro-prosumer engagement to the most suitable APCG is analyzed in this step. The overall performance of prospect agro-prosumers overall performance is assessed at the end of the evaluation period. Figure 8 is a flowchart showing this process. As discussed in the previous section, the Stability Index, based on an agro-prosumer’s performance index, is calculated throughout the evaluation period. Additionally, agro-prosumers’ rate of staying in temporary APCGs during the entire evaluation period is assessed. Equation (6) is utilized to identify the combined value of the agro-prosumer being allocated to the permanent APCG. The APCG which shows the highest joined index is chosen as that prosumer’s final permanent APCG. (6)${\displaystyle IPr\left(P_i\in APC{G}_{Fr}\right)=I_{pi}\times Rate\left(P_i\in APC{G}_{Fr}\right).}$

Framework 3: Goal management framework

The input for the framework includes diverse goals for agro-prosumer community groups.

The solution framework consists of a goal management component. The outcome of the goal management phase is an optimized set of overall goals for the community-based, harvest-sharing network. The processes involved in goal management are shown as an overview of the framework in Fig. 9.

Goal management

The goal management stage is responsible to attain ideal goals structure out of overall goals. The purpose involves solving diverse conflicting goals in the APCG to obtain best solution in terms of goals priority. The feature of MCGP (Ravindran, 2009) and an approach utilised in smart grid goal management (Rathnayaka et al., 2015b) is referred to design best possible solution for conflicting goals. To achieve this, each and every identified objective is attached with a rank based on their priority. High rank objectives are treated as goals to work out first, and therefore attempts are made to find a solution which is close to the pre-ranking set of goals. Goal programming minimises the deviation between the theoretical goals and realistic achievements. These deviation can be both positive and negative, thus an objective function is used to minimise the deviations based on the relative importance of the goals.

Various areas has utilised goal programming model benefits such as environment, energy, smart grid, academic and health planning (Rathnayaka et al., 2015a), and shows success in solving diverse conflicting goals. In this framework, we adapt MCGP techniques for our framework. Figure 10 presents the algorithm for the goal programming model, where the parameters and equations are explained in the following section. The model has six parts:

1. APCGs goal recognition,

2. Summary of variables,

3. Objective classification,

4. Objective ranking,

5. Goal equation formation, and

6. Generating objective functions.

Part 1: APCGs goal recognition

APCGs diverse goals are identified in this phase. These objectives are explained below.

1. Carbon content objective (C1): The “carbon-capture objective” refers to the use of organic farming methods to maximize carbon capture, which will increase the carbon content which can be traded with external companies. More carbon capture will result in more carbon sequestration and less emission.

2. Food security within the network (F1): The goal is secure the vegetable/fruit demand of local members within the APCGs. Realistically, some members within an APCG may struggle producing sufficient quantity to meet their own consumption needs. Hence, food security of APCG members have been targeted.

3. Providing local food access to wider community (L2): With growing local food, APCGs can make locally grown vegetables available to the extended community such as external customers or supermarkets, greengrocers, and external consumers who are not registered with an APCG.

4. Income and Incentive objective (I3): The “income and incentive objective” focus is to earn income and incentive from selling surplus production of APCGs to vegetable/fruit buyers and trading carbon tokens with industries.

5. Maintenance cost reduction objective (M4): This goal refers to reducing the cost of APCGs maintenance over time. For example, “maintenance cost” may represent the one time cost to build APCG platform and maintaining the database and transaction records etc. Cost related to collection and distribution of products/vegetation from members, to stores, etc. Additionally providing benefits to the members may require a payment gateway which may incur cost.

6. Stable APCG objective (S5): The increase in the number of active APCG members, that is, those who dynamically participate in the production-sharing or carbon-sharing network, is a “stability objective”.

Part 2: Summary of variables

In order to use MCGP all variables and their deviations are identified. For APCG the idea is to identify variables and summarize their deviations to achieve ideal set of goals. The production amount and carbon tokens generated by each group will be counted as variables and maximizing/minimizing the value is considered as deviation.

Part 3: objective classification

The objectives are classified as definite and flexible constraints based on the previous objectives (part 1). At this point, the “definite goals” are outlined as mandatory requirement on the variables, whereas the “flexible goals” are outlined as the objectives nice to have but not necessary (Zeleny, 1976). The classification of goals are as follows:

1. Definite goals: Maximum carbon capture objective (C). For example, the APCG’s base is environmental sustainability. Thus, ecological methods must be used for APCG production.

2. Flexible goals: Goals such as local food security (F1), extended community and customer demand objective (L2), income & incentive objective (I3), maintenance cost objective (M4), and stability of APCG (S5). Refinement of these goals helps in achieving the ideal goal set, which would benefit APCG. The variables summaries is defined as: maximum C1, minimum F1, minimum L2, minimum I3, maximum M4, and minimum S5; these are termed “expected values” in the goal programming model.

Part 4: Objective ranking

To make sure important goals met first, the priorities of the goals have been assigned. This step discusses ranking out the goals by assigning a weight (or rank) to each goal. As mentioned earlier, goals can be mutually exclusive; i.e., one goal may be achievable at the expense of another. This makes it critically important to assign weights to the goals, so that least important goals are only met after the important ones. Keeping local network food security (F1) as priority, total goal set can be determined as 4!, thus in total 24 structures will be formed such as F1L2I3M4S5, F1L2M4S5I3…F1S5M4I3L2.

Part 5: Goal equation formation

Mathematical relations are developed in this section for the definite and flexible goals. Equations are as follows-

1. Carbon capture Objective (C): Organic farming methods should be used for APCG produce to increase the carbon token value.

2. Food security local demand objective (GC1): Satisfying food security of APCG should be focused. Thus, the purpose of this goal is to minimise the negative deviation from the quantity of surplus production of each APCG. Let A_pi E_i be the extra production produced by i^th APCG, k ₀ and l ₀ be negative and positive variance respectively, and t be the number of APCGs; then the equation for food security local demand objective (F1) would be: ${\displaystyle A_{pi}\times E_i\ge 0;\forall i\le t}$ (7)${\displaystyle A_{pi}\times E_i+k_0-l_0=0;\forall i\le t}$ Considering 4 APCG groups for this framework, 4 equations will be formed (m =4) for each group; N_p1 × E₁ + k₁ − l₁ = 0;   …N_p4 × E₄ + k₄ − l₄ = 0; 

3. Local community demand objective (L2): The purpose of L2 is to minimise the negative variance of the total surplus production of all APCG. Assuming requirement from external supermarket is R. And positive and negative variance be s and q, respectively; then the equation will be formed as ${\displaystyle \sum _{i=1}^m{E}_i\times A_{pi}\ge R}$ (8)${\displaystyle \sum _{i=1}^m{E}_i\times A_{pi}+q-s=R}$

4. Income & Incentive objective (I3): Obtaining higher income is another requirement of the framework. The minimum income expectation of the ith APCG be Ii, and positive and negative variance be q1 and s1 respectively; then the equation for this objective will be minimizing negative variance ${\displaystyle \sum _{i=1}^n{I}_i\times E_i\times A_{pi}\ge I}$ (9)${\displaystyle \sum _{i=1}^n{I}_i\times E_i\times A_{pi}+q1-s1=I}$

5. Maintenance cost objective (M4): Let say the maintenance cost allowances be M, and the positive and negative variance be q2 and s2, respectively; equation for the maintenance cost objective (GC4) is obtained with Equation 5.6, where Ci is the coefficient, represents the cost rate of ith APCG. ${\displaystyle \sum _{i=1}^n{C}_i\times E_i\times A_{pi}\le M}$ (10)${\displaystyle \sum _{i=1}^n{C}_i\times E_i\times A_{pi}+q2-s2=M}$

6. Sustainability objective (GC5): Let P be the minimum number of prosumers who are participating in APCG, and positive and negative variance be q3 and s3, respectively; then, the formula for the sustainability objective (G5) would be: ${\displaystyle \sum _{i=1}^n{A}_{pi}\ge P}$ (11)${\displaystyle \sum _{i=1}^n{A}_{pi}+q_3-s_3=P}$

Part 6 Development of objective functions

Finally the objective function of each goal is formulated and, best possible solution is formed by minimizing the deviations from each goal. The objective functions here are the [(k1, k2, k3, k4), q, q ₁, s ₂, q ₃]. Partitioning algorithm is used to solve this linear goal programming problem.

Goal programming solution

As discussed previously, 24 priority goal structure sets are identified along with different ranking order. The partitioning algorithm is utilized as a solution here, in order to solve the linear goal programming problem (Rathnayaka et al., 2015b). The solution working principle implies on the definition of priority structures which implies that higher-order goals must be optimised before lower-order goals are even considered. The solution procedure is shown in Fig. 11 which consists of solving a series of linear programming sub-problems by using the solution of the higher-priority problem solved prior to the lower-priority problem. All the sub-problems assigned to a higher priority goals are solved first using the partitioning algorithm. The ideal tableau for this sub-problem is then examined for alternative ideal solutions. If none exists, then the present solution is ideal for the original problem with respect to all the priorities.

The algorithm then substitutes the values of the parameters for the flexible goals of the lower priorities to calculate their satisfaction levels, and the problem is solved. However, if alternative ideal solutions do exist, the next set of flexible goal and their objective function terms are added to the problem. This brings the algorithm to the next sub-problem in the series, and the optimisation resumes. The algorithm continues in this manner until no alternative ideal exists for one of the sub-problems or until all priorities have been included in the optimisation (Ravindran, 2009; Zeleny, 1976).

Goal management problem provides the best solution by comparing the achievable set of goals when compared to the predetermined goals. Additionally the identification of the necessary alterations to parameters are explained well in order to achieve all the goals in different priority structures.