Attitudes towards Italian Mafias Scale (AIMS): development and validation

Attitudes and their measures towards mafia, criminal organizations and crime

Along the decades scholars have proposed different models regarding attitudes structure: for instance, the “Multi-attribute Measurement Model” by Fishbein (1963) views an attitude as the summation between a certain belief of the attributes of the attitudinal object and the evaluation of those attributes. In the “Vector Model” by Calder & Lutz (1972) attitudes occupy a two-dimensional space made of an affective and a cognitive component. Extending these two-component models, the most prominent framework of attitudes structure comprises tripartite models, whereby attitudes consist of an affective, a cognitive, and a behavioral component (Rosenberg et al., 1960). The most known of these models is the ABC model by Eagly & Chaiken, for which the affective component refers to feelings and emotions, the cognitive component refers to beliefs, and the behavioral component to behavioral intentions towards the attitudinal object (Eagly & Chaiken, 1998; Eagly & Chaiken, 2007).

In many research fields attitudes have been measured through qualitative as well as quantitative methods, and research on crime and mafia make no exception (e.g., Iacolino, Pellerone & Ferraro, 2017; Sarno, 2014; Travaglino, Abrams & De Moura, 2016). Surveys and questionnaires might be particularly useful as they are easy to administer to large numbers of people, straightforward to understand, and give standardized results. Unfortunately, due to its contextual specificity, research and tools for measuring people’s attitudes towards Italian mafias have been lacking. Travaglino et al. (2014) and Travaglino, Abrams & De Moura (2016) used a set of items to measure the mediating role of the attitudes towards Camorra (the Campania regional variation of mafia) in the relation between masculine and honor ideology and collective actions against Camorra itself. Although their measure displayed a good reliability, it has not been subjected to psychometric validation, and thus may not turn out to be the best tool for measuring attitudes towards mafias. In a similar vein, different associations against mafias periodically administer useful and relevant surveys but without submitting them to a proper validation procedure (see, for instance, Della Ratta-Rinaldi, Ioppolo & Ricotta, 2012).

Measurements of the attitudes towards other illegal phenomena (e.g., drinking and driving; Sprang, 1997), anti-social behaviors, and attitudes towards crime in general are somewhat more reliable, as they underwent to proper systematization and psychometric validation. One of the first and widely employed measures is the “Criminal Sentiment Scale” (CSS; Gendreau et al., 1979), which consists of five subscales assessing the attitudes towards different crime-related entities (i.e., law, court, police, tolerance towards violations and identification with criminal others). In a similar vein, the “Measure of Criminal Attitudes and Associates” (MCAA; Mills, Kroner & Forth, 2002) is a two-part questionnaire that asks people to indicate whether and to what extent their important others are criminals (part 1), and a series of attitudes towards criminal other themselves, violence, entitlement (i.e., a person’s sentiment to obtain whatever they want) and anti-social intent (part 2). A more general approach is instead employed in the “Attitudes towards Crime Scale” (ACS; Ortet-Fabregat & Pérez, 1992), which measures the attitudes towards three distinct dimensions, namely the (i) causes, (ii) prevention and (iii) treatment of crime. It is worth mentioning also the “Psychological Inventory of Criminal Thinking Styles” (PICTS; Walters, 2007), which assesses the eight thinking styles that are thought to support and reinforce the four behavioral patterns of criminal activity specified by the “Lifestyle Criminality Screening Form” (LCSF; Walters, White & Denney, 1991). However, none of the above instruments are tailored for the specific mafia phenomenon, which, due to the involvement of cultural, social, and political elements, differs from “classic” criminality. Moreover, although spread in different geographical contexts, mafia is a typical Italian phenomenon, which needs to be dealt with primarily within its context of origin.

With this in mind, this research aims at filling this gap by developing a reliable and psychometrically valid instrument for measuring the attitudes towards Italian mafias (AIMS). Such a tool could help researchers and policymakers to grasp the sentiment of the Italian population and promote ad hoc interventions to reduce the support for mafia-like organizations and their extremely negative societal consequences. To achieve this objective we conducted two studies, each with specific aims:

in Study 1 we aimed at reducing the number of the items generated in a previous stage and exploring the latent structure of the scale; in Study 2 the main purpose was to confirm the scale’s structure explored in Study 1. In addition, we aimed at cross-validating the scale by testing (i) its discriminant validity against the revised version of the Criminal Sentiment Scale (CSS; Gendreau et al., 1979; Shields & Simourd, 1991); (ii) its criterion predictive validity in relation to donating real money to an association against mafias; (iii) its measurement invariance in participants born in the five deep-rooted mafia regions (DRMR; i.e., Apulia, Basilicata, Calabria, Campania, and Sicily) and participants born in the rest of Italy.

Items generation

To create the items of the AIMS we used a deductive method, also known as “classification from above” (Hunt, 1991 in Boateng et al., 2018), whereby we reviewed the existing literature on Italian mafias in different research fields (e.g., psychology, sociology, investigative journalism, law, history, etc.). Moreover, we asked two psychologists working for the Police and with experience with mafia criminals to review the content of the items. As the point of reference in research on attitudes, we developed our scale based on the framework comprising tripartite models of attitudes (Eagly & Chaiken, 1998; Eagly & Chaiken, 2007; Rosenberg et al., 1960). Thus, we created items referring to the emotional (i.e., emotions), cognitive (i.e., thoughts and beliefs), and behavioral (i.e., behavioral intentions) components.

The initial pool of 76 items consisted of at least twice the number of items of the expected scale (see Supplementary materials for the complete list) in accordance with the suggestion by Schinka, Velicer & Weiner (2012).

Study 1

Items reduction

To reduce the high number of items we followed both a theory-driven and a data-driven approach. In the former, the decision to remove or retain a certain item was based on a thorough examination of its content and theoretical significance for the scale. In the latter, items that correlated less than —.2— with the total score (obtained by averaging all items of the scale) were considered for removal. In fact, an item that correlates less than —.3— or —.2— with the total score might be measuring something different compared to the other items in the scale (Field, 2005). Here, aware of the potential consequences of including items whose content might be very far from the construct of interest, we opted for the more conservative criterion of —.2—. Moreover, we excluded the items whose median scores corresponded exactly to the extremes of the response scale (i.e., 1 or 7). In fact, extreme response style (ERS) might affect the validity of the construct measurement (Moors, 2004 in Batchelor & Florida, 2016). Items with such extreme values were also those that might have had the least discriminant power. Thus, we started by eliminating these items, then we eliminated one item (E3-“How much warm/cold do you feel with respect to mafia organized crime?”) because, upon closer inspection, its labels were ambiguous: for instance, “warm” might have been interpreted both in terms of positive (e.g., passion) and negative (e.g., anger) emotions. We eliminated two items (E10, C32) because we realized that they did not precisely measure the attitudes towards mafias. We eliminated three items (B6, C31, C33) because they correlated indeed less than —.2— with the total score of the questionnaire. However, we retained five items (B22, B23, C8, C23, C25) because, even if poorly correlated with the total score, they were deemed as theoretically important.

The final pool consisted of 28 items (see Table 1).

Study 2

Confirmatory factor analysis

We conducted a confirmatory factor analysis (CFA) on the 23-item of the AIMS by means of the R packages lavaan (Rosseel, 2012) and semTools (Jorgensen et al., 2021). Due to the ordinal nature of the response scale, we used the diagonally weighted least squares (DWLS) estimation method. No missing values were present in this study.

Given the results of the EFA in Study 1, which supported the theoretical basis of this research and represented by the tripartite model of attitudes, we performed the CFA on three main models, each assuming a structure with three components. However, to explore the possibility that the latent structure of the AIMS might still be univocal, we tested a fourth unidimensional model.

Thus, we built:

(1) a bifactor model, in which all the items loaded onto a general factor, and sub-sets of items loaded onto three grouping orthogonal factors;

(2) a second-order model, in which three correlated sub-sets of items were explained by a superordinate factor;

(3) a tridimensional model, in which sub-sets of items loaded onto three specific factors, but no general or superordinate factor was assumed;

(4) a unidimensional model in which all the items loaded onto one factor.

According to the standard criteria for the goodness of fit indexes (Baumgartner & Homburg, 1996; Hu & Bentler, 1999), the unidimensional model resulted in an insufficient fit. Conversely, the tridimensional and the second-order model had much better fits. Nonetheless, the best fit was achieved by the bifactor model (see Table 2). Model comparisons confirmed that the bifactor model (χ²(227) = 401.10) significantly outperformed the other models (tridimensional: χ² (207) = 942.15, p <. 001; second-order: χ² (207) = 942.15, p < .001; unidimensional: χ²(230) = 2192.81, p < .001), indicating that a structure with a general factor (Mafia Attitude) and three specific factors (Behaviors, Cognitions, Emotions-Cognitions) was the most compatible with our data.

However, in terms of local fit, a closer inspection of the bifactor model revealed that some items loaded poorly on the general factor Mafia Attitude, which might signal a negligible contribution to the scale, if not possible misspecification issues. Thus, we decided to remove those items that loaded less than —.3— onto the general factor (E9, C8, C23 and C25) which resulted in a 19-item scale. A new inspection of the model showed that one further item (C10) resulted in a loading below the cutoff of —.3—, and for this reason was removed. The new 18-item bifactor model showed a better fit than the original 23-item model, but a slightly worse fit compared to the 19-item model (see Table 3). Nonetheless, because this model appeared simpler and in accordance with the criterium used for item removal (i.e., loadings <—.3— into the general factor), we decided to retain the 18-item model for the following analyses (see Table 4 for items list and factor loadings).

We performed another model comparison between the aforementioned models relative to the new 18-item scale: again, the bifactor model(χ²(117) = 228.16) significantly outperformed the other models (tridimensional: χ² (132) = 507.77, p < .001; second-order: χ² (132) = 507.77, p < .001; unidimensional: χ²(135) = 1173.08, p < .001).

Internal consistency

To measure the scale’s internal consistency we computed the McDonald’s ω (Flora, 2020) for the general factor and for the three specific factors of the bifactor model: as shown in Table 5 omegas total were good to excellent for the general factor Mafia Attitude and for Behaviors and Emotion-Cognitions subscales (Nunnally, 1978, see also Nájera Catalán, 2019), while still acceptable for Cognitions subscale. However, omega hierarchical indices demonstrate that a substantial proportion of scale internal consistency can be attributed to the general factor.

Construct validity

To test the AIMS’ discriminant validity we confronted participants’ scores on the scale and their scores on the CSS. To do so, we performed a CFA on three different models:

(1) a discriminant model where we specified two macro-factors, one for the AIMS and one for the CSS with their subscales;

(2) a unidimensional model where we specified one macro-factor comprising the items of AIMS and CSS all together;

(3) a convergent model where we specified a macro-factor comprising both AIMS and CSS and their subscales.

To avoid under identification problems due to the high number of parameters to be estimated compared to the number of observations, we used a robust maximum likelihood (MLR) estimation method. Model comparison revealed that the discriminant model outperformed the other two, giving evidence for discriminant validity (see Table 6).

To furtherly support this claim we performed a correlational analysis between participants’ scores on the AIMS and their scores on the CSS. To do so, we first created an index for the AIMS by averaging its items, such that higher scores indicated pro-mafia attitudes. We then created an index also for the CSS by following the procedure described in Shields & Simourd (1991). To increase its interpretability, we reversed the CSS scores such that higher values indicated pro-crime attitudes.

We performed a Spearman’s rank-order correlation for non-parametric tests between the two questionnaires: results showed a positive and significant correlation (rho = .45, p < .001), indicating that a positive attitude towards mafias corresponded to a positive attitude towards crime. The moderate value of this correlation, combined to the results of the CFA, gave evidence for discriminant validity of the AIMS. However, as a final proof, we calculated the covariance score between the AIMS and the CSS obtained from the Discriminant model, which yielded a r = .40, 95% CI [.26, .54], well below the suggested value of .8, above which there might be discriminant validity problems (Rönkkö & Cho, 2022).

Criterion validity - predictive validity

We performed a logistic regression using the R package car (Fox & Weisberg, 2019) with Donation as our dichotomic dependent variable and AIMS score as our predictor.

The model (χ²(1) =18.98, p < .001) showed that AIMS did predict donation behavior (b = 0.58, SE = 0.14, z = 4.20, p <. 001). Specifically, AIMS scores 1 sd below the mean corresponded to 62% probability of donating, while AIMS score 1 sd above the mean corresponded to only 40%, suggesting that pro-mafia attitudes corresponded to a lower probability of donating to an association against mafias.

Crucially, to discriminate the predictive validity of the AIMS from the CSS, we performed a logistic multiple regression with Donation as our dichotomic dependent variable and AIMS and CSS scores as our predictors. Results indicated that - when CSS was held constant- the AIMS still predicted whether participants donated their reward (b = −0.60, SE = 0.15, z = 3.92, p <. 001), but this did not hold true for the CSS (b = −0.06, SE = 0.15, z = −0.41, p = .68).

Measurement invariance

Since experiences with mafias vary across different territories, it is reasonable to assume that individuals from regions where the presence of mafias is more tangible may have a different social representation of the phenomenon compared to those from areas where mafias are less pervasive. Indeed, mafias are influenced by cultural and social elements that are specific to the regions where they originated (see Travaglino et al., 2014; Travaglino, Abrams & De Moura, 2016). In this vein, certain situations described in the scale items may be perceived differently in different regions, potentially impacting the structure of the scale itself. Thus, to deal with this issue, we tested measurement invariance between participants from the five DRMR (i.e., Apulia, Basilicata, Calabria, Campania and Sicily) and those from the rest of Italy. Due to the high number of parameters to be estimated and since some response categories in some items had no observations, we merged 6 and seven response categories into the same category to obtain a 6-point response scale (instead of the original 7-point response scale). Although treating ordinal data as continuous can be an acceptable alternative when having more than five response categories (Distefano & Morgan, 2014), at this stage we decided to respect its original nature and collapse response categories, which seems a more valid approach when dealing with sparse data or small frequency cells (Distefano, Shi & Morgan, 2021). We then performed a multi-group CFA using the diagonally weighted least squares (DWLS) estimation method. Following Svetina, Rutkowski & Rutkowski (2019) procedure for categorical variables, we started by testing the configural invariance by fitting a baseline unrestricted model, which showed a very good fit (χ² = 293.45, df = 234.00, comparative fit index [CFI] = 0.995, Tucker-Lewis index [TLI] = 0.994, standardized root mean square residual [SRMR] = 0.063, root mean square error of approximation [RMSEA] = 0.036). Then, we fitted a model where we forced the thresholds to be equal in the two groups, and this did not significantly decrease the goodness of fit compared to the baseline model (χ² baseline = 293.45, χ² thresholds = 312.13; p = .63). Finally, we fitted a model where both thresholds and loadings were set to be equal between the two groups and compare its fitness to the thresholds only model. Again, results showed no significant decrease in goodness of fit (χ² thresholds = 312.13, χ² loadings = 386.71; p = .08), which confirmed that the AIMS measured the latent construct in the same way among participants from the five DRMR and those from the rest of Italy.

Comparing AIMS scores between deep-rooted mafia regions and rest of Italy

Once established the measurement invariance, we explored if participants from the five DRMR and those from the rest of Italy differ in their AIMS scores. Indeed, parallel to the purported differences in item perception that could impact the structure of the scale, participants’ attitudes may depend on their familiarity with contexts characterized by mafia’s pervasive and tangible presence (as in the DRMR) or contexts where this presence is comparatively more subtle or even imperceptible.

We used JASP (Version 0.16.3; JASP Team, 2022) to perform a t-test on participants’ mean scores (AIMS means scores), obtained by the widely used procedure of averaging the responses to the items of the scale. Here we reported the analyses and the results on the scores derived from the model built for testing measurement invariance, where we reduced the response categories from the initial seven to six. We performed the same analyses also on the scores based on seven response categories, yielding the same pattern of results (see Supplementary materials).

Moreover, given the exploratory nature of these analyses, we applied both a frequentist and a Bayesian approach to test the robustness of the results and to provide a quantification of the evidence in favor of the hypothesis that there would be a difference between the two samples over the hypothesis that there would not be any difference.

Finally, to strengthen our results, we performed the same analysis also on participants’ latent general factor scores (AIMS fscores) obtained from the CFA, a method that weighs the contribution of each item to the scale. Nonetheless, it has been observed that, if the scale tends toward unidimensionality (such as our bifactor solution), using averaged (or summed) scores might be even better. Indeed, when the scale is unidimensional these scores are less sensitive to specific characteristics of the sample recruited (Widaman & Revelle, 2023). Analyses of the AIMS fscores are reported in the Supplementary materials.

Frequentist approach

Since it is common practice to treat aggregated scores deriving from an ordinal Likert scale with five or more response categories as continuous and perform parametric analyses (e.g., Norman, 2010), we performed a parametric t-test on AIMS scores.

Results on AIMS scores revealed a significant difference t(391) = −2.14, p = .03, Cohen’s d = −.22, 95% CI [−.41, −.02]), with more negative attitudes towards mafias expressed by the DRMR participants (M_DRMR = 2.43) compared to those from the rest of Italy (M_Rest = 2.60).

Bayesian approach

We performed two Bayesian independent sample t-tests on the same variables with a Cauchy prior distribution set at the default scale value of 0.707. Contrary to the results obtained with the frequentist approach, the analysis on AIMS mean scores suggested that we do not have enough evidence (BF₁₀ = 1.02) to support a difference between DRMR participants (M_DRMR = 2.43, 95% CI [2.32, 2.54]) and those from the rest of Italy (M_Rest = 2.60, 95% CI [2.49, 2.70]). The error percentage was equal to 0.02%, which indicated a good stability in the algorithm that was used in this analysis. Moreover, across a wide range of widths the Bayes factor appeared to be relatively stable, ranging from 0.55 to 2.08.

Finally, we tested if the two groups differ for other relevant demographic characteristics: we found no differences in terms of age (t(388.86) = 1.21, p = .22; M_DRMR = 28.24, M_Rest = 27.23), socio-economic status (t(384.16) = −0.73, p = .46; M_DRMR = 44.02, M_Rest = 45.33) or education (χ²(4) = 5.70, p = .22). Conversely, they differ for their income(χ²(5) = 19.76, p = .001), with participants from the rest of Italy being richer than those from the five DRMR regions.