Text-mining forma mentis networks reconstruct public perception of the STEM gender gap in social media

Towards a cognitive approach to information processing

Although Twitter messages provide several types of information, such as visual cues (e.g., pictures and videos) or multi-language cues (e.g., emojis and hashtags), their nature is mainly textual (Welles & González-Bailón, 2020; Jansen et al., 2009). In this way, the problem of quantifying how an audience perceives a given topic can be related to stance detection from their tweets (Mohammad, 2016). Approaching stance detection through human coding becomes quickly intractable when faced with the large volumes of messages exchanged daily on the Twittersphere. This limitation leads to the above need for developing and adopting efficient techniques for knowledge extraction from massive amounts of text as exchanged on social media. Given the extraordinary possibility for humans to communicate their mental constructs through language (Aitchison, 2012), a key way of detecting perceptions is through communication. Both in computer science and linguistics, the problem of detecting positive or negative perceptions from language is known as stance detection (Mohammad, 2016). Rather than focusing on language in itself, this work shifts the attention to the cognitive reflection of language in the human mind, in the so-called mental lexicon (Malt, Wolff & Wolff, 2010; Aitchison, 2012; Dóczi, 2019; De Deyne et al., 2019). Such a lexicon includes semantic memory, a well-studied repository of conceptual meanings and word features (Kenett et al., 2017; Dóczi, 2019), and also other memory supports storing syntactic/phonological/orthographic and even affective knowledge, together with other aspects of language (Dóczi, 2019), whereas syntax, phonology and orthography express word-level conceptual knowledge (Aitchison, 2012), affective knowledge links concepts to the emotions they elicit (Malt, Wolff & Wolff, 2010); therefore, it represents an important component of any stance/perception.

Harnessing the complex cognitive structure of the mental lexicon means accessing how conceptual knowledge is structured and emotionally perceived by individuals as the outcome of their previous experiences and current attitudes (Aitchison, 2012; Stella et al., 2019; Stella, 2020). In other words, accessing the conceptual and affective representation of knowledge in the mental lexicon means reading minds(ets), with strong repercussions for information processing. Discovering how an audience perceives a given topic, i.e., performing mindset reconstruction, can provide crucial knowledge for understanding and intervening upon specific trends (Welles & González-Bailón, 2020; Amancio, Oliveira Jr & Costa, 2012; Stella et al., 2019). An example is represented by the finding that mindsets vehiculating positive emotions reach larger audiences on social media whereas negative emotional content can spread at faster rates (Ferrara & Yang, 2015). Another example for the relevance of mindset reconstruction is uncovering and acting upon traces of science anxiety in student populations in order to improve their learning experiences (Stella, 2020; Stella & Zaytseva, 2020) or detecting sexual harassment through large-scale web surveys (Karami et al., 2020).

A key area where mindset reconstruction is particularly promising is understanding the sources and dynamics of the gender gap in science (Hogue & Lord, 2007; Moss-Racusin et al., 2012).

As outlined by recent Big Data Analytics studies considering decades of scientific careers (Huang et al., 2020; Odic & Wojcik, 2019; Chavatzia, 2017), gender disparities in science cannot be explained by intrinsic differences in attitudes to science between genders but rather have to be traced in the establishment of implicit gender biases promoted by news media and by social media representations of science (Shapiro & Williams, 2012; Moss-Racusin et al., 2012; Madsen & Andrade, 2018; Steinke, 2017). In this way, the reconstruction of mindsets vehiculated by information systems becomes a key point for understanding, acting upon and closing the gender gap in science. Given the recency of the above mentioned Big Data Analytics studies and the methodological issues in reconstructing mindsets with black-box machine learning techniques (often neglecting contextual information;  Nasar, Jaffry & Malik, 2019), reconstructing online perceptions of the gender gap in social media is a challenge not fully explored yet.

Research aim

This work introduces textual forma mentis networks (TFMNs) as quantitative tools for reconstructing the mindset of online users engaging in social discourse with the research aim of investigating in detail the mindset emerging from user-generated content about the gender gap in science. In other words, this work uses textual forma mentis networks with the aim of reconstructing how online users discussed and perceived messages revolving around the STEM gender gap. In order to mirror the general perception of this gap, a window without special events about women in science was chosen. TFMNs quantified the general stance from 10,000 tweets publicly available on Twitter, produced between October 8 2019 and October 22 2019, containing the words or hashtags “science” or “stem” and “women” or “gendergap”. These tweets encapsulated information about how the online authors perceived women in science, with relevant impact for education research and the computational social sciences.

Before providing additional details about the adopted methodology, it is necessary to compare TFMNs against past approaches, also relating forma mentis networks within the relevant literature about the gender gap in science.

Literature review on relevant past approaches

It has to be underlined that only recently automatic text-mining studies started investigating gender gap in online discourses (Teso et al., 2018; Chavatzia, 2017). However, these approaches mainly focused on detecting differences in language use among different genders. Differently from other information processing investigations aiming at identifying emotions on social media in relation to phenomena like hateful speeches (Waqas et al., 2019) or disinformation spreading (Pierri, Artoni & Ceri, 2020), gender-focused investigations of social media did not explore large-scale mappings of the online perception of the gender gap in science as embedded in the messages exchanged between social users of any gender. Within an information management setting, the work closest to an investigation of the overall mindset about gender biases was the study by Karami and colleagues (Karami et al., 2020). The authors investigated online self-reports of sexual harassment experiences and through a topic analysis they highlighted evidence for sexual harassment in academia mainly targeting women and involving coercion, gender discrimination and retaliation. Building upon the knowledge extraction approach of Karami et al. (2020), this study shifts its attention from explicit sexual harassment to the larger topic of gender biases in science, which includes harassment itself but also implicit biases (Shapiro & Williams, 2012), gender pay gaps (Courey & Heywood, 2018) and stereotypical perceptions about leadership (Pennington et al., 2016; Ely, Ibarra & Kolb, 2011). Furthermore, rather than focusing on self-reports, this study aims at tackling a different information system, namely Twitter, where social users can engage in social discourse and reach large audiences (Welles & González-Bailón, 2020). This reconstruction of social media attitudes about the gender gap in science is the main gap that the current work aims to fulfil by using a network-powered approach, which has several similarities and novelties in comparison to past network frameworks.

Within the literature of network-powered analyses of large volumes of text (De Arruda et al., 2019; Koponen & Nousiainen, 2019), previous works successfully used word co-occurrences in text (e.g., “like” and “stem” occurring one after the other in text, cf. Cancho & Solé, 2001) in order to characterise language content through average statistical markers (Amancio, Oliveira Jr & Costa, 2012; Amancio, 2015) or time-evolving dynamics (Akimushkin, Amancio & Oliveira Jr, 2017). These networks were considerably powerful at a global level and very successful in tasks like author identification (Amancio, 2015; Akimushkin, Amancio & Oliveira Jr, 2017; De Arruda et al., 2019). However, the validity of co-occurrence networks as representations of the mental lexicon at the microscopic level of individual conceptual associations has been recently reconsidered (Ninio, 2014; Rizvi, 2018). In fact, co-occurrences can lead to spurious conceptual links, whose influence vanishes in global statistical approaches and that do not represent syntactic similarities in text (Ninio, 2014). In order to overcome this limitation and achieve a more faithful microscopic representation of the mental lexicon, TFMNs directly harness the full ensemble of syntactic associations of a sentence (e.g., “like” being a verb referring to the object “stem”, cf. Cancho, Solé & Köhler, 2004) and enrich them by considering also semantic overlap between words (e.g., “appreciate” and “like” being synonyms across different sentences, cfr. Miller, 1998; Amancio, Oliveira Jr & Costa, 2012).

Textual forma mentis networks automatically extract these conceptual associations from text without requiring human supervision and are therefore suitable for processing large volumes of text. The resulting network structure is informative of the cognitive layout of conceptual associations emerging from a given textual corpus and hence represents how text authors organised, structured and associated microscopically their knowledge around topics and concepts. This makes TFMNs “glass boxes” (Nasar, Jaffry & Malik, 2019), where the knowledge structure of a certain stance can be accessed and directly read, differently from previous “black box” machine learning approaches which accurately reproduced the positive or negative nature of a stance without providing information on its semantic content (Mohammad, 2016; Teso et al., 2018; Rudkowsky et al., 2018; Nasar, Jaffry & Malik, 2019). In addition to conceptual associations, TFMNs are endowed also with sentiment labels, indicating the sentiment (Warriner, Kuperman & Brysbaert, 2013) and the basic emotions (Ekman & Davidson, 1994; Mohammad & Turney, 2013) elicited by a given concept in a population of individuals involved in behavioural studies. Sentiment scores of positive/negative affect are also called word valence in psycholinguistics (Warriner, Kuperman & Brysbaert, 2013; Recchia & Louwerse, 2015) and represent how positively or negatively a given concept was perceived in a behavioural study. Word valence and emotional profiles add more emotional contextual information about the stance reconstructed by conceptual associations (Stella, 2020). Although large-scale datasets about sentiment and emotions have only been recently made available to the scientific community by cognitive studies (Warriner, Kuperman & Brysbaert, 2013), they have quickly become predominant in predicting a wide variety of human behaviour (Li, Baucom & Georgiou, 2020) and information processing patterns such as consensus formation in social networks (Konstantinidis, Papadopoulos & Kompatsiaris, 2017) or information sharing on microblogs (Ferrara & Yang, 2015). The combination of network patterns and sentiment data is important, as considering only the frequency of sentiment-labelled content in short texts has been reported to lack interpretative contextual power for estimating how people really feel about a given topic (Beasley & Mason, 2015).

Different types of forma mentis networks

As outlined above, text-based forma mentis networks represent multiplex lexical networks (Stella et al., 2018) of concepts interconnected through syntactic and semantic associations and enriched with sentiment and emotional labels. Previous works showed how lexical networks of concepts including multiple layers of associations (i.e., “multiplex”) were better than single-layer complex networks at predicting a variety of cognitive processes involved in information acquisition (Stella et al., 2018; Stella, 2019) and search (Castro & Stella, 2019; Siew et al., 2019). However, these approaches did not explore the influence that emotions and sentiment can have over information processing (Warriner, Kuperman & Brysbaert, 2013; Dóczi, 2019; Rudkowsky et al., 2018; Li, Baucom & Georgiou, 2020). The introduction of behavioural forma mentis networks (BFMNs) (Stella et al., 2019; Stella & Zaytseva, 2020) tackled such a gap by combining conceptual associations with sentiment scores and providing access to knowledge structure and positive/negative perceptions of concepts in a given mindset. However, mindset reconstruction of BFMNs comes at the cost of involving individuals in a cognitive experiment (cf.  Stella et al., 2019). This limitation translates into the possibility of reconstructing only the mindsets of those who participated in the experiment, preventing the monitoring of remote systems like social media. Textual forma mentis networks do not require behavioural experiments but can be built starting from any written text, instead. As a consequence, TFMNs are suitable for the investigation of speech over online platforms, whereas BFMNs are unstuitable for such task. However, as outlined above, both TFMNs and BFMNs exploit the theoretical framework of mental lexicon representations (Dóczi, 2019) and language processing in cognitive network science (Siew et al., 2019). Relying on these foundations, TFMNs are applied here for reconstructing the mindset of online discourse about the gender gap in science, as motivated in the next section.

Recent literature identifies a link between the gender gap and distorted mindsets

Overwhelming evidence indicates that the gender gap in science is a complex phenomenon deeply affecting society, economics and science advancement (Ely, Ibarra & Kolb, 2011; Madsen & Andrade, 2018; Hogue & Lord, 2007; Pietri et al., 2018). The gender gap in the scientific, technological, engineering and mathematical (STEM) disciplines is a disparity of how different genders enter in and progress through a career in science (Shapiro & Williams, 2012). Between 2014 and 2016, UNESCO estimated only around 30% of all female students in higher education enrolled in STEM-related fields of study at University level (Chavatzia, 2017). This gender gap in STEM education and participation is almost absent at the level of primary education but then becomes particularly apparent during upper secondary education, coincidentally with subject selection, and it gets worse at higher levels of education (Ely, Ibarra & Kolb, 2011; Chavatzia, 2017; Hogue & Lord, 2007).

Globally, only 28% of all the world’s researchers are women. This disparity is deeply embedded in educational systems, in particular in terms of attitudes and perceptions towards science (Shapiro & Williams, 2012; Hogue & Lord, 2007; Huang et al., 2020). The PISA 2015 report Excellence in Education reported that across 35 OECD countries, only 22% of 15-years old girls intend to pursue a career in STEM, less than half the proportion (48%) of STEM driven 15-years old boys (cf.  Chavatzia, 2017). In the last few years, a great attention has been devoted to explaining such gap by using either external, e.g., system embedded discrimination (Chavatzia, 2017; Ely, Ibarra & Kolb, 2011), or internal factors, e.g., implicit stereotypes (Shapiro & Williams, 2012; Pietri et al., 2018). Studies trying to explain gender gap through gender-based learning achievements reported a complex landscape, with boys and girls being more or less proficient than each other according to the task being measured (cf.  Chavatzia, 2017). This complex mosaic of multiple findings makes it complicated to state that any gender is more or less advanced in any given STEM subject: the gender gap has to be rooted in more subtle forms of discrimination, stereotypes and perceptions (Moss-Racusin et al., 2012; Lane, Goh & Driver-Linn, 2012). Despite this fragmentation, the data unanimously indicate that women are paid less and leave STEM careers way more than their male colleagues either at the University level or at subsequent stages of professional growth in science (Courey & Heywood, 2018).

Even professional impact in STEM is influenced by a strong gender gap. The recent longitudinal study by Huang and colleagues (Huang et al., 2020) considered professional impact of women in STEM through bibliographic Big Data spanning over 60 years of publications. The authors reported evidence for gender differences in the cumulative productivity of research output but not in the annual rate of publication or career-wise impact. Through a quantitative, longitudinal analysis, Huang and colleagues related such gender gap to dropout rates and differences in length of publishing careers between men and women. The authors could not explain such disproportion only in terms of intrinsic gender-based proficiency in STEM, thus providing additional large-scale evidence for the presence of strong contextual factors affecting everyone’s experience of the STEM gender gap. Analogous differences through Big Data approaches were recently found also by Odic and Wojcik in psychology, a field where three in four students are women whereas three in four academic professionals are male (Odic & Wojcik, 2019).

The need to expose distorted mindsets about the gender gap in information systems

The predominance of strong academic biases in contrast with educational patterns suggests the presence of hidden roots to the STEM gender gap, as indicated by several independent studies on the topic (Pennington et al., 2016; Shapiro & Williams, 2012; Pietri et al., 2018). Hence, understanding the overall experience and subsequent perception of this gender gap in a large audience can provide key elements for better detecting the presence of potentially subtle yet strong gender-based stereotypes, as perceived and communicated by the main actors of STEM. Given that most of these stereotypes can act at a subconscious level and take place without the explicit awareness of those perpetrating them (Pennington et al., 2016; Madsen & Andrade, 2018; Lane, Goh & Driver-Linn, 2012), detecting the presence of such distorted perceptions remains a difficult challenge.

Monitoring and detecting the diffusion of stereotypical endorsements in social media represents an important way of better understanding and countering gender discrimination in science (Karami et al., 2020), especially in the current society dominated by virtual social media (Welles & González-Bailón, 2020; Jansen et al., 2009).

Manuscript organisation

This manuscript is organised in several subsections. The “Methods” section contains quantitative details about the implemented methodology and analysed data. The “Results” section is split in two subsections. The first part is a quantitative benchmark, reporting on the effectiveness for TFMNs in finding semantically relevant concepts and topic features in short texts annotated by authors. The second part explores the reconstructed mindset of online users towards the gender gap by considering the stance towards key domains and aspects of the gender gap, e.g., concepts like “woman”, “man”, “person”, “scientist”, and related aspects, e.g., “gender”, “gap” and “stem”. The detected conceptual associations and emotional patterns are investigated and related with previous studies within the “Discussion” section.

Twitter dataset

The main dataset used in this investigation was a collection of 10,384 tweets publicly available on Twitter and produced between October 8 2019 and October 22 2019. Tweets were gathered through the ServiceConnect[] function for Twitter crawling implemented in Mathematica 11. Crawling was performed in accordance with Twitter’s policies via the account of Complex Science Consulting, which received authorisation by Twitter for research-focused text mining. Tweets were included in the dataset if they contained either the hashtags (or the words) “#science” or “#stem” and “#women” or “#gendergap”. This means that the combinations required for the twitter query were: (i) “science” and “women”, or (ii) “science” and “gendergap”, or (iii) “stem” and “women”, or (iv) stem and “gendergap”. The query service implemented in Mathematica works analogously to the search service implemented on Twitter, which can search tweets both at the start of conversations or in middle of them as long as they contain the above hashtags or words. Searching for either hashtags of words and including only two requirements for query made the search itself flexible in identifying a variety of tweets over such a specific topic, in a time window outside of specific events focusing over the gender gap.

Users’ replies and mentions containing the same keywords as above were included in the analysis. However, threaded tweets or users’ replies and mentions not including the above keywords were not added to the dataset. This choice made the dataset more focused over individuals’ original statements about the gender gap, giving more voice to those users starting a conversation (with the above considered keywords) in order to express their points of view. This focus was motivated by the cognitive grounding of TFMNs, which combines linguistic knowledge and emotional perceptions as directly coming from people’s mindsets/mental lexica, without strong influences from other people’s behaviours or stances that might be present in users’ replies (Tagg, 2015).

Twitter’s policy prevents the redistribution of these tweets outside of the Twitter platform. Nonetheless, for the sake of scientific reproducibility, the IDs of these tweets were attached to this manuscript as Supplementary Information.

Each tweet contained a short text. Pictures and emoticons were discarded. Hashtag characters were removed. Tweets with less than three words were not included in the analysis but constituted less than 1% of the whole dataset.

Network construction

Text-based forma mentis networks were built in three different stages:

1. Extraction of syntactic relationships between concepts/words from a sentence;

2. Addition of semantic relationships (synonyms) between concepts as indicated by an external dataset;

3. Addition of valence labels (“positive”, “negative”, “neutral”) to each single concept as indicated by another external dataset.

The above three steps were repeated for all the sentences in a given tweet. At the end, all connections between the extracted and labelled concepts formed a multiplex lexical network (Stella et al., 2018) where nodes represented words/concepts and were connected across multiple linguistic layers, namely: (i) a semantic layer indicating meaning overlap between words (e.g., “famous” and “notable” sharing the same meaning), and (ii) a syntactic layer indicating dependencies in meanings as encapsulated within a sentence. In general, syntactic structure defines how entities (e.g., nouns) are specified in a given sentence through verbs, determiners, prepositions, adjectives and adverbs (Cancho, Solé & Köhler, 2004). Although more or less convoluted syntactic structures can be encoded in sentences, the simplest form of the syntactical dependencies is a subject being specified as an object. For instance, in the sentence “love is weakness”, “love” is specified as “weakness” through the verb “is”. The verb “is” does not encapsulate any intrinsic meaning but it can be replaced by a link between “love” and “weakness”. Syntactic dependencies can be more general, for instance in the sentence ”the cat sat on the chair” the nominal subject “cat” is linked to the object “chair” through the verb “to sit”, which retains some meaning. The determiner “the” and the preposition “on” do not retain meaning by themselves but connect the other parts of speech and are thus essential for extracting syntactic dependencies. The specification of such dependencies was implemented through the TextStructure[] command in Mathematica 11, which produces all syntactic relationships between the parts of speech of a given sentence. From the resulting network of directed dependencies, prepositions and auxiliary verbs were removed as nodes and replaced by syntactic links. In order to avoid the inflection of lemmas with the same meaning, i.e., having “weak” and “weakness” in the same network, word stemming was performed at the network level.

After the extraction of syntactic dependencies, the resulting syntactic network identified a set of connected, stemmed concepts. Syntactic structure can be informative about conceptual links in the mental lexicon and be predictive of language learning and processing (Cancho, Solé & Köhler, 2004; Stella et al., 2018). However, syntactic dependencies neglect the possibility of using and exchanging synonyms in the same syntactic structure providing the same meaning (e.g., saying ”he has a quiet character“ conveys the same meaning of saying ”he has a quiet nature”). In order to account also for meaning overlap, the syntactic structure was enriched with synonym relationships from WordNet 3.0 (Miller, 1998), as implemented in the curated repository WordData[] available in Mathematica 11. Meaning overlap between concepts is also representative of semantic memory patterns in the mental lexicon, with previous works showing how synonym networks can predict language learning and facilitate conceptual navigation (Stella et al., 2018; Siew et al., 2019). Notice that in this analysis, “word” and “concept” are used interchangeably.

After the construction of the network layers of syntactic relationships and of synonyms, every concept was connected either by syntactic or semantic links. From the affective mega-study of Warriner, Kuperman & Brysbaert (2013), each concept was endowed with a valence label, e.g., “positive”, “neutral” or “negative”. These labels represented the average valence attributed to each concept by a population of individuals involved in a large-scale behavioural study rating more than 13,900 English words (Warriner, Kuperman & Brysbaert, 2013). Words were classified as positive, neutral or negative, respectively, according to their location in the upper quartile (higher valence), interquartile range (neutral valence), or lower quartile (lower valence) in the distribution of valence scores for over 9,000 different word stems. These valence scores were obtained from averaging over words with the same stemmed root.

A single TFMN was made by considering all 10k tweets together, with syntactic links bridging words within the same sentence. Since the same words could be repeated across sentences, the overall syntactic network bridged different sentences together, either from the same or from different tweets.

Figure 1A provides an example of a text-based forma mentis network. A TFMN can be represented either as an edge-coloured graph or as a multiplex network (Stella et al., 2018). In network science, multiplex networks contain different sets of relationships/associations between elements of the same group, e.g., the same set of words being linked either through synonyms or syntactic dependencies. All the links expressing a certain type of relationship form a so-called network layer. A multiplex network is then a collection of multiple network layers, each one featuring replicas of the same set of nodes, e.g., see Fig. 1A. In edge-coloured graphs, replicas are aggregated together and all different relationships are expressed on one layer, e.g., see Fig. 1A. In TFMNs, where there is no explicit transition between syntactic and semantic replicas of the same word/node, these two representations are equivalent and their only purpose is to distinguish between syntactic and semantic links.

Figure 1A provides a pictorial representation of the way a TFMN is built from text. The starting point is the assumption that knowledge and emotions about the real world are encoded in people’s mental lexica. Individuals express their opinions, combining knowledge, sentiment and emotions, through language. TFMNs are based on the syntactic/semantic links between words as extracted from text. The resulting network structure identifies the organisation of conceptual associations as expressed by online users in social discourse. Network neighbourhoods correspond to the syntactic/semantic frame of concepts associated to a given word and investigated by the so called semantic frame theory (Fillmore, 2006). Analysing the associates of a concept gives information about: (i) how it was framed in terms of associated meanings, (ii) how it was perceived in terms of valence and emotions. With reference to Fig. 1B, a concept linked to mostly positive concepts acquires by association a positive connotation in text, which corresponds to a positive aura on the TFMN. In the following, a valence aura will identify the most frequent sentiment label (e.g., positive, negative, neutral) attributed to the semantic associates of a concept, analogously to previous studies (Stella et al., 2019; Stella, 2020). Also emotional auras will be considered, indicating the contextual feelings and emotions elicited by the conceptual associates of a word in a given semantic frame.

Additional cognitive data

In order to test the power of text-based forma mentis network in identifying relevant concepts in text, additional cognitive data was used as benchmark. Semantic similarity and relevance of scientific concepts, as extrapolated from short scientific texts, was investigated and compared against free association data from the Complex Forma Mentis project (Stella et al., 2019). The benchmark text adopted here was the booklet “Complexity Explained”, co-authored by several researchers in complexity science and composed of 7 short paragraphs describing one specific concept each (cf. https://complexityexplained.github.io/, Last Accessed: 16/03/2020). The analysed paragraphs were about: “interactions”, “emergence”, “dynamics”, “self-organisation”, “adaptation”, “interdisciplinarity” and “methods”. The results of such benchmark are reported at the beginning of the Results section.

Emotional profiling and emotional flowers

Compared to previous approaches using behavioural forma mentis networks (Stella, 2020; Stella & Zaytseva, 2020; Stella et al., 2019), the current analysis introduces also an emotional profiling of stances as encapsulated in the reconstructed mindsets. Emotional profiling is defined in terms of considering how many associates of a concept, in the TFMN, elicit a given emotion. Concepts linked to a negation (e.g., “not”) were transformed into their antonyms, so that both original concepts and their negated meanings were considered for reconstructing an emotional profile. Considered emotions included:

• Anger, a negative emotion representing reactions of irritation and rage towards an external threat;

• Disgust, a negative emotion indicating aversion and closure;

• Fear, a negative emotion indicating a need to actively avoid and prevent potential threats;

• Trust, a positive emotion indicating openness towards the outer world;

• Joy, a positive emotion of excitement and satisfaction;

• Sadness, a neutral emotion, neither positive nor negative, indicating states of sorrow, thoughtfulness and inhibition;

• Surprise, a neutral emotion relative to being upset or startled by an unexpected event;

• Anticipation, a neutral emotion indicating one’s projection into future events, including desire or anxiety.

The above constitute basic building blocks of a wide spectrum of emotional states (Ekman & Davidson, 1994). For a more detailed description of emotions, please see Ekman & Davidson (1994) and Mohammad & Turney (2013). The mega-study by Mohammad and colleagues is also known as the Word-Emotion Association Lexicon, indicating how human participants associated individual concepts to emotional states. In other words, the resulting dataset indicated which emotions were elicited or evoked in human subjects reading individual concepts. The study used crowd-sourcing on Mechanical Turk for achieving a large-scale mapping of English words (including 14,000 unique lemmas) and was used also in other successful investigations about affect patterns (cf. Mohammad & Turney, 2013).

Comparing the emotional richness of different semantic frames is problematic, as those frames might involve different numbers of words. Comparing an observed emotional profile with a null model represents a more appropriate comparison, especially considering that there are more words eliciting fear and joy than anticipation and sadness in the dataset. A statistical comparison can answer questions like “Was trust the most prominent emotion just because there are more words eliciting trust in language?”.

As a null model for comparison with an observed emotional profile, random sampling was used here. Fixing the same number m of words eliciting at least one emotion and observed in an empirical semantic frame, m random samples were selected uniformly at random from the Word-Emotion Lexicon. The emotional profile of this random sampling was then computed and stored. This operation was repeated 1,000 times, obtaining a random distribution of emotional profiles from which mean, standard deviation and z-scores were computed. The expected mean and standard deviations were used in visualising the emotional richness of a semantic frame through a bar plot. Instead, z-scores were organised in a bar sector chart reminiscent of the atlas of emotions (Ekman & Davidson, 1994). In visualisation, the rejection region relative to z < 1.96 is a semi-transparent circle. Bars are shaped like petals, one per emotion, and coloured differently. Those petals going out of the rejection region indicate a semantic frame with jargon eliciting a given emotion more than random expectation (within a significance level of 0.05). Concentric rings indicate units of z-scores after 2. For their resemblance with flowers, these visualisations are called emotional flowers.

Cognitive measures of semantic similarity on language networks

This work built networks of conceptual associations between concepts. Several independent studies have reported that on such networks, metrics like network distance are powerful proxies for quantifying semantic relatedness (Kenett et al., 2017; Stella & Zaytseva, 2020). Network distance d_ij is defined as the minimum number of links (here conceptual associations) connecting any two words i and j on a network structure (Siew et al., 2019). On multiplex lexical networks, links of any colour/layer can be used (Stella et al., 2018; Stella, 2019). Measures based on network distance such as closeness centrality have been found to identify also concepts of relevance from a cognitive perspective when detecting key words for word learning (Stella et al., 2018; Stella, 2019), language processing (Kenett et al., 2017; Siew et al., 2019; Castro & Stella, 2019) and knowledge exploration (Stella & Zaytseva, 2020; Akimushkin, Amancio & Oliveira Jr, 2017; Amancio, 2015). Closeness centrality (Siew et al., 2019) c(i) is attributed to node i by checking how distant it is to its connected neighbours, in formulas: (1)${\displaystyle c\left(i\right)=\frac{N}{\sum _{j=1}^N{d}_{ij}}.}$

Notice that the above formula applies to fully connected components of a network but does not enable direct comparison of components including different numbers of nodes (e.g., a component including only 5 nodes versus a component with 1,000 nodes). Remember that TFMNs are multiplex but do not feature any weight or inter-layer links, hence for them multiplex closeness is equivalent to the single-layer closeness on the aggregate network obtained by considering all edge types as equivalent, as considered in previous cognitive approaches (Stella et al., 2018). Therefore, multiplex closeness on the TFMN considers both syntactic and semantic links when outlining shortest paths between concepts.

Forma mentis networks included also structural features of the mental lexicon of individuals combined with affective patterns (Warriner, Kuperman & Brysbaert, 2013). This unique combination allowed for Stella et al. (2019) to introduce the network metric of valence auras, i.e., the mode of valence labels in a given neighbourhood. In Stella et al. (2019) a word was defined as having a positive (negative) valence aura if mostly linked to positive (negative) words. This measure was used also in the current analysis, although it has to be underlined that in here these affect measures represented how a large-scale population, independent from the one producing the texts, perceived individual concepts. Since valence auras still depended on the connectivity of conceptual associations as assembled by text authors, this metric quantified how groups of these authors structured and perceived their knowledge. In the current analysis, valence auras rather than single-word valence labels were key to reconstructing positive/negative perceptions of individual concepts by checking their conceptual associates in the analysed online discourse.

Benchmark of text-based forma mentis networks on short texts

This subsection reports results of the benchmarking analysis of “Complexity Explained” (see Methods) through TFMNs. For each of the seven paragraphs of the booklet a TFMN was built. Closeness centrality was used for identifying the most central concepts in every TFMN and produce rankings of the most relevant words in each text. Overall, the resulting forma mentis network contained a median of 49 concepts and were fully connected. Table 1 reports the 10 most central words in each network and their underlying topic.

Table 1 indicates that, beyond an overall agreement between key concepts and topics, as identified by TFMNs, the network topology of syntactic/semantic associations identified the most distinctive conceptual features of topics. For instance, while the ranking for the topic “Interactions” reported mainly concepts relative to structure (e.g., component, part, whole, consist), the ranking of “Dynamics” identified concepts related to the evolution over time of a system (e.g., change, state, behaviour, dynamic). Words expressing resilience and robustness to attacks, like “damage”, “function”, “evolve”, “adapt” and “robust”, were found to be central in the textual forma mentis network of “Adaptation”. TFMNs detected “domain” as being a relevant concept in the “Interdisciplinarity” paragraph. This was expected in a text describing complexity science as an umbrella for different research areas. The “Methods” paragraph revolved, as expected, around quantitative concepts, e.g., forecast, compute, model, method, analysis, rule. In the same paragraph, mathematics was found to be highly relevant, in agreement with the overall necessity of a mathematical language for investigating complex systems (Cancho, Solé & Köhler, 2004; Siew et al., 2019).

Although the above qualitative analysis indicated an overall agreement between topics and identified key concepts, a more quantitative approach was further pursued. With the aim of assessing whether the identified concepts were more or less semantically related to the topics designed by text authors, free associations and semantic network distance were adopted. In networks of free associations, nodes/words are linked if they elicit a quick recall of each other in a behavioural task (De Deyne et al., 2019). Previous studies have shown how semantic network distance on networks of free associations are a good proxy of semantic relatedness, superior also to semantic latent analysis (Kenett et al., 2017). Although several datasets for free associations are available in the literature, this benchmark considered two: the large-scale Small World of Words gathered by De Deyne et al. (2019) and the small-scale Complex Forma Mentis project gathered by Stella et al. (2019). Many of the scientific terms present in Complexity Explained were absent in the Small World of Words but present in the STEM free associations provided by Stella et al. (2019). Therefore, this analysis focused on the second dataset of free associations.

The network distance between every relevant word and its reference topic was computed for all paragraphs within the the network of free associations by Stella et al. (2019), i.e., a behavioural forma mentis network based on the collective mindset around science of 59 complexity researchers. Hence, relevant words were extracted from textual forma mentis networks (of syntactic/semantic relationships). The semantic proximity of these relevant words with their reference topic was tested on a behavioural forma mentis network (of free associations). The reference topics were “interaction”, “emergence”, “dynamics”, “self-organisation”, “interdisciplinary” and “methods” and the relevant concepts were the ones in Table 1. The resulting set of empirical semantic network distances was then compared against a reference null model with randomised TFMNs having the same number of links and nodes of the original networks but with randomly reshuffled links (i.e., configuration models Stella et al., 2018). The reshuffling disrupted semantic relationships between network structure and meaning (Stella et al., 2018). A location test between the empirical and the randomised network distances (over 50 network realisations) indicated a statistically significant difference between the clustering of relevant words around each topic and the null model (Mann–Whitney test, U = 17827, p-value: 0.0267 < 0.05) at a significance level of 0.05. In the networked free associations representing the scientific knowledge of complexity researchers, the words identified as relevant for a topic by TFMNs tended to be closer to their own topic (median network distance: 3.1) than on the randomised networks (median network distance: 3.7). Since semantic network distance on networks of free associations indicates semantic relatedness (Kenett et al., 2017), these results confirmed that closeness centrality on TFMNs was capable of identifying concepts relevant to the specific topic underlying a given text.

Given the successful outcome of such benchmark, TFMNs can therefore be applied to extracting relevant features of other short texts. The following section reports on the results of TFMNs when used for analysing over 10,000 tweets about gender gap in science.

Analysis of the perception of gender gap in STEM with cognitive networks

The textual forma mentis network obtained from processing the selected tweets included a largest connected component including 3,005 stemmed concepts and 28,004 connections (24,693 in the syntactic layer and 3,311 in the synonyms layer). The 10 most central concepts in terms of closeness centrality were: “STEM”, “science”, “we”, “do”, “you”, “learn”, “make”, “get”, “work”, “need”. Notice that closeness centrality captures semantic prominence, i.e., a concept being semantically related to all other concepts connected to it. The identification of “STEM” and “science” as semantically prominent in the dataset confirms the scientific scope of the latter. This finding importantly indicates that TFMNs are able to identify semantically prominent concepts not only in well-ordered, short texts (see the benchmark) but also in the collection of short texts represented by tweets. This should not be given for granted, since tweets also include hashtags, emojis and linguistic abbreviations that do not appear in well-ordered paragraphs and which were not included in TFMNs. Despite this limitation, TFMNs identified words capturing the general context of science of the investigated online discourse. Concepts like “woman” and “man” ranked 103rd and 246th, respectively, mirroring the relevance of women in science for the selected online discourse. “Scientist” ranked higher, at the 54th position, further confirming the scientific scope of the dataset.

Valence auras and global network metrics highlight an overall positive online stance towards STEM and gender gap

The 3,005 stemmed concepts extracted from relevant tweets and connected in the forma mentis network were rated as positive (1045), negative (430) and neutral (1943) according to the procedure described in the Methods. Positive concepts were found to have a higher median degree than negative concepts (Mann–Whitney test, numerosity n₁ = 1045, median degree k₁ = 9, n₂ = 430, k₂ = 4, U = 3⋅10⁶, p < 10⁻⁶). Positive concepts were also found to be more central in the reconstructed mindset in terms of requiring fewer syntactic/synonymy associations in order to reach any other connected concept, i.e., centrality as expressed by multiplex closeness (Stella et al., 2018), (Mann–Whitney test, numerosity n₁ = 1045, median degree k₁ = 0.3601, n₂ = 430, k₂ = 0.3307, U = 3⋅10⁶, p < 10⁻⁶). These comparisons indicate that in the analysed corpus of twitter language focusing on women and STEM, positive concepts were more predominant, more well connected and more central than negative concepts. The richer network structure of conceptual associations of positive concepts translated into a generally positive attitude of twitter users towards the STEM gender gap.

Figure 2A reports the emotional auras (Stella et al., 2019; Stella, 2020) of hub concepts in the reconstructed mindset of social media users. While single-word labels were determined from the sentiment of a general population, i.e., participants in an affect mega-study (Warriner, Kuperman & Brysbaert, 2013), auras emerged from the specific syntactic/synonymy relationships traced in the analysed data and therefore characterised the reconstructed mindset of the population of interest. Concepts like “STEM”, “gender” and “gap” are neutral in commonly spoken language but were associated mostly with positive concepts in the language of the online discourse, i.e., surrounded by a positive emotional aura. This indicates an overall positive perception of these topics that would go undetected when investigating “STEM”, “gender” and “gap” in isolation. Differently from other investigations with forma mentis networks (Stella et al., 2019; Stella, 2020), the sampled online audience reported a strongly positive perception/aura of “student”. Also “bias”, a negative concept, was surrounded by a positive emotional aura, indicating an overall mixed stance trying to figure out positive aspects of a bias (e.g., how to overcome it). Further analysis is required in order to better understand the above perceptions.

Figure 2 reports the mindset structure around “bias” (B) and other concepts like “unique” (C), “hurt” (D), and “journal” (E). The forma mentis neighbourhood of “bias” included associations to positive concepts like “face”, “win”, “clear” and to negative concepts like “fight”, “cost” and “stereotype”, all revolving around contrasting and overcoming biases. Therefore, the reconstructed TFMN indicates how social media users discussed about “bias” in terms of a negative entity to be contrasted, overcome and won, thus explaining the above mixed perception. Other words associated to bias gravitated around the gender pay gap and economic implications (Courey & Heywood, 2018), e.g., “earn”, “cost”, “value”. Links with “unconscious” and “fact” indicate that social media users were aware of hidden, unconscious gender biases affecting the gender gap (Pietri et al., 2018).

Figure 2C also reports how “unique” was associated in the online discourse around the gender gap. The mixed perception of “unique” included negative associations to concepts eliciting loneliness, e.g., “oppress”, “fail”, “alone” and “pretend”, which are unexpected when considering the positive perception of “unique” itself. Hence, the TFMN indicates that when associated in messages focusing on women in STEM, the meaning of “unique” shifted from positive to mixed and included negative connotations of social exclusion and sense of failure. This is an example of contextual valence shifting (Polanyi & Zaenen, 2006), a phenomenon in which a concept changes its valence according to its semantic context. TFMNs represent a valuable quantitative tool for identifying valence shifting through semantic associates and emotional profiling.

The multiplex structure of TFMNs can influence an emotional aura. Synonymy links (green) do not come from text but rather from a pre-determined lexicon of synonyms (i.e., WordNet Miller, 1998) that has general validity over common language. Instead, syntactic (blue/red/gray) associations come from the specifically analysed language. In this way, TFMNs can identify also missing conceptual associations or the predominance of synonymy over syntactic links.

The negative aura of “hurt” in Fig. 2D was mostly due to general synonyms whereas specific syntactic associations included positive concepts like “teach”, “like” or “health”, thus indicating a lack of hurtful/painful conceptual associations in the investigated online discourse over women in STEM. Network structure was informative also about the negative aura attributed to “journal”, which was linked to “politician”, “stereotype”, “bias” and “policy”. These syntactic links indicated a social awareness about journals and news media potentially perpetrating gender stereotypes, a role that was investigated in previous studies (Steinke, 2017).

The online social perception of “woman”, “man” and “person” reconstructed by TFMN

Although it is expected for the online discourse to feature also automatic accounts and social bots, previous research identified human users as being more predominant in driving and diffusing message exchanges on social platforms (Stella, Ferrara & De Domenico, 2018). Comparing how people identify themselves in a discourse where they are the main drivers can be informative about social roles and users’ self-perception (Varol & Uluturk, 2020).

Figure 3 reports and compares the forma mentis network around “woman” (A), “man” (B) and “person” (D). All these three concepts, commonly perceived as positive entities, were surrounded by positive emotional auras in agreement with the overall positive features reported above of the whole TFMN.

The reconstructed social perception of “woman” within the online discourse of gender gap in STEM identified mainly semantic clusters expressing three aspects:

1. leadership and professional recognition in STEM (e.g., “enterpreneur”, “award”, “recognize”, “leader”, “power”, “career”, “success”, “deserve”, “science”, “valid”);

2. learning and education (e.g., “teacher”, “inspire”, “learn”);

3. social welfare (e.g., “finance”, “advocate”).

All these aspects were dominated by positive, concrete concepts eliciting a sense of achievement and professional establishment. The resulting picture was an overall positive sentiment/perception of the online social discourse about the figures of women in science. However, within this positive landscape, a closer look at the local community of concepts clustering around “woman” identified also negative associations, further highlighted in Fig. 3C.

The analysed text associated women also to “indebted”, “kill”, “lone” and “cry”, highlighting a multifaceted perception of women in STEM and including negative traits that might originate from stereotype threat (Shapiro & Williams, 2012; Pennington et al., 2016). In cognitive psychology, stereotypes can advantageously form a belief about or characterise key traits of groups of people through little cognitive effort. Stereotypical perceptions come at the cost of being not grounded in empirical knowledge, providing inexpensive but potentially erroneous information that can: deeply affect perception and cognitive processing (Pennington et al., 2016), induce anxiety, and reduce performance under pressure, e.g., causing a sense of threatening. For instance, STEM stereotypes like “girls are not good at maths” have been reported to affect even the self-perception of female STEM students, causing unconscious biases which reduced girls’ performance and retaining of STEM subjects (Shapiro & Williams, 2012; Steinke, 2017; Chavatzia, 2017). The TFMN reported in Fig. 3A, and its inset (C), highlighted the stereotype of the “oppressive, lonely, big-shot woman in STEM”, relative to women in STEM achieving successful careers only at the cost of sacrificing empathy and other positive personality traits (Ely, Ibarra & Kolb, 2011; Madsen & Andrade, 2018). Recent studies have overwhelmingly exposed such stereotype, cf. Steinke (2017), and the permanence of these negative associations in online social discourse represents evidence of how such stereotype is still deep-rooted in daily communication. Notice that this might have also positive repercussions, as improving the awareness of the effects of stereotype threat by simply talking about it can boost self-perception and promote the implementation of effective countermeasures fighting stereotypes (Madsen & Andrade, 2018).

The reconstructed online perception of women was overwhelmingly positive and it indicated how women’s success and leadership in STEM deserve recognition (Madsen & Andrade, 2018). This perception has to be compared against the one of males. Figure 3B considers the forma mentis network around “man”, which included mostly positive concepts like “science”, “passion” and “smile”. However, syntactic links associated “man” also to negative concepts such as “rape”, “issue” and “racist”, indicating the presence in the online discourse of forms of accusations or condemnations of the role played by men in social issues like sexual harassment, rape and racism (Karami et al., 2020). In the forma mentis around “man”, special attention has to be devoted also to the negative particle “not”, which is syntactically linked to “god”, “think”, “make” and “consider”. The negation of all these concepts, as indicated by the syntactic links, provided a multi-faceted perception where men in STEM were related to a superiority complex, a stereotypical self-perception of superiority in science-related achievements, for instance in mathematics assessments (Leyva, 2017), promoted by preliminary and incomplete studies (cf. Leyva, 2017; Chavatzia, 2017). This quantitative result, embedded in language and highlighted by forma mentis networks, indicates that fighting the gender gap in STEM means also changing men’s stereotypical roles in science.

In terms of emotional profiles, both “woman” and “man” included associates evoking mainly trust, joy and anticipation, see Figs. 3E–3F. The TFMN around “woman” was slightly richer in anger-eliciting words, like “kill” or “shot”, than the TFMN around “man”, but both levels were compatible with random expectation.

In Fig. 3D, the TFMN around “person” was devoid of any negative concept or sign of stereotype threat. Word clusters related “person” to a multifaceted perception about career progression, mindfulness and education. The associates of “person” evoked mostly trust (emotional profile not shown for brevity), further confirming the positive perception of “person” itself. Given the gender-less dimension of “person” in English, it is interesting to underline how the above negative perceptions are more tightly connected to gender-rich words for human beings like “man” and “woman” and not to genderless words like “person”. Since words in language do not only describe reality but rather contribute to forging it (Malt, Wolff & Wolff, 2010), the above result suggests that reducing the incidence of the gender gap might be possible also by speaking more and more in terms of “persons in STEM” rather than in terms of contrasting “men versus women in STEM”, always while respecting individual differences.

The online social perception of gender gap and scientist stereotypes as reconstructed by TFMN

This subsection investigates “gender” and “gap” and their online perceptions. Figure 4 considers the TFMN around “gender” (A,B) and “gap” (C,D), which were both considered as neutral concepts in language.

The online discourse over the social platform associated these two concepts with each other, as expected, and related both of them mainly to positive words. Hence, the overall online emotional aura of “gender gap” was mostly positive but it also included some negative associates. The community structure for the TFMN of “gender” identified three perceptual dimensions in the way online users talked about gender:

• a positive dimension of respect and consideration (with associates like “balance”, “close”, “consider” and “respect”);

• a research dimension in understanding the science of gender (with associates like “research”, “support”, “future”, “highlight” and “explain”);

• a mostly negative dimension of gender imbalance and biases (with associates like “doubt”, “imbalance” and “attack”).

Associations of “gender” with concepts like “stereotype”, “unfounded”, “break” and “tackle” (see Fig. 4B) indicated an attitude of opposition, among online users, against gender stereotypes in the considered online communications.

Analogously to “gender”, also “gap” included negative associations to “bias” and “attack” and positive links to concepts like “overcome”, “close” and “consider” (see Fig. 4C). The conceptual cluster around “gap” highlighted in Fig. 4D contained associations mostly focusing on:

1. the expectation to overcome and reduce the gender gap in the future (e.g.,“reverse”, “break”, “crack”, “bridge” and “reduce”);

2. the economic impact of the gender gap (e.g., “pay”, “income”, “wage”).

These two dimensions indicate a positive intention for the online discourse to tackle and reduce gender gaps, while displaying awareness about the gender pay gap (Courey & Heywood, 2018) and also an emotion of anticipation or projection of such challenge in the future (see emotional profiling in Figs. 4E–4F). This provides additional quantitative indication for the importance of achieving equal wages in STEM in order to reduce the gender gap, in agreement with previous studies (Courey & Heywood, 2018). Figure 4 contains also negative associations between “gap”, “bias” and “unconscious”, indicating a remarkable awareness of online users about the gender gap being rooted in unconscious bias and gender preconceptions that are difficult to detect and act upon (Karami et al., 2020; Steinke, 2017; Moss-Racusin et al., 2012; Pietri et al., 2018).

As already mentioned above, a prominent mechanism of unconscious bias is stereotype threat, where unconscious perceptions cause anxiety and negative latent emotions that influence performance, e.g., girls aware of the preconception that “girls are not good in science” end up performing worse than males in STEM tasks (Moss-Racusin et al., 2012; Shapiro & Williams, 2012; Madsen & Andrade, 2018; Chavatzia, 2017). Given the awareness of online users about unconscious biases, it becomes relevant to explore the networked TFMN mindset in search of signs of stereotype threat.

Because of the STEM scope of the dataset, the focus here was devoted to detecting stereotypical perceptions in the figure of “scientist”, whose neighbourhood is reported in Fig. 5A. The TFMN quantified positive semantic associations surrounding “scientist”. Clusters of more tightly associations identified a strong perception of scientists in relation to innovation (e.g., “tech”, “society”, “entrepreneur”, “innovate”, “discover”, “leadership”). A closer look (cf. Figure 5B) revealed also enthusiasm-eliciting concepts (e.g., “great”, “notable”, “famous”, “celebrant”, “dream”, “congratulate”) and career-related jargon (e.g., “showcase”, “assess”, “work”, “peer”, “academia”). Hence, scientists were perceived as successful professionals by online users, analogously to what other social groups like high-school students did in other studies (cfr. Stella, 2020). Even within this success-centred perception, online users linked scientists to concepts like “suffer”, “pain” and “gain”, displaying awareness about the cost of success and the need for hard work.

Overall, the mental construct of “scientist” represented by the TFMN identified a well-rounded, balanced and positive online perception of scientists in terms of successful, hard-working professionals, devoid of any significant patterns of stereotype threat associating scientists only to a male-gender sphere, differently from stereotypical perceptions found in other datasets and groups via the Implicit Association Test (Lane, Goh & Driver-Linn, 2012; Steinke, 2017).

In line with the semantic analysis, according to the emotional profile in Fig. 5C, scientists elicited strong feelings of trust, joy and anticipation.