Using transformers and Bi-LSTM with sentence embeddings for prediction of openness human personality trait

Motivation and significance

The motivation to determine and study the characteristics of personality arises from the merging of theoretical and practical benefits that are used to understand the complications of human behavior and emotions as well as the study of their cognitive level based on variety of data nature (Song et al., 2023b). This curiosity helps in revealing the facts that help in shaping personality discoveries according to the individual traits and differences involving their personal thoughts and it focuses on the mechanism that controls human societal interaction. The application of natural language processing (NLP) technique personality of various traits of personality traits of individuals. This analysis of personality helps in finding proper individuals for marketing purposes, finding customers regarding products and services (Malik & Bilal, 2024), career counselling (Fatima et al., 2022) and other aspects of personalization (Salminen et al., 2020). The dramatic changes in personality research are supported with advancements in computational techniques and an increase in large data (Naz et al., 2024). The latest AI-based algorithms provide predictive analytics in the study of personality offering improved opportunities in revealing hidden patterns and getting deeper insights. In all this scenario exploring personality traits ensures that it will enhance our knowledge and understanding of real-world challenges in all fields (Christodoulou & Gregoriades, 2023).

This research also addresses the gap in understanding how linguistic features associated with abstract thinking, such as creativity and exploration, contribute to detecting the openness trait. Previous studies have not thoroughly examined how these features manifest in informal language, such as social media posts or personal narratives, which this study explores (William et al., 2023). This study advances the field by utilizing an advanced approach that combines traditional machine learning models, such as support vector machines (SVM) and random forest (RF), with cutting-edge deep learning techniques, including long short-term memory (LSTM) and Transformer-based models. By integrating both word and sentence embeddings, we capture deeper semantic features related to openness, which previous studies have largely overlooked. Sentence embeddings have been applied to capture the detailed semantic characteristics of textual data to predict openness personality traits. That is, the proposed method of advanced embeddings overcomes limitations of other methods based on word-level embeddings or statistical features for utilizing and understanding sentence meaning while preserving context and allowing for highly accurate prediction of outcome. Most importantly, this study examines the potential of sentence embeddings in relationship to MBTI personality prediction. The essence of embeddings is its ability to capture contextual relations within sentences, the features produced by which can improve both traditional and deep learning models (Alsini et al., 2024). Furthermore, our work provides new insights into the specific linguistic patterns that correlate with the openness trait, contributing to more accurate and interpretable personality trait predictions from textual data.

In this research study, our aim is to predict open personality traits from social media posts relying on MBTI dataset. For empirical analysis, the MBTI dataset utilized by applying feature extraction, we used standard features of text features with ML models like Term-Frequency Inverse Document Frequency (TF-IDF) and parts of speech (POS); and for deep learning, word embeddings and sentence embeddings have been used. Traditional ML models such as SVM, decision tree (DT), logistic regression (LR), naïve Bayes (NB), K-nearest neighbor (KNN), and ensemble models like RF, gradient boost (GB), adaptive boosting (AdaBoost), and extreme gradient boosting (XGB) used textual features. Whereas, deep DL including LSTM, and bidirectional long short-term memory (Bi-LSTM) use deep features. In addition, the state-of-the-art Transformers-based model bidirectional encoder representations from transformers (BERT) model is also applied. The results are examined using standard evaluation measures of accuracy, recall, precision, f-measure, area under the curve (AUC), and receiver operating characteristic (ROC). From these results, the proposed features to be effective in term of increasing classifier accuracy for the researchers to consider AI applications in personality traits is an active area of research.

Our main research contributions in this research study are as follows:

• Applying comprehensive feature engineering for personality trait detection including textual features of TF-IDF, POS tagging and word embeddings of word2vec, and GloVe.

• Proposing the advanced approach to derive the semantic features for MBTI openness trait using sentence embeddings as they are particularly useful in retaining the contextual relationships that is necessary for higher accuracy.

• Examination of diverse traditional ML models, and advanced DL algorithms like LSTM and Bi-LSTM and transformer-based model BERT.

• Conducting detailed comprehensive analysis acquiring highest results of 90.52 with accuracy measure with Bi-LSTM coupled with advanced word embedding technique sentence embedded on MBTI dataset as compared to existing literature.

• Achieving overall, the top results with state-of-the-art Transformer-based model Distill BERT, accuracy of 92% to predict the personality trait.

The rest of the article is structured as follows, shown in Fig. 2: “Related Work” presents the in-depth analysis of existing studies based on AI-based approaches, “Proposed Research Methodology” presents the proposed framework sharing steps of the research methodology, “Experimental Setup” highlights the experimental setup investigating datasets and evaluation measures, “Results and Discussion” presents visualization analysis with results of this study discussed in a comprehensive way, “Conclusion and Future Directions” provides the conclusion of the research along with future directions.

Personality trait prediction using machine learning

The research area still captures change patterns of demographic on human behaviour features and the openness or openness as one of sensibility characteristics. Table 2 defines the summary of existing studies based on ML models. The machine learning model, specifically an SVM, achieved classification accuracy for the traits ranging from 60% to 75%, which is considered optimal results using feature extraction bag of words and TF-IDF. For the openness trait in the hold-out dataset, the accuracy reached 63% (Zhi et al., 2023). NLP-based approach to data-centric prediction of MBTI personality types enhances depersonalized text representation with concerning features generated using sentimental, grammatical, and aspect analyses were applied to each MBTI classifier. The dataset’s representativeness varies, with a significant imbalance in the Intuition vs. Sensing trait representation applied with TF-IDF approach following traditional ML models (Basto, 2021). A web platform was developed to collect data for personality assessment using adjectives selection, which was then input into Machine Learning architectures based on GB. Two models were developed: one uses regressors to provide a precise score for the Big Five personality traits, while the other GB employs classifiers achieved F1-score of 83% to provide a categorical output (Fernandes et al., 2020). Another primitive approach developed with the help of binary transformer with feature extraction Term Frequency & Inverse Gravity Moment is suggested to be employed with three dataset—Facebook, Twitter and Instagram. Results of maximum entropy classifier to predict personality trait with up to 83% accuracy (Kamalesh & Bharathi, 2022). One exception approach using clustering, SVM and LR is utilized to predict human behavior using personality trait with the help of my personality dataset to categorize high dimensional feature space (Shah & Modi, 2021). Furthermore, supervised learning (Song et al., 2023a) and unsupervised learning techniques (Ghassemi et al., 2024) based on facial detection also utilized in literature for the prediction of human natures exploring emotions, expressions and signals of individuals, that indicate to personality traits, with the integration of big data methods (Alexander, Mulfinger & Oswald, 2020) can predict a learner’s ability to effectively use self-regulated learning strategies to achieve academic goals with the influence of personality trait levels. The potential for using personality profiles in design of educational experiences and highlights the importance of effective time management and motivation in the self-regulation process (Bruso, Stefaniak & Bol, 2020). The efficiency of various features and regression models for prediction through language centered on psycholinguistic traits that take advantage of the relationships between different personality models (Radisavljević, Rzepka & Araki, 2023). Ensemble learning and SVM are trained with TF-IDF, BoW demonstrates the new classifier for traits prediction utilizing the indicator for personality classification to develop text-based classifiers to categorize dialogues (Ramezani et al., 2022). Twitter hate content is analyzed to identify the level of identification score with the help of users’ profile to classify personality trait by applying ensemble approach RF and XGB comparison with traditional ML models such as SVM, and LR achieving an accuracy of 80% (Perera et al., 2023).

Personality trait prediction using deep learning

Table 3 shows the analysis of existing studies based on DL models. Psychological factors prediction with deep learning approach has vast application using social media to employ word embeddings on textual data require computational resources and large dataset for training emphasizes the successful use in DL models such as CNN, LSTM and RNN (Santos et al., 2023). The recommendation system of using Neural network model also uses the Word2Vec and LSTM layers to analyze textual data to predict personality features (Silpa et al., 2023). Text oriented approach was introduced knowledge graph content to categorize traits separately integration of BERT+NLP using personality and essays dataset achieving an accuracy of 81% (Johnson & Murty, 2023). The works, LSTM for selecting career based on traits prediction (Naik et al., 2022), LSTM CNN (Sujatha et al., 2023), and Bi-LSTM for personality classification using textual content improving with word embedding for text representation achieves 61% accuracy using MBTI dataset (Khattak et al., 2024). To improve accuracy and reliability of the prediction models have XGBoost classifier is used with the BERT transformer model. particularly analyzing user behavior and traits based on social media activities that combine sentiment, grammatical, augmentation, and imbalanced data for better performance for richer text interpretation (Lucky, Roslynlia & Suhartono, 2021). The proposed model effectively combines emotion and personality trait detection tasks, utilizing a model-independent meta-learning framework for efficiency. The learning process significantly improves the approach, leading to root performance in both detection tasks. Leveraging ML and NLP classifications, improving text representation, and feature generation using bidirectional encoder BERT to predict sixteen personality axes by using online social media collected dataset based on MBTI (Rauf, Johan & Aziz, 2023). To optimize DL with limited data by using data fusion techniques and source mapping the essays data was mapped into the big five traits utilizing the CNN-AdaBoost with 59% approach demonstrating its efficacy in automated personality detection by using social media texts (Deilami, Sadr & Tarkhan, 2022). Based on various emotions, to indicate user personality utilizing various type of data (Leekha et al., 2024) enhance the significance of other filed such as for mental health with the help of personality trait predictions the analysis of logic-based approach was conducted using the type indicators that enhance the MBTI by incorporating reasoning factor, allowing for analysis of personality types that can effectively capture the subtleties and complexities of human behavior (Zim, Hanif & Kaur, 2024).

Limitation of existing studies

In this literature, numerous studies have explored various ML and DL to predict traits from textual data. The majority have primarily relied on traditional features methods such as bag of words or TF-IDF, and with deep features like word2vec and GloVe to present text data. One significant limitation observed in the existing literature on personality traits prediction is the dearth of study leveraging openness personality traits. Therefore, the absence of research by presents a notable gap in current literature. This research focuses on the existing knowledge of how features linked to abstract processing, including creativity and exploration, helping to determine the openness trait. Earlier research has not focused effectively on how such aspects are realized in colloquial text, including Twitter posts and blog entries, which are investigated in this study. Therefore, this study innovates the previous literature by incorporating both the classical ML algorithms together with the new generation’s deep learning algorithms. By incorporating word embeddings with those sentences, we can obtain even more semantically nuanced features concerning openness which have remained largely unexplored in prior work. Moreover, our study offers novel information on the grammatical and lexical features that are connected to the open personality trait, which is instrumental in better and more explainable personality trait estimation from textual data.

Data preprocessing

MBTI dataset, which requires meticulous data cleaning and preprocessing to ensure optimal performance of predictive model. Data on MBTI must be preprocessed to make it ready for analysis and ML techniques. The data cleaning process involves several steps:

• Converting all letters to lowercase to maintain consistency and avoid case sensitivity issues.

• Removing links to eliminate irrelevant and unwanted information.

• Elimination of punctuation to simplify the text and focus on the core content.

• Removal of stop-words, such as “and” “the,” and “in,” which do not contribute meaningful information for personality prediction.

• Lemmatization to reduce words to their base or root forms, ensuring the different forms of the same words are treated as single entity.

• Character normalization was applied to standardize characters, handling issues like accented letter ( from “è” to “e”) to ensure uniformity.

Following the data cleaning process as shown in Fig. 4, data pre-processing is proceeded with tokenization step. Tokenization is the process of splitting the cleaned text into individual words or tokens. Once the text is tokenized, the data is preprocessed into a suitable format for ML models.

Feature engineering

Feature Engineering for text data involves transforming raw text into meaningful features that models can utilize. In this study, textual features such as TF-IDF which quantifies the importance of words in documents, and POS tagging, which identifies syntactic content are conducted for th analysis. Another approach such as, word embeddings to capture semantic relationships by representing words in continuous vector spaces, while sentence embeddings extend this by encoding entire sentences, capturing their broader meaning and context. These methods enhance the model’s ability to understand and predict on textual data (Saeed et al., 2023).

TF-IDF used to calculate the significance of a word in a document comparative to a collection of documents by merging two parameters: One is term frequency (TF) and second is Inverse Document Frequency (IDF). By combining these two metrics, through Eq. (1), to ensure that each term’s weight reflects both its frequency within a document providing full normalized form and its significance across the entire corpus (Ahmed et al., 2022), providing a balanced measure of term importance. All parameters are defined in Table 4 for understanding.

(1) $$TF-ID{F}_{norm}\left(t,d\right)={\displaystyle \frac{\begin{array}{c}\left({\displaystyle \frac{log\left(1+f\left(t,d\right)\right)}{{\sum }_{k}log(1+f\left(k,d\right))}\times log\left(1+{\displaystyle \frac{N}{1+\left|\{d\in D:t\in d\right|}}\right)}\right)\\ \end{array}}{\sqrt{{\sum }_{t^{\prime }}(\left({\displaystyle \frac{log\left(1+f\left(t^{\prime },d\right)\right)}{{\sum }_{k}log(1+f\left(k,d\right))}\times log\left(1+{\displaystyle \frac{N}{1+\left|\{d\in D:t^{\prime }\in d\right|}}\right)}\right){)}^2}}}$$

POS tagging identifies the sequence of POS tags with the highest chance of yielding the observed words which tagging entails identifying to which grammatical type a particular word belongs (for example noun, verb, adjective etc.) Khan et al. (2019) represented as $w_1,w_2,w_3,\dots \dots .,w_n$ in a sentence, corresponding tags $t_1,t_2,t_3,\dots \dots .,t_n$, the combine words and tags probability $P(w_1,w_2,w_3,\dots \dots .,w_n$, $t_1,t_2,t_3,\dots \dots .,t_n)$ is given by using chain rule computed as in Eq. (2).

(2) $$P(w_i)=P(t_1)\times \prod _{i=2}^{n}P\left(t_i|t_{i-1}\right)\times \prod _{i=1}^{n}P\left(w_i|t_i\right).$$

Word2Vec is a word embedding technique used to convert words into continuous vector representations, capturing semantic relationships between them. Word2Vec chose word embedding strategies to change the text data map into dense vector representations, capturing semantic relationship among words and sentences based on their co-occurrences in massive textual content corpora. Equation (3) representing the embedding of a word w in the vocabulary is:

(3) $$v_w={\displaystyle \frac{1}{\left|C\left(w\right)\right|}\sum _{cϵC\left(w\right)}{v}_c.}$$

GloVe is a method for generating word embeddings by leveraging the co-occurring of words in a data. International phrase-phrase co-incidence records generate embeddings that emphasize each local and global context facts by leveraging the global word-word co-occurrence statistics from a corpus is defined as in Eq. (4).

(4) $$J=\sum _{i,j=1}^{v}f\left(X_{ij}\right)(w_i^T{\varpi }_j+b_i+{\overline{b}}_j-\log X_{ij}{)}^2.$$

Sentence embeddings representations in the form of vector notation of whole sentences an effort to capture both the meaning of the sentence and its grammatical phrases. Figure 5 presents the working of embedding, allowing for sentence similarity and comparison by measuring the distance of vectors, as defined in Eq. (5).

(5) $$v_{sentence}={\displaystyle \frac{1}{p}\sum _{i=1}^p{p}_{wor{d}_i}.}$$

Applied algorithms

We therefore employ a detailed strategy of using computational models for the prediction of openness trait. Prediction using supervised and unsupervised ML and DL exposes how methods of text classification can be grouped according to focus on language. Measured from relatively basic models such as probability models LR, and DT to sophisticated models SVM, and XGB. Likewise in DL, language models are central to NLP tasks, a gets sample of complex framework for developing and interpreting human language. The language has been categorized using varieties such as LSTM, Bi-LSTM and self-attend and Agg transformers, BERT to enable applications such as text summarization, chatbots and other dialogue systems (Ahmed, Khan & Munir, 2024). We applied traditional ML models that provide a baseline for comparison and are typically faster to train and interpret. ML models are effective for initial analysis, helping to identify patterns and relationships in the data. Ensemble models were carried out due to improve prediction accuracy by combining the strengths of multiple models, as ensemble models reduce the variability and lead to better generalization. Advanced deep algorithms are used for their ability to capture long-term dependencies and sequence information in text data, which is crucial for understanding complex personality trait. Transformer-based model is justified for its state-of-the-art performance in NLP tasks, offering deep contextual understanding and higher performance. It’s particularly valuable for capturing subtle patterns in language that are indicative of personality traits. Here is the brief methodology of algorithms conducted in this research.

Traditional machine learning

Traditional machine learning refers to a class of algorithms and techniques used to perform various tasks such as clustering, regression, and classification (Daud et al., 2023). In the realm of personality trait prediction, our methodology offers a pragmatic approach including SVM, DT, NB, LR, and KNN model, to analyze features extracted from various sources, such as social media post, surveys, and blogs to predict individual personality trait openness with the help of N/S labels from MBTI data presents an insightful exploration of conventional ML approach in personality prediction, models including:

• Support vector machine offers a robust framework for predicting personality trait openness by effectively defining boundaries between personality types. SVM high potential in predicting complex relationships within data, that labels exhibit intricate interdependencies with other factors. By leveraging with labels N/S, SVM can discern patterns allowing for accurate classification of openness level. SVM’s ability to identify the optimal hyperplane for separating data points facilitates the extraction of meaningful insights from feature space.

• Decision tree utilized for recursive partitioning of features space into the subsets relying on the target feature, aiming to create decision rules that best separate individuals with different openness levels. Each internal node shows a feature, and each leaf node corresponds to a predicted class (e.g., high, or low openness).

• K-nearest neighbor is the simplest yet powerful algorithm. The predicted class of a data point is categorized by the class label of its closest neighbors in the feature space. KNN is utilized to capture complex correlation in data and control the non-linear decision boundaries.

• Naïve Bayes is a probabilistic classifier that features are conditionally independent given the class label. It is computationally efficient and requires minimal data for training to compute the probability of each class features and select the class with the highest possibility as the prediction.

• Logistic regression is a statistical model, the possibility of a given input relates to a particular class by using the logistic function to map any real-valued number into the [0,1] range. Classifying N and S labels, LR can effectively model the probability of a given input belonging to one of these classes.

Ensemble learning models

Ensemble learning models combine multiple learners to improve the prediction accuracy for personality trait predictions models like GB, RF, XGB and AdaBoost to enhance the overall working and predictive accuracy of the system. The essential idea at the back of the ensemble studying is to leverage the collective intelligence of more than one model instead of relying on single model. These techniques aggregate the predictions from different individual models such as decision tree to mitigate overfitting and enhanced generalization (Ishfaq et al., 2022). For instance, random forest creates a multitude of decision trees and average their predictions, reducing variance and improving stability. Gradient boosting, sequentially builds trees to correct errors from previous ones, enhancing model performance. Adaptive boosting adjusts the waves of misclassified instances, focusing on difficult cases. By leveraging ensemble learning, prediction of openness based on NS labels becomes more accurate and dependable, as the combined strength of multiple models can capture complex patterns and interactions better than single models.

Concurrent deep learning

Deep learning is a subpart of machine learning that works on concepts of neural networks with multiple-layer model complex patterns and representations in data. These models are particularly powerful for task involving large model data and intricate feature interaction. The training of DL models using parallel processing techniques to accelerate training time and improve accuracy refers to the concurrent deep learning models (Khan et al., 2023). Applied algorithm; LSTM networks are type of recurrent neural network used to capture temporal dependencies and long-range correlation in sequential data. In the realm of predicting personality trait openness and the LSTM can effectively model patterns in time series data or sequence of texts such as social media posts or questionnaires that reflect personality traits. LSTM utilized memory cells and gating mechanism to maintain and update information against N/S labels to retain relevant information from earlier in the sequence and use it for current predictions, which is crucial to understand how past behaviors or responses correlate with personality trait. Other model Bi-LSTM network extend LTMS by processing sequences in both forward and backward directions allowing the model to capture content from past and future states (Kazi, Khoja & Daud, 2023). This bidirectional approach increases the ability to understand and understand the patterns in data sets for personality prediction. Bi-LSTM can leverage the full context of data, providing a more comprehensive analysis of start traits like openness manifests through patterns associated with N/S labels.

Integrating these capabilities, LSTM and Bi-LSTM network can effectively model complex patterns and dependencies in personality trait data, leading to more accurate prediction of openness based on MBTI NS label.

Transformer-based model

Transformer based models like BERT predict personality trait by leveraging their powerful ability to understand context and semantics in the text data. BERT utilizes transformer architecture, which applies the self-attention mechanism to derive the connection of word from the left and right context (Hayat et al., 2019). This bidirectional approach allows the word to develop a deep understanding of language for predicting personality traits such as openness using MBTI in Intuitive (N) and Sensing (S) labels, BERT fine-tuned on text data where the labels are known. During this fine-tuning process, BERT learns to associate certain linguistic patterns, phrases, and contextual cues with specific personality traits. By processing text data from sources like social media post to identify subtle indicators of traits like openness. Another model, DistilBERT is smaller and faster than pointing BERT with pretraining distilled for a better lightweight version. It does so use a mechanism called “knowledge distillation,” in which a “student” model that’s significantly smaller is exposed to the behavior of a larger “teacher” model—here, BERT. In this way, DistilBERT achieves about 97% of BERT’s language comprehension ability, consuming only 40% of its space and providing 60% faster inference time. Figure 6 displays the working of DistillBERT model.

Thus, DistilBERT is designed to be a lighter version of BERT with only six layers instead of 12 layers in BERT base to retain attention mechanics such as multi-head self-attention and feed forward neural networking. During training phase, not only the student model (DistilBERT) gets trained on the labeled data but also from the softer probabilities generated by the teacher model, as defined in Eq. (6).

(6) $$\mathcal{L}=\alpha \cdot {\tau }^2\cdot KL(P_{teacher}^{\tau }||P_{student}^{\tau })+\left(1-\alpha \right)\cdot {\mathcal{L}}_{CE}$$

Such softer probabilities tend to offer deeper insight into the discrimination between classes than just the probability values so aiding the student model to generalize better due to its size. DistilBERT offers a stronger and more efficient option to the BERT model’s true adaptation for the real world’s complicated scenarios and requirements. This makes it particularly effective for complex personality prediction tasks as it can discern the ways in which personality trait manifest in language.

Dataset

In this research study, we selected MBTI (https://www.kaggle.com/datasets/datasnaek/mbti-type) dataset from Kaggle repository. The dataset consists of text data linked to MBTI personality types. Providing a rich source for analyzing the relationship between language use and Personality. The dataset has 8,675 rows and includes a variety of text inputs, such as social media posts, blogs, forum comments, tweets, YouTube links along with corresponding MBTI labels. In research, targeted dataset attends as useful material for studying personality traits, enabling the development, and testing of models that predict MBTI types from distinct personality data, especially in studies we target Intuitive (N) and Sensing (S) labels that is mapped on openness where text analysis and NLP approaches are used to infer personality types from written content.

Assessment metrics

For the analysis of classification models to predict personality trait, we often employ several performance criteria which are presented in Table 6.

Main performance assessment indicators include accuracy, precision, recall, F1-score and ROC-AUC. are carried out to ensure a comprehensive and fair assessment of the models. These metrics provide a balanced view of how well the models perform across various standard evaluation dimensions, such as overall correctness (accuracy), handling of false positives (precision), capturing true positives (recall), and achieving a balance between precision and recall (F1-score). Moreover, ROC-AUC evaluates the model’s ability to distinguish between classes at different thresholds. Using these standard measures ensures the robustness, comparability, and generalizability of the models across different datasets and applications.

Traditional machine learning models with textual features

The results obtained from applying various ML models to predict openness trait with label N/S using different feature representations offer valuable insights into effectiveness of each approach. Across both TF-IDF and POS tagging representation, the SVM consistently emerges as the top-performing model, showing robustness in handling high-dimensional and noisy textual data. With TF-IDF the highest prediction rate is 89% with SVM models to classify personality openness trait. While other models, such as NB, LR and KNN, demonstrate competitive performance across both features. Similarly, SVM with POS tagging with 86% accuracy highlighting its adaptability to different types of features. Results with machine learning are shown in Table 7.

The findings to quantify openness personality traits based on MBTI N/S labels with TF-IDF and POS tagging features explain aspects of model accuracy and feature utility. Such models were able to capture the critical semantic and contextual features because of TF-IDF formula that reduces term frequency in documents by the number of documents. SVM had the highest accuracy of 89% proved high dimensional data handling capability and subtle pattern detection on textual features in addition to high precision, recall and F1 score. LR also showed good results by yielding 87% accuracy with balanced metrics as it works best with TF-IDF, which creates a linear model. The results showed KNN with a good accuracy of 86% verified the usefulness of the system even though it weighed heavily upon the feature scaling factor NB and DT, while receiving a comparatively lower accuracy of 75%, pointed out the inability of the system to handle the high degree of complexity, TF-IDF could not capture.

Meanwhile, the features that based on POS tagging demonstrate moderate performance since here, there is more emphasis on grammatical structures and syntactic roles. SVM was once more identified as the best performer with a mean accuracy of 86%, implying that the method is less sensitive to the types of features used. LR and KNN gave very similar results, suggesting their capability compatible with basic variables. Nonetheless, NB and DT attained 70% and 75% accuracy respectively somewhat lower than the results found by SVM due to the diminished semantic value of POS tagging.

Summing up, the results obtained emphasize the critical role of feature representation in increasing model effectiveness. Hence, TF-IDF turned out to be a better feature engineering technique for personality trait prediction, which contains semantic values and enabled enhanced models such as SVM and LR. The results corroborate the specified methods and stress the significance of integrating optimal feature extraction with highly stable personality prediction algorithms.

Ensemble learning models with textual features

The provided results demonstrate the performance of ensemble learning algorithms for prediction of trait using TF-IDF and POS features. Ensemble learning techniques such as GB, RF, XGB and AdaBoost, are known for their ability to combine multiple models to improve predictive accuracy. Across both features, GB and XGB achieve the highest accuracy with TF-IDF is 90% for both and with POS tagging 86% and 85% respectively as shown in Table 8. When predicting openness personality traits from the MBTI N/S labels based on advanced feature engineering of ensemble models in combination with TF-IDF, POS tagging, the conclusions are boosted by rigorous and modern machine learning architectures. Specifically, the chosen features based on TF-IDF values yielded high accuracy, reaching 90% on average for both GB and XGB and complex patterns of individual word interactions, while providing similar levels of precision, recall, and F1 scores 89%. These models stand out for their ability to adjust the tree boosting process which improves learning through feature extraction. AdaBoost also emerged well, with accuracy of 89% and metrics very close to each other, this underlines its ability in enhancing poor performers through adaptive boosting. RF seems quite stable yielding an accuracy of 86%, but the F1-Score of 79% shows that there is room for better accuracy in a way that can optimally balance Precision and Recall.

The performance of the models with POS tagging features which draw attention to syntactic structures was moderate for the ensemble models. XGBoost outperformed all other algorithms with 87% accuracy while the model remained stable with small contract features that are not semantically richer. Gradient Boosting was closely behind with 85% accuracy proving it can handle a wide variety of features. The accuracy in RF was at 86%, hence it recognized the parameters within the grammatical structure. Nonetheless, AdaBoost turned out to be less accurate with only 78%, along with lower precision, recall, and F1-measure values because of restricted positivity of the POS tagging features.

Such results indicate that the ensemble models have returned best accuracy, especially by GB and XGB which have incorporated richness of semantic TF-IDF for computation. They emphasize the feature representation for boosting model ability and illustrate that TF-IDF offers a better understanding of textual data rather than POS tagging. The results prove the applicability of ensemble learning methods to enhance the accuracy of the algorithm for personality trait prediction. These results highlight the efficacy of ensemble learning techniques such as GB and XGB in capturing intricate patterns within textual data, thereby enabling more accurate predictions of personality trait. The comparison between TF-IDF and POS features for prediction N/S patterns, with TF-IDF features, all models achieve high accuracy scores ranging from 84% to 90%. Particularly the consistently high performance of SVM, GB, XGB and Adaboost models indicating their ability to effectively distinguish between intuitive and sensitive individuals. In contrast, models trained on POS tagging features exhibit lower accuracy and performance metrics compared to TF-IDF based models. While some models, such as SVM, LR and KNN, achieve high accuracy and precision with POS features, others like NB and AdaBoost show lower performance. The next Table 9 presents the hyperparameters of applied models.

Based on overall ROC-AUC score produced from feature selection through TF-IDF and POS tagging, it was observed that sentiment analysis model yielded significant performance as depicted in the Figs. 11 and 12. When using TF-IDF with SVM, both GB and XGB yielded high accuracy and f-score as well. This suggests that TF IDF works well in capturing information for discriminating against innate and perceptive characteristics. POS tagged exhibit lower accuracies, with SVM, RF and GB achieving the highest accuracy of 86%, but overall low as compared to TF-IDF. The overall performance of these proposed classifiers with textual TF-IDF and POS feature for personality trait detection are helpful for classifiers to detect traits more accurately. Combined ROC is computed of all models evaluated on dataset for comparative analysis of model performance to identify the superior model results with textual features TF-IDF and POS tagging.

The findings with cross validations show that personality traits based on specific features of ML and ensemble learning models proved as determinative, for example, determining the “openness” personality trait with N and S labels. Different textual features that belong to TF-IDF and POS tagging were applied and tested by cross-validation on various models, as shown in Table 10. Consequently, it was found that GB model provided excellent results for TF-IDF features in terms of accuracy, precision, recall, and F1 score of 90%, 89% and 89% respectively than XGBoost and RF. Most of the models, including the SVM model also gave satisfactory results with similar performance measures. In these models, Naive NB was the worst performing as it struggles when dealing with sparse data features such as TF-IDF.

Same trends were noted for POS tagging but the performance rates were slightly lower than the TF IDF feature. GB once more proved to be the best performer yielding an accuracy of 89%; it can be concluded that the package has a good generalization capacity for other complicated features such as POS tags. Another impressive performance was noted with RF and XGBoost models, as ensemble methods are efficient at capturing as many feature relations as possible. For POS features, the results of NB and LR are relatively lower than other models which may indicate that these models do not make proper use of the structural information which is present in POS tagging.

These results provide overwhelming evidence for the efficiency of the choice of ensemble learning models such as GB and XGBoost for predicting the open personality trait based on textual characteristics. Hypothetically, TF-IDF seems to pay better attention to more closely related semantic distinctions than POS tagging does according to all the models, as in term of accuracy analysis shown in Fig. 13. Applying cross-validation made the evaluation reliable and it was clear that these methods promised possibilities to predict openness with N and S labels. These results open the potential for adding new sophisticated functions and applying deep learning methodologies aimed at improving the predictions of personality traits more accurately.

Deep learning algorithms with word embeddings

When predicting the openness personality traits depending on MBTI N/S labels, deep models and word embedding features demonstrate the advantages of the identified approaches in extracting semantic and contextual data. LSTM and Bi-LSTM computed on the selected features on the test dataset for training to discover the differences between sensing and intuition mapped on the openness trait. The models learn to map the textual features of the corresponding corpus in terms of word2vec, glove and sentence transformer. The next Table 11 presents the parameters indicating the architecture of a model: These embeddings can be utilized by the models to arrive an improved understanding of the traits from text, due to the richness of the linguistic information encoded in them. LSTM and Bi-LSTM Model with embedding performance metrics are represented in Table 12 showing with the intention to predict traits on the base of input dataset.

In the given experiment of personality classification of intuition and sensing of a person, Word2vec followed with LSTM has recorded an accuracy of 86%. The experiments using GloVe with LSTM have relatively high accuracy in predicting the personality as sensing and intuitional of 84%. The experimental result shows that LSTM when used in Sentence Transformer has an 88% accuracy in classifying personality traits as sensing and intuitional. This proves the effectiveness of Sentence Embeddings for defining deep semantic similarity and contextual correlation in textual information best suited for higher order interpretation of textual data. Bi-LSTM based on Word2vec demonstrates high efficiency in signal and intuitive personality classification with an accuracy of 85%. Likewise, the proposed Bi-LSTM model brought improvements in each case for all the types of embeddings and was the best with the Sentence Embeddings as it offered an accuracy of 90.52%, precision of 89.23%, and F1 score of 89.21%. While Bi-LSTM is handling texts bidirectional, creating sequential structure which can provide both past and future context to any given text making it better in handling of complex languages. This advantage is further enhanced when used together with Sentence Embeddings which has a much higher semantics content representation.

However, Word2Vec and GloVe embedding browsers had high results, where Word2Vec got the maximum accuracy of 86.41% at LSTM and 85.50% at Bi-LSTM while GloVe got a maximum 86.27% at Bi-LSTM. These results suggest using them in representing word-level semantics, but they are somewhat less informative than sentence embeddings, which provide deeper context. Hence, it is also clear that the enhancement of architectures of Bi-LSTM to slightly better performance of GloVe indicates that incorporating advanced architectures in combination with improved feature representations are paramount.

Sentence Transformer with Bi-LSTM exhibit promising performance in identifying personality with the accuracy rate of 90.52% as combined outcomes are shown in Figs. 14 and 15. On balance, it can be affirmatively claimed that the Bi-LSTM with sentence embeddings reveals the greatest performance metrics, alluding to its appreciation of relevant and contextually significant features to drive the accurate forecast of openness traits effectively. These results prove the notion of using deep models with complex embeddings as highly efficient for personality prediction problems.

Prediction with transformer technique

Transformer based model, use of BERT and DistilBERT in our case, which is an attention mechanism that provides high quality representation of text, as working of BERT model shown in Fig. 16. Basically, it has an encoder stack of RNN that encodes the text bidirectional, implying it reads as well as behind to recall context from the words trained on masked language modeling; this predicts tokens that are hidden during training, and sentence prediction determines the relationship between given sentences. This mechanism trained every word in the input in order incorporate improved contextual awareness. In predicting the openness personality traits based on the MBTI N/S labels using the transformer-based models, the machine learning prospects of powerful ML have been clearly shown. The best results were obtained by DistilBERT, that has accuracy of 92.51%, precision of 88.32%, recall of 92.50%, F1-score of 89.21%. This proves how it can work as a good memory-storage device, as well as fast-computing device because it lacks unnecessary hardware components. In large-scale learning personalities’ prediction, DistilBERT has high capability in performance and computational assets, then suitable for the large-scale tasks. Compared to this, BERT gave out an accuracy of 88.24%, the precision of 74.92%, while the recall was 70.55% and F1-score was 71.52%. As can be seen, semantic understanding and contextual representation of the BERT model are a bit lower than those of DistilBERT, but still acceptable. The lower performance in comparison to DistilBERT could be attributed to combinatorial fine-tuning or optimization steps that were employed in training the two because both are transformer-based. The achieved results using proposed model based on variants for prediction with 88.24% accuracy to anticipate trait based on encoded text representation. In comparison to other deep models used for predicting personality traits, the BERT encoder highlighted competitive performance, to classify trait.

These findings provide further evidence of transformer-based models and based especially the DistilBERT, when it comes to identifying and interpreting n-gram patterns and contextual semantics of the textual data. These results further demonstrate that DistilBERT not only performs better but also has sufficient resilience and applicability for tasks that are both precise and fast, reestablishing its position as one of the most effective models in natural language processing for personality prediction.

To compare the effectiveness of the DistilBERT model with BERT model in estimating the openness personality trait. In both cases, SBERT (Sentence-BERT) was used as the encoder because it has been demonstrated to generate accurate sentence embeddings for sentence-level similarity tasks when fine-tuned version of BERT. Table 13 shows the results achieved on both models. Although both the extraction was done in a similar way of feature extraction, DistilBERT outperformed BERT having accuracy of 92% as compared to the accuracy of 88% of BERT. This improvement simply proves that even though DistilBERT is a lighter and computationally efficient model, it can capture complex semantic patterns within the text in a much better way. The knowledge distillation in DistilBERT due to the precise training technique, which guides the model toward learning from the teacher BERT without introducing redundant layers to the network. DistilBERT achieves a perfect balance between performance and efficiency by removing unreactive layers within the transformer architecture and fine-tuning self-attention mechanisms. This means that while training the distilled model the contextual factors were included and when the prediction was made it was much more accurate with other data sets.

These outcomes are especially valuable for the openness trait, which is usually identified by fine differences in the language reflecting such aspects as imagination, creativity or curiosity. The authors credit the higher accuracy of DistilBERT to its better model of these complexities in comparison to the standard BERT model. In conclusion, results approve the selection of DistilBERT model as more accurate and computationally efficient solution for personality trait prediction tasks.

Overall results performance

Comparing the performance findings across ML, Ensemble, DL, and Transformer-based methods, the research underscores the benefit of sophisticated procedures for determining the openness personality dimension using MBTI N/S labels demonstrate a remarkable performance. The utilization of comprehensive set of models like SVM, DT, RF, LR, KNN, NB, GB, XGB and Adaboost as well as DL models like LSTM and Bi-LSTM and transformer-based model DistilBERT, presenting a novel study apart from no existing work is available to predict trait, from conventional to state-of-the-art approaches are followed in chronological order, overall highest results shown in Table 14. It can be seen from Tables 7 to 8 that among the ML models, SVM achieved an accuracy of about 89% with TF-IDF features which indicate the ability of this model to handle textual data. GB and XGB achieved 90% accuracy with TF-IDF; the result underscores the effectiveness of stacking multiple learners in the process. The additional models, Bi-LSTM with Sentence Embeddings, gave an additional accuracy of 90.52%, thus confirming the deep learning model’s ability to capture semantic and contextual parameters within text. However, the Transformer-based Distil- BERT developed in this study yielded the highest accuracy with a value of 92.51% coupled with the best precision, recall and F1-measures across the approaches used. These results pinpoint the strength of Transformer models in identifying various language patterns and therefore chooses DistilBERT as the proposed model in this study given its high accuracy and efficiency in predicting personality traits.

The integration of diverse textual and word embeddings features further enhances the predictive power of the models. With traditional ML models with SVM + TFIDF, subsequently the highest accuracy from all models achieved by GB + TFIDF which shows the remarkable performance of boosting models. Moreover, sentence embedding outperforms the highest accuracy rate of 90.52% with the Bi-LSTM model underscores the effectiveness of advanced deep learning architectures for personality trait prediction. In addition, Transformer-based model, which has gained significant attention in NLP task due to its contextual understanding of language, DistilBERT is employed to capture the linguistic patterns from social media for personality traits prediction with the highest accuracy rate of 92% as compared to existing studies. Overall, these results in accuracy rates highlight the outstanding results of proposed methods with model potential for exploring more domain for research purposes in different ways for the prediction of personality trait from textual content.

The discussion of results focuses on the possibility of applied methodologies to predict the open personality trait: Intuitive (N) and Sensing (S). The results substantiate the hypothesis by revealing that advanced text analysis can accurately analyze a multifaceted personality trait “openness.” Only those models that incorporated both textual features and contextual representations proved to be robust at distinguishing the trait related linguistic features. These patterns suggest that the Intuitive individuals use conceptually associative thought processes when it comes to language, while the Sensing, individuals would be inclined toward sensorimotor language use.

These subtleties were especially important in restructuring by including new advanced word embeddings and transformer techniques for increased prediction performance. Primordial analysis of text data was given by TF-IDF for word count and word importance; on the other hand, contextual embeddings inclusion of sentence embeddings made it more possible to gain further in depth understanding of words’ semantics. Such findings confirm that personality characteristics can indeed captured from the patterns of language, and these findings showed huge informative value for psychological diagnostics, communications with specific goals, and subsequent individualized interventions. The predictive accuracy in identifying openness traits shows the possibility of natural language processing in interpreting and characterizing human actions.

One limitation of this study is that it focuses exclusively on the prediction of Openness personality trait. While Openness is a critical dimension of personality trait, there are other equally important traits that also significantly contribute to the overall understanding of individual differences in behavior. Moving forward, we will explore this research by adding more traits using LLM techniques, thereby enhancing the applicability of and robustness of our findings across the full spectrum of personality dimensions.

Comparison of proposed results with existing studies

For further enhancement of the predictive power of the models, diverse textual features such as TF-IDF, POS tagging, and word embedding including Word2Vec, GloVe, and advanced technique sentence embeddings are utilized in this research. Moreover, TFIDF+GB, XGB achieves the highest accuracy of 90% to predict the trait. However, we implement the advanced feature sentence embedding with approximately achieving the high accuracy rate of 90.52% with the Bi-LSTM model underscores the effectiveness of advanced deep learning architectures to capture the linguistic patterns from social media for personality traits prediction with the highest accuracy rate as compared to existing studies. Overall, this considerable enhancement of the results proves the efficiency of the proposed approaches over existing methodologies, as evidenced by Table 15, indicating the prospects of the work for further development of the research of personality trait prediction from social media content.