All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
I confirm that the authors have also addressed the last comments and therefore the manuscript is ready for publication.
[# PeerJ Staff Note - this decision was reviewed and approved by Mehmet Cunkas, a PeerJ Section Editor covering this Section #]
This must be edited by a proficient English speaker
It appears that the introduction of the paper is missing some critical references, as it seems to contain a series of unsupported assertions (including almost the entirety of a long paragraph, from lines 116 to 145, where no references are cited) regarding the similarity of the Openness trait of the Big 5 personality model to the Intuitive/Sensing modality of MBTI. Please supply these references in your revision.
**Language Note:** The Academic Editor has identified that the English language must be improved. PeerJ can provide language editing services - please contact us at [email protected] for pricing (be sure to provide your manuscript number and title). Alternatively, you should make your own arrangements to improve the language quality and provide details in your response letter. – PeerJ Staff
The authors provide satisfying answers to all comments.
No additional findings.
No additional findings.
No additional comments.
Please perform these remaining minor revisions
The revised mansucript has been significantly improved, and I recommed for publication.
The revised mansucript provided well desiged experiments.
The revised mansucript improved the presentation and discussion, with clear methodology and experiment support.
I believe the authors of the study have successfully addressed or provided additional context for most of my previous comments. I recommend that the editor publish the results of this research. However, I still encourage the authors to carefully review their text, as I have noticed several minor (technical) issues:
Lines 74-75: "The widely used model personality trait model" → "The widely used personality trait model"
Line 378: "Traditional Machine Learning, also known as traditional machine learning, refers to a class of..."
Tables 7, 8, 14, and 15: These tables still contain terms such as "% age" and "Shallow machine learning", which I had previously pointed out in my review.
I am satisfied with the way how experiment source code was reorganized.
Nothing to report here.
Nothing to report here.
The authors are requested to explain clearly the novelty of the proposed work, the validity of the proposed design and the results and include statistical analysis to demonstrate the significance of the results achieved with the proposed methods compared to the current literature. It is recommended to promptly answer the queries of the reviewers and update the manuscript with relevant details.
[# PeerJ Staff Note: It is PeerJ policy that additional references suggested during the peer-review process should *only* be included if the authors are in agreement that they are relevant and useful #]
Strength: 1) the study employs a wide range of machine learning (ML) and deep learning (DL) models, including shallow ML, ensemble models, and state-of-the-art architectures like Bi-LSTM and BERT. This comprehensive comparison provides a robust foundation for understanding model performance on personality prediction tasks. 2) Feature engineering is well-executed and compared, incorporating a mix of traditional (TF-IDF, POS tagging) and deep learning (word2vec, GloVe, and sentence embeddings) approaches. This breadth provides comprehensive comparison and understanding.
Weakness: 1) the manuscript provides detailed performance metrics for various models but lacks sufficient explanation or interpretation of the experimental results. This gap limits the reader's ability to understand why certain models perform better than others and what insights can be drawn from these results. 2) The manuscript does not clearly articulate the specific research gap it aims to address. While the study emphasizes applying ML and DL models for predicting the openness personality trait, it fails to specify how this work advances the field beyond existing literature or addresses unresolved questions.
1. The second contribution of this work is “Examination of diverse shallow ML models, ensemble models and advanced DL algorithms like LSTM and Bi-LSTM and transformer-based model BERT.", which seems less convincing given that models like LSTM and BERT are no longer considered cutting-edge. Incorporating Large Language Models (LLMs), such as GPT or LLaMA, as alternatives for embedding generation could provide a more contemporary approach.
2. The English language in this paper requires refinement to enhance clarity and ensure it is easily understood by an international audience. The current phrasing poses challenges to comprehension, with issues such as grammatical errors observed in lines 24–28 and unclear wording in lines 37–40 and 73–75 (Each trait represents a spectrum, with individuals adapting in the degree to which user reveals each trait according to the behavior). Thoroughly proofreading your manuscript is essential to ensure clarity, coherence, and the elimination of errors.
3. Another important point is that in Table 10, reference [38] did not conduct experiments on the MBTI dataset. This raises questions about the source of the reported 87% result and its relevance to the comparison.
4. The relationship between the NS dimension and openness within the Big Five personality framework should be further clarified. In particular, the rationale behind Table 1 requires stronger justification. Linking it to foundational works on MBTI and the conceptualization of openness in the Big Five framework would provide more depth and context.
5. More automatic personality recognition studies should be discussed to provide reader for a more comprehensive contexts. The below are some examples:
[1] Ghassemi, Sina, Tianyi Zhang, Ward Van Breda, Antonis Koutsoumpis, Janneke K. Oostrom, Djurre Holtrop, and Reinout E. de Vries. "Unsupervised multimodal learning for dependency-free personality recognition." IEEE transactions on affective computing (2023).
[2] Leekha, Maitree, Shahid Nawaz Khan, Harshita Srinivas, Rajiv Ratn Shah, and Jainendra Shukla. "VyaktitvaNirdharan: Multimodal Assessment of Personality and Trait Emotional Intelligence." IEEE Transactions on Affective Computing (2024).
[3] Song, Siyang, Shashank Jaiswal, Enrique Sanchez, Georgios Tzimiropoulos, Linlin Shen, and Michel Valstar. "Self-supervised learning of person-specific facial dynamics for automatic personality recognition." IEEE Transactions on Affective Computing 14, no. 1 (2021): 178-195.
[4] Song, Siyang, Zilong Shao, Shashank Jaiswal, Linlin Shen, Michel Valstar, and Hatice Gunes. "Learning person-specific cognition from facial reactions for automatic personality recognition." IEEE Transactions on Affective Computing 14, no. 4 (2022): 3048-3065.
[5] Junior, Julio CS Jacques, Yağmur Güçlütürk, Marc Pérez, Umut Güçlü, Carlos Andujar, Xavier Baró, Hugo Jair Escalante et al. "First impressions: A survey on vision-based apparent personality trait analysis." IEEE Transactions on Affective Computing 13, no. 1 (2019): 75-95.
**PeerJ Staff Note:** It is PeerJ policy that additional references suggested during the peer-review process should only be included if the authors are in agreement that they are relevant and useful.
The experimental design are generall comprehensive and good
no comment
no comment
The authors have effectively explained the core idea of the study and provided a solid overview of the theoretical foundations necessary for their analysis. The paper presents a comprehensive review of various artificial intelligence techniques used for predicting the openness personality trait based on content shared on social media. The literature review is thorough, and the comparison of machine learning (ML) and deep learning (DL) methods for predicting openness is well-structured.
Strengths
1. The authors offer a clear explanation of the theoretical basis and its connection to the applied techniques.
2. The comparison of AI methods provides valuable insights into the performance of different approaches.
Experimental Design
1. Data Splitting: For a fair comparison of ML and DL algorithms, the dataset should be split into training and holdout sets before training the models. Splitting the data after applying each algorithm may introduce bias and create unequal conditions for comparison.
2. Hyperparameter Tuning: Hyperparameter values are only provided for DL models. The hyperparameters for ML models, especially those directly compared in the results, should also be included, along with details on how they were determined (e.g., empirical methods or specific optimization techniques).
3. Cross-Validation: The code does not indicate the use of cross-validation for traditional ML models. Employing cross-validation would have been the best way to evaluate model accuracy and reduce variability in results.
4. Feature engineering: For ensemble models, it would have been beneficial to apply advanced feature engineering techniques, as ensemble methods often perform well when enhanced with better features.
5. Result Interpretation: The differences between the results of various models are relatively small. Care should be taken when interpreting these differences, as they could stem from statistical errors or random initialization of the models.
Reproducibility and Code Organization
1. The code should be organized into separate sections for each group of methods (e.g., ML, DL, Transformers), which would make it easier to follow and reproduce experiments.
2. To ensure reproducibility, the authors should specify the versions of Python libraries used or provide a requirements.txt file listing the libraries and their versions.
While the paper provides valuable insights and demonstrates a thoughtful analysis of ML and DL techniques for personality prediction, improvements in the organization, experimental methodology, and presentation would enhance its clarity and scientific rigor. Addressing the concerns raised above would significantly strengthen the study and its contribution to the field.
Title and Abstract
• The title does not adequately reflect the content of the study. A more suitable title, such as "Comparison of Experimental Results of Machine Learning and Deep Learning Algorithms for Predicting the Openness Personality Trait," would better represent the focus of the paper.
• The abstract is too generic and fails to effectively summarize the paper's content. It should be revised to highlight the study's core contributions and findings.
Figures and Tables
• Figures 4, 5, and 6: The authors should explicitly discuss the observations derived from these figures and explain how dataset characteristics, such as balance, impact the experiment. For instance, is the dataset balanced? If not, was any technique used for balancing, and what were the results?
Terminology:
• Replace uncommon terms such as "% age" (e.g., in Table 9) with "percentage" and "Shallow ML" with "traditional ML techniques." Abbreviations like "acc" in Tables 9 and 10 should be replaced with "accuracy."
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.