All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
The authors satisfied the requests of the previous reviewers and editors and therefore I can recommend this article for acceptance.
[# PeerJ Staff Note - this decision was reviewed and approved by Shawn Gomez, a PeerJ Section Editor covering this Section #]
-
-
-
Thank you for the revision. I find the latest version of the manuscript, the implemented changes, and the responses to the reviewers’ comments to be adequate. Best regards.
Please pay particular attention to the comments from Reviewer 3.
**PeerJ Staff Note:** Please ensure that all review, editorial, and staff comments are addressed in a response letter and that any edits or clarifications mentioned in the letter are also inserted into the revised manuscript where appropriate.
This is a very good manuscript that presents novel and significant information. I have made some comments and suggestions to improve the paper.
This manuscript effectively enhances technology.
The abstract is adequate and suitable for this kind of paper. A brief description of the structure (layout) of the paper may be added at the end of the introduction.
The Optimized explanation framework and Discussion provide a general summary of the study, but lack a clear emphasis on the key contributions and their broader significance. It would be beneficial to revise this section to more explicitly highlight the main innovations of the research, particularly the novel aspects of diabetes. To this end, your manuscript MUST include at least 8 additional relevant references published in ACADEMIC JOURNALS in 2021, 2022, 2023, and 2024.
What is the future direction of this study that must be included in the conclusion section?
Experimental work is missing and is not up to the mark
No validation is done
All the Figures should be visible. Authors must revise the manuscript and check the entire manuscript for typos, spelling mistakes, and grammatical errors.
Language & Clarity
• The language used in the manuscript is professional and technical but may require refinement in some parts to ensure unambiguous communication, particularly for an international audience. Some sections (especially when describing complex methods and results) might benefit from clearer explanations to avoid confusion.
Introduction
• Clarity: The introduction adequately presents the subject—Type 2 Diabetes (T2D) and its association with the gut microbiome. It defines the problem and outlines the importance of improving predictive models for T2D through the use of explainable AI.
• Motivation: The motivation for the study is well-established, particularly in highlighting the challenge of identifying effective explainers for machine learning models in medical contexts. However, it could delve deeper into why current methodologies are insufficient and how this work fills that gap.
Literature Review
• Relevance & Referencing: The manuscript cites several relevant studies in the field of AI and microbiome research for T2D. However, the literature could benefit from more recent studies that highlight advancements in explainability for machine learning models, especially regarding cross-dataset generalization. Some additional references to other explainability frameworks may strengthen the argument.
Structure
• PeerJ Standards: The manuscript follows the general structure expected for scientific articles. The introduction, methodology, results, and discussion sections are clearly defined. However, the flow between sections could be improved to provide smoother transitions. The manuscript adheres to PeerJ guidelines but does not fully justify the reasons for some methodological deviations, such as the integration of multiple explainers.
Relevance
• The content aligns well with the aims and scope of the journal. The focus on explainable AI for medical predictions is relevant and contributes to the current literature on AI applications in healthcare.
Rigorous Investigation
• Technical and Ethical Soundness: The methods described are technically sound. The manuscript uses well-known models (CatBoost and FT-Transformer) and explainability tools (LIME, SHAP). There’s sufficient detail about the model architectures and the data used for training. Ethical considerations about the use of human microbiome data are not explicitly discussed, which might be an area for improvement.
• Replication: The article includes sufficient details to replicate the methods, including dataset references and feature extraction methods. However, the inclusion of code or detailed workflows (like GitHub links) would enhance replicability.
Data Preprocessing
• Adequacy: The data preprocessing pipeline is well-described, with steps for quality control, trimming, and removing human genomes. This is important, as microbiome data often comes with noise and biases. The description of preprocessing steps like quality filtering (using fastp) and removal of human genomic data (via Bowtie2) is clear. However, it would be useful to include more specifics on how missing data or outliers were handled.
Evaluation Methods
• Clarity: The evaluation methods are adequately explained. The manuscript uses a robust set of metrics (AUC, ACC, Recall, Precision, F1-Score) to assess model performance. The explanation of why these metrics were chosen is clear, and they are appropriate for the dataset's characteristics (imbalanced classes).
• Model Selection: The selection of CatBoost and FT-Transformer is well-justified given their performance on tabular data. The justification for why these models were chosen over others is also valid, though additional models could have been considered for comparison to strengthen the argument.
Reproducibility
• The results show good consistency across different trials, and the study offers a clear methodology for reproducing the results. The AUC values are consistently reported, and the models perform well across both datasets (EW-T2D and C-T2D). However, including the exact code and data access information would ensure the study’s reproducibility.
Data Consistency
• Results Consistency: The results support the conclusions drawn in the manuscript. The top 100 KO features identified by the CIAE framework consistently perform well in T2D prediction tasks. The manuscript discusses the generalization ability of the model across datasets, showing that the findings are not overfitted to a single dataset. This suggests the models are robust.
Limitations
• Stated Limitations: The manuscript briefly mentions the limitations of using high-dimensional data in the context of T2D prediction, including the challenge of dealing with a low sample size. However, the discussion on limitations is limited. It could be more detailed, particularly concerning potential biases in the microbiome datasets and how they could affect the model’s generalization.
• Overall Quality: The manuscript is of good quality, with robust experiments and an innovative approach in integrating multiple explainers. The explanation framework (CIAE) is an interesting contribution to the field of AI in healthcare. However, the manuscript could benefit from a more thorough exploration of the ethical implications of using human microbiome data. Additionally, the literature review could be more comprehensive by including more recent studies.
• Clarity of Writing: The manuscript could be more concise in some sections, especially when explaining the technical details of the algorithms used. Some complex sentences could be simplified for better readability.
All comments have been added in detail to the last section.
All comments have been added in detail to the last section.
All comments have been added in detail to the last section.
Review Report for PeerJ Computer Science
(CIAE: A Consistency- and Informativeness-Aware Explanation framework for improving type 2 diabetes prediction and mechanistic insights)
1. This study proposes the Consistency- and Informativeness-aware Explanation Framework (CIAE), which integrates multiple explainers to improve the interpretability of machine learning models analyzing the gut microbiome's role in Type 2 Diabetes, achieving robust and generalizable performance while highlighting key KO features linked to glucose metabolism and diabetes.
2. In the introduction, both the importance of the subject and the literature are included, and it is recommended to add a detailed literature table to emphasize the place of this study in the literature more clearly.
3. Using more than one dataset in the study, instead of relying on a single dataset, has increased the quality of the study. In addition, it is very important that the dataset has undergone important preprocessing steps instead of using it raw.
4. Very basic classifications were used. However, the results obtained show that the classifiers are at a sufficient level in terms of affecting the problem solution.
5. Although the results are sufficient, there are some deficiencies, especially in terms of evaluation metric types. From this perspective, it is recommended to obtain the receiver operating characteristic curve and Matthews correlation coefficient score.
In conclusion, the study has a high potential to contribute to the literature, but the above sections need to be taken into consideration.
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.