All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
The authors have improved the manuscript. I am glad to recommend the paper for acceptance.
[# PeerJ Staff Note - this decision was reviewed and approved by Jyotismita Chaki, a PeerJ Section Editor covering this Section #]
Based on the comments received from the reviewers, I am glad to recommend the paper for acceptance should the authors be willing to incorporate the minor changes recommended by the reviewers. Overall, the paper is in better shape now and the authors have attempted to address the comments of the reviewers.
no comments
no comments
no comments
The revisions have been made and article is now up to the mark.
1. The manuscript has some persistent issues, which were pointed out during the first round of a review. A graphical or tabular view of employed techniques is needed, to replicate the work for further investigation in the future, as asked in comment # 7 in the previous version of the revision. Still, the setup is left ambiguous.
2. Authors have inferred information from confusion matrices, without making any statistical analysis to illustrate the significance of obtained results. A comprehensive statistical analysis is required before the acceptance of the final version of this paper.
NA
NA
no comment
no comment
no comment
No
We have now received the comments from the respected reviewers. Based on the comments, we are asking for a re-submission of the paper after the authors carefully address the comments.
In addition, I would suggest that the authors include a section on Limitations of the work, that should clearly summarize the limitations of the dataset and the classification model. As one reviewer identified that the tasks such as annotation are subjective and should be done by more than one person, it is important that the authors mention this in the manuscript as a limitation of the dataset, particularly, the subjective voting of the human annotators or the inherent bias if the annotation was done by a single person.
The authors used clear and unambiguous, professional English in the article. The literature provided was relevant and sufficient. The raw data was shared and the figures and tables were professional. The results were relevant and self contained.
The research questions were well-defined, relevant and meaningful.
All underlying data have been provided. Conclusions are well-stated and linked to original research questions.
1. There are some abbreviations which were not explained like. TF-IDF
2. In section 5.4 Error Analysis, The results are presented in Table 5, but Table 6 is referred instead. Please recheck it.
The research work mainly focused on the deployment of CNNs and RNNs to classify tweets either offensive or not, trained over the dataset developed by the authors themselves. Authors have to consider the following points prior to the acceptance of their work.
1. Very generic abstract: authors should completely revise the abstract part.
2. Authors should work on the write-up part, by removing all grammatical errors and vague statements. There are numerous statements in the manuscript where the claim of an author is not supported by any reference.
3. As Pashto has been spoken in many regions of southeast Asia, which dialect of it is targeted in your work? Furthermore, data collection is one of the most sensitive steps in any project, and there is no detail that how biases and sentimental vague statements are handled by the authors.
4. The random selection of tweets for training purposes might increase the chances of skewness to a certain class, which may tarnish all the claims/results made by authors. That is one of the major points of concern.
5. Author has been claiming that it is one of the first works, which targets offensive language detection in Pashto, which seems doubtful.
6. It would be better if authors incorporate some graphical illustration of a complete layout/structure of the work.
7. Authors have employed DL models, but no specification (or structural layout) of these models has been mentioned. A list of parameters, hyperparameters, and even the dataset splitting method is ambiguous.
8. There should be a statistical analysis of the obtained results and the significance of true positives/true negatives should be compared with the false positives/false negative results.
Insufficient information.
The validity of the findings must be statistically analyzed by the author, by mentioning the set of parameters/hyperparameters they used, so that the doubt of model deployment as a black box is vanished.
Maybe, find the comments section.
Somehow, yes
yes
An interesting idea is presented in the article titled “Pashto Offensive Language Detection: A Benchmark Dataset and Monolingual Pashto BERT”. The authors have developed a benchmark dataset for experimental and evaluation purposes. In this paper, the authors have applied deep learning and tuned the BERT model for Pashto offensive language detection used on social media. The contents are presented in a comprehensive manner; however, I have a few concerns regarding the article:
1. During the development of the POLD dataset, what was the search query that was used to accumulate tweets from Twitter using Twitter API?
2. Provide a full description of the tweets (Twitter data) in the Dataset development section. How many of these were retweeted? how many were hashtags? Provide details of gender contribution and demographics locations (see help from)
(a) Khan, S., Khan, H. U., & Nazir, S. (2021). Offline Pashto characters dataset for OCR systems. Security and Communication Networks, 2021, 1-7.
3. In the Introduction section, the authors provided three objectives for their research work. However, a short description is needed along with each objective to improve readability and understandability. For example, in the second objective, the authors claimed that “we have developed…” So, how many records are there in the database, and how these records are distributed for the identification task?
4. In section 3.2.3 the authors claimed that the tweets are manually annotated. Do all the authors know the Pashto language? If not then how a single person annotated all the data? Mostly, in the annotation process, more than two people contribute. During a conflict, if more than half are agreed upon, a decision then it is followed. In this work, I didn’t find such a contribution. In short, this obligates your annotation task.
5. In the revised version can you generate a word-frequency diagram to show, what are the most “offensive” and “non-offensive” words or phrases used in the Tweets?
6. Also, can you generate a table to show what are the targeted people/user for offensive behavior? I mean can you perform this type of clustering using your model?
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.