All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
Dear Authors,
You have addressed all of the reviewers' comments.
In its current form, the paper has improved significantly and it ready for publication.
M.P.
[# PeerJ Staff Note - this decision was reviewed and approved by Claudio Ardagna, a PeerJ Section Editor covering this Section #]
No further comments (the resubmitted version of the paper addressed all my previous concerns and comments to the authors).
No further comments (the resubmitted version of the paper addressed all my previous concerns and comments to the authors).
No further comments (the resubmitted version of the paper addressed all my previous concerns and comments to the authors).
The authors have addressed all my initial concerns and comments in the paper. In its current form, the paper has improved significantly and is now suitable for consideration for acceptance in the journal.
I would like to thank the authors for their thorough revisions and for addressing all my comments and concerns. I appreciate their efforts in improving the quality of the paper.
Dear Authors,
please take in to account the suggestion given by reviewer #1. In particular, look at the 3rd column in the annotated attached file.
Thanks,
M.P.
The manuscript is written in professional English, and contracted forms have been addressed in this revision, as requested. However, some sections could benefit from clearer phrasing. Ambiguous terminology should be refined to improve readability.
The literature references are relevant and well-integrated. However, the authors should ensure ensure all citations include full details, such as access dates where applicable (e.g., Ballatore, 2015), is necessary.
While additional details on classification approaches have been added, there is still a need to clarify how RAT improves upon previous classification methods. While the authors discuss its application in various studies, they do not provide empirical data quantifying its effectiveness.
The methodology for evaluating RAT's performance remains unclear and necessitates further revision. Further clarification and cleared details are needed on data collection to ensure transparency and replication. The revised explanation of barriers to data access is useful, as well as the additional discussion on search engine self-interest. The revised explanation of the software's focus is an improvement.
I believe that this can be addressed by providing further clarification to my queries raised in this round.
The paper highlights that RAT has been used in multiple studies, which is a strong indication of its relevance and applicability. The authors have successfully addressed several key aspects of RAT’s application. However, the methodology and validity of the collected data require further refinement. While the authors have made efforts to describe their approach, more detailed documentation on data integrity and methodology is needed to ensure reliability (please refer to the attached file with reviewer 1's comment column). Researchers who wish to replicate the results or apply RAT to their own studies would benefit from more detailed information on the implementation process. Additionally, more validation steps should be incorporated to ensure robustness in automated data collection.
Overall conclusions are well stated and link to original research question, but they need further support from the results and methodology.
Please refer to the attached file for more details.
Overall, this paper presents an important contribution to search engine data analysis by introducing RAT as a versatile tool. The study is well-structured, and the methodology is promising for advancing research in multiple domains.
However, addressing the concerns outlined above and in the attached file (particularly in terms of data integrity, bias mitigation, and generalisability) will strengthen the impact and reliability of the findings. Providing additional empirical validation and ensuring transparency in data will further improve the study's contribution to the field. I appreciate the author's efforts and look forward to seeing these improvements incorporated.
Good
Good
Good
No
Dear Authors,
Please follow the suggestion given by the reviewers.
M.P.
The article uses clear and professional language throughout, but some areas could benefit from further clarification. For example, in the section discussing health and media studies, more explicit explanations of how RAT could be integrated into existing research could be added to improve clarity. Additionally, the focus of the software needs to be more clearly defined earlier in the document to avoid any confusion about its scope. Authors should avoid contractions throughout the document for a more formal tone.
The research question is well-defined, particularly in terms of exploring the potential applications of RAT. However, the article would benefit from a more direct explanation of how RAT fills an existing gap in current research practices. The mention of the issue where the scraper returned fewer results than expected is an important point to address, as it could potentially undermine the validity of results. A more thorough explanation of why this issue occurs and how it is being resolved would add to the rigor.
The data seems to be robust, but the issue of incomplete data scraping (as mentioned earlier) needs further clarification. The conclusions are aligned with the original research questions, but more emphasis could be placed on how RAT directly improves research practices.
Please see attached file.
GOOD
ENOUGH
ENOUGH
The authors have made the requested improvements to a large extent, but I know that giving url information in parentheses in the text does not comply with the journal format. First, the Web platform name is written and details of citation should be provided in the references section. Please check the journal format again.
Some terms should be verified e.g. analysis --> anlayzing
Please address all the comments from the reviewers
Overall, the article presents a clear and concise exploration of the RAT and its functionalities. However, several areas required clarification and enhancement to ensure the comprehensive and effectiveness of the study. For instance, the concise manner in which the tool is discussed may present challenges for readers seeking a more detailed understanding of its functionalities. The literature references are sufficient and up to date, although some references feature broken links requiring rectification. This oversight should be addressed to ensure the integrity of the citation network. Figures numbering should be reviewed as for some figures is not sequential throughout the manuscript.
Please see attached file with the full list of comments.
Overall the introduction provides a clear overview of the motivation behind developing the RAT software and its intended functionalities. The background information provided about the lack of access to search engine data for researchers helps contextualise the need for RAT. However, some sections could be enhanced by providing more context, references to support the need of developing such software and the limitations researchers face in accessing search engine data. The introduction briefly mentions existing tools that cover some of RAT’s functionalities but highlights the limitations of these tools, such as being outdated or lacking flexibility. Providing a more detailed comparison with existing tools, including their strengths and weaknesses, could strengthen the argument for the necessity of RAT and further highlight its contributions to the field. Furthermore, the authors should emphasize RAT’s flexibility and discuss more effectively how the tool address the diverse needs of researchers.
Both methods and limitations are sufficiently detailed throughout the manuscript, and code has been provided for transparency.
The authors provide a clear statement of purpose, indicating the aim of developing the RAT software and bringing focus to the purpose of the research and the proposed solution for addressing data accessibility limitation from search engines. While the study effectively outlines the challenges associated with web scraping search engines, such as HTML and CSS changes and IP blocking, the discussion of results could be expanded to provide a deeper analysis of the impact of the RAT tool. Elaborating on the findings would not only enhance the understanding of how the tool addresses these challenges but also highlights its significance in terms of data accessibility.
Your research stands out as a remarkable achievement, making a substantial scientific contribution and yielding a noteworthy final product. Strengthening the clarity of your responses to the following questions in your article could enhance its impact.
Some revisions expected:
"A SERP is the page a search engine returns after a user submits a search query. If the structure of the SERPs has changed, it is mandatory to update the search engine. Jobs can also fail when the search engine operators temporarily block the scraper’s IP address because of too many requests from the same server have been detected." -Please expand the statement to provide more details on how challenges related to updating scrapers due to changes in search engine results pages (SERPs) are managed and mitigated by RAT.
-What is the max no used by the tool? Specify the max number of attempts allowed when stating "RAT is designed to automatically reset and restart the jobs after a configurable time up to a maximum number of attempts. Errors can also occur when capturing the source code of a URL, e.g., when the URL is inaccessible. In this process, jobs are automatically repeated up to a predefined maximum number of attempts…"
-Provide concrete examples or explanations for how user feedback has directly influenced the development or improvement of RAT, despite mentioning usability testing and a feedback management system.
-Authors can demonstrate the practical benefits of RAT in different fields to promote wider adoption beyond its potential impact on case research.
-By considering various features of search engines, authors could compare RAT with other search engines; It may be helpful to include a comparison table highlighting the differences, advantages, and disadvantages. Authors can expand the statement "Some software tools that at least cover some of RAT’s functionality have been developed in the past. However, these tools have either been developed for a single study only..."
Minor corrections:
Consider to use passive sentences and academic article jargon instead of first-person sentences like "we will speak" and "we designed."
Rigorous investigation performed to a high technical & ethical standard.
Conclusions are well stated, linked to original research question & limited to supporting results
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.