All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
I have gone through the changes you made to the paper in response to my minor corrections, and I am generally happy with them. There is one sentence that is still ungrammatical: on line 104, you write: "How long of the impending saccade is related to parafoveal and foveal processing". I think you wanted to write:
"How is the length of the impending saccade related to parafoveal and foveal processing"
Please fix that in your final submission.
[# PeerJ Staff Note - this decision was reviewed and approved by Claire Fletcher-Flinn, a PeerJ Section Editor covering this Section #]
This revision is much improved and the authors have satisfactorily addressed the substantial issues raised by the reviewers, but there are still a number of minor issues that need to be addressed. I list these below.
1. English issues:
line 380:
"that each Chinese character
subtended a lager visual angle" -> larger
line 104:
"How long of saccade length is related to parafoveal and foveal processing (Liu et al., 2015,
105 2018)." Not sure what is intended here.
line 256:
"A fixation time of 4268 contributed to the analyses." What does this sentence mean?
2. Around line 288:
"We
employed models for analyzing the data of log-transformed ISL for younger and older readers separately to clarify this interaction, and observed a negative interaction of Predictability × Frequency in young readers (b=-0.054, SE=0.039, t=-1.391, p=0.172) and a positive interaction in older readers (maximal random effect structures model did not converge, and so this model on only included the random effect of participants; b=0.013, SE=0.017, t=0.748, p=0.454)."
I don't understand how you can claim you found a negative interaction when your p-value is 0.172. Please rephrase this sentence to state that you didn't find an interaction. The same goes for the positive interaction claimed above; please restate that you didn't find an interaction. The same problem as above arises in line 295 on:
"To further investigate the three-way interaction (Group × Predictability × Frequency) in OSL, we employed models for analyzing the data of younger and older readers separately, which yielded a negative interaction value of Predictability × Frequency in young readers (raw OSL: b=-0.026, SE=0.157, t=-0.168, p=0.867; log-transformed OSL: b=-0.010, SE=0.035, t=-0.287, p=0.776) and a positive interaction in older readers (raw OSL: b=0.234, SE=0.185, t=1.265, p=0.213; log300
transformed OSL: b=0.052, SE=0.024, t=2.168, p=0.036)."
3. line 321:
Please rephrase this, removing the word prove and replacing it with suggest:
"For the interaction effects involving age with the linguistic characteristics of predictability
and frequency, our findings further prove that age promotes the effects of frequency."
The two reviewers have provided many constructive and useful comments on the paper and on the statistical analyses (including the inferences that can be drawn from these). I would like to request you to revise the paper following the reviewers' suggestions. In your response letter to me, please provide a point-by-point response to each of the reviewers' comments, explaining how you dealt with suggestion/comment from the reviewer in the revision.
Please make sure that the revised paper has been edited by a native or fluent speaker of English before submitting the revision.
The second reviewer analyzed your data himself, his code is appended below. His analysis may be useful to you as you revise your paper; but if you use his code please acknowledge it in a footnote or in the acknowledgements section.
It would be very helpful if your data and code release is in fully reproducible form. The reviewer had to reconstruct the analyses from the description in the paper; this is not optimal. As an example, I wanted to point you to how I release data and code with my papers. https://osf.io/eyphj/. This is just an example, and I am sure one can do better; however, the guiding principle should be that the reader should be able to re-run the analyses described in the paper without having to guess at what was done.
##############################
library(tidyverse)
library(lmerTest)
# NOTE: Before the data can be loaded, '#NULL!' needs to be replaced
# by the empty string in the data file.
read_csv("peerj-40725-CSVPreFreAging120IASReportTTsub-PEERJ.csv") %>%
mutate(
group = factor(group, levels=1:2, labels=c("young", "old")),
freq = factor(freq, levels=c("L", "H")),
pred = factor(pred, levels=c("L", "H")),
ref = as.logical(ref),
skip = as.logical(skip),
ffd = 10**logffd,
gzd = 10**loggd,
tvt = 10**logidt) %>%
rename(isl = amplitude) %>%
select(-logidt, -logffd, -loggd) -> d1
#
# Duration measures:
#
# Descriptive stats:
d1 %>%
group_by(freq, pred, group) %>%
summarize(
mtvt = mean(tvt, na.rm=TRUE),
mffd = mean(ffd, na.rm=TRUE),
mgzd = mean(gzd, na.rm=TRUE))
d1 %>%
group_by(freq, pred, group, item) %>%
summarize(
mtvt = mean(tvt, na.rm=TRUE),
mffd = mean(ffd, na.rm=TRUE),
mgzd = mean(gzd, na.rm=TRUE)) %>%
summarize(
mtvt = mean(mtvt),
mffd = mean(mffd),
mgzd = mean(mgzd))
# Inferential stats:
contr.sum(2)/2 -> contrasts(d1$group)
contr.sum(2)/2 -> contrasts(d1$freq)
contr.sum(2)/2 -> contrasts(d1$pred)
# Running intercept-only models because running full models takes too
# long. Just need the approximate picture here:
lmer(log10(ffd) ~ freq * pred * group +
(1 | id) +
(1 | item),
d1) -> m.ffd
summary(m.ffd)
lmer(log10(gzd) ~ freq * pred * group +
(1 | id) +
(1 | item),
d1) -> m.gzd
summary(m.gzd)
lmer(log10(tvt) ~ freq * pred * group +
(1 | id) +
(1 | item),
d1) -> m.tvt
summary(m.tvt)
#
# Skipping:
#
glmer(skip ~ freq * pred * group +
(1 | id) +
(1 | item),
d1,
family = binomial()) -> m.skip
summary(m.skip)
#
# Refixation:
#
glmer(ref ~ freq * pred * group +
(1 | id) +
(1 | item),
d1,
family = binomial()) -> m.ref
summary(m.ref)
#
# Incoming saccade amplitude:
#
# Descriptive stats:
d1 %>%
group_by(freq, pred, group, item) %>%
summarize(
misl = mean(isl, na.rm=TRUE)) %>%
summarize(
misl = mean(misl))
# Inferential stats:
# Running intercept-only models because running full models takes too
# long. Just need the approximate picture here:
lmer(log(isl) ~ freq * pred * group +
(1 | id) +
(1 | item),
d1) -> m.isl
summary(m.isl)
#
# Outgoing saccade amplitude:
#
read_csv("peerj-40725-CSVPreFreAging120IASSacT-PEERJ.csv") %>%
mutate(
group = factor(group, levels=1:2, labels=c("young", "old")),
freq = factor(freq, levels=c("L", "H")),
pred = factor(pred, levels=c("L", "H"))) %>%
rename(osl = amplitude1) -> d2
# Descriptive stats:
d2 %>%
group_by(freq, pred, group, item) %>%
summarize(
mosl = mean(osl, na.rm=TRUE)) %>%
summarize(
mosl = mean(mosl))
# Inferential stats:
contr.sum(2)/2 -> contrasts(d2$group)
contr.sum(2)/2 -> contrasts(d2$freq)
contr.sum(2)/2 -> contrasts(d2$pred)
# Running intercept-only models because running full models takes too
# long. Just need the approximate picture here:
lmer(log(osl) ~ freq * pred * group +
(1 | id) +
(1 | item),
d2) -> m.osl
summary(m.osl)
######################
### END OF CODE
######################
[# PeerJ Staff Note: The review process has identified that the English language must be improved. PeerJ can provide language editing services - please contact us at [email protected] for pricing (be sure to provide your manuscript number and title) #]
The writing is generally easy to follow but it weak in some sections, including when the predictions and presented, and would benefit from a revision by a native or highly skilled English language user.
The authors need also to write for non-expert audiences. They use technical expressions such as gaze duration in the abstract without explanation and need also to detail what is measured by the different eye movement metrics they use.
The authors also use technical terms such as Skip.pro rather than transparent terms when discussing findings. They should revise the use of this technical language to describe what they find more clearly.
The hypotheses are focused on saccade metric effects. Effects on reading times for words are also reported and the effects described here were in principle predictable in advance based on previous research. The manuscript would be stronger with predictions for these effects included. Without encouraging HARKing, it should be possible to include predictions for these measures that would follow from the previous Chinese research reviewed in the manuscript.
The manuscript reports only word-level findings. It would also be beneficial, both to establish that the older adults are typical of older readers in other research and to provide data for meta-analysis, to report age differences in basic sentence-level effects. As above, it should be possible to generate straightforward predictions based on previous research findings without encouraging HARKing.
With respect to the review of the prior literature, the manuscript is overly focused on saccadic effects and should say more about reading time effects. With reference to predictability effects, it needs to review recent research looking at age differences in such effects in alphabetic languages (Choi et at., 2017, Psychology and Aging; Steen-Baker et al., 2017, Psychology and Aging; Zhao et al., 2019, Psychology and Aging).
Article structure is generally good although the Discussion is a little weak and could be more clearly structured around the different aspects of the findings. There is appropriate use of tables and figures.
The table showing example sentences should include the sentences in Chinese as well as the translations into English.
The reporting of statistical findings is difficult to follow and requires greater clarity. I would suggest removing the statistics to a table and then clearly explaining the pattern of effects in the text. The absence of predictions make this section difficult to follow. There appears to be an age-group x frequency interaction, which would replicate previous findings. This should be clearer. By comparison, there does not appear to an age-group x predictability interaction, which is inconsistent with previous findings. This needs to be clear too. I would suggest reporting the regression-path / go-past measure as several recent studies show age differences in predictability effects in this measure which may be evidence for age differences in sentence integration effects (Steen-Baker et a., 2017, Psychology and Aging; Zhao et al., 2019, Psychology and Aging).
The complex interaction effect in saccade length for age group and frequency requires more detailed explanation. What does it show and how do the statistics support that interpretation?
The reviewers need to discuss their findings more clearly in relation to previous investigations of age differences in effects of frequency and predictability on reading. It's unfortunate that a study was published only very recently showing an age difference in predictability effects in Chinese reading (Zhao et al., 2019). They should consider their findings with respect to this study.
The patterns of findings concerning saccade metrics need to be investigated more carefully (see comments above) but also described more clearly in the discussion section and linked to the conclusions more precisely. At present the link is rather vague and leads to a very general conclusion.
The target word stimuli appear to be well constructed.
The research questions are well-defined. However, there are weaknesses to the hypotheses as noted above. The study provides an important attempt to replicate key findings, an original investigation of the combined effects of aging, predictability and frequency and an original investigation of these effects on localised saccade metrics.
The eye-tracker has a relatively low spatial and temporal resolution for the questions that are being addressed. I would like this difference in methods compared to other studies to at least be acknowledged in the manuscript.
Methods generally are described in sufficient detail. However, in a study focused on saccade metrics involved in incoming saccade effects of word frequency and predictability, we need to know about the characteristics of the word(s) to the left of the target word, as these will mediate such effects. This information is absent from the report and so it not possible to interpret the saccadic findings clearly.
There was a marked difference in comprehension scores for the two age groups, with older readers scoring an average 77% accuracy. This is worrying. The authors attempt to address this but I remain concerned. How many older adults scored less than 70% correct? The possibility that older readers may have experienced basic comprehension difficulty needs to be taken seriously and acknowledged when discussing findings.
The findings with respect to reading times for words are valid and interesting. These show an interaction between age group and word frequency that replicates previous findings. The absence of an interaction between age group and predictability is interesting and counter to previous findings. It would be interesting to know if this effect is still null if untransformed (rather than log-transformed data) are examined. I would suggest reporting both transformed and untransformed analyses for completeness.
The saccade metrics are difficult to interpret because launch site of incoming saccades is not known nor the characteristics of the preceding words. Similarly, the launch site of the outgoing saccade is also not reported but may systematically vary across young and older adults. These should be reported. The findings are also likely to be affected by the higher re-fixation rates of the older adults. Such issues should be considered carefully in the Discussion.
The authors report an interesting study investigating age effects in Chinese reading. The paper is well structured and the discussion of the existing literature looks appropriate to me. Some comments regarding the presentation:
1. The patters in the data are reported verbally and in terms of numerical statistical inferences. Given the complexity of the findings (e.g., three-way interactions), readers would greatly benefit from a graphical representations of the results, both for descriptive stats and inferential stats.
2. The paper does not follow APA guidelines, e.g. with respect to the document structure. Not sure, thought, whether APA is relevant for PeerJ.
3. There is no information about the nature of the comprehension questions and their frequency. This should be added.
4. Fillers were apparently not used. That’s okay in this particular case, but since it is unusual it is worth mentioning.
5. The reporting with regard to the statistical analysis was incomplete and I was unable to reconstruct the precise analysis that the authors conducted:
- In the LMM analysis, what was the contrast coding for age group?
- From the data, I can infer that log10 was used to transform the reading time measures, but this is not mentioned in the paper. This information is needed to make sense of the parameter estimates and should be added.
- It is not clear whether saccade length measures were only for first-pass saccades or also for later saccades. Please clarify.
6. I suggest that the authors have a look at the paper below on the effect of age on German reading behavior. It reports relevant findings on many of the issues discussed in the present manuscript.
- von der Malsburg, T., Kliegl, R., & Vasishth,
A. (2015). Determinants of scanpath regularity in reading. Cognitive
Science, 39(7), 1675–1703. http://dx.doi.org/10.1111/cogs.12208
7. The paper is understandable but there are some problems with English grammar and some typos that should be fixed. This may require input from an English native speaker.
I don’t speak Chinese but the design looks largely appropriate to me. There is one potential issue though with the design of the sentences: The HF-P and the LF-U conditions have a different sentence prefix than the LF-P and HF-U conditions (see Table 2). I think this could give rise to differential spill-over effects which could introduce spurious interaction effects or mask true interaction effects. It would be good to discuss this issue briefly in the manuscript and to qualify conclusions accordingly.
The statistical analysis look largely appropriate and I could replicate most findings in my own analysis of the data. My code is attached to this review; please share with the authors. The biggest issue is multiple comparisons. The author’s hypotheses are relatively vague and each hypothesis is tested multiple times (details below). The best way to address this is using Bonferroni corrections. I examined the impact of Bonferroni corrections in my own analysis and luckily most (but not all) of the findings hold up. For details on the issue of multiple comparisons in the context of eye-tracking studies I recommend that the authors read this paper:
- von der Malsburg, T., & Angele, B. (2016). False positives and other statistical errors in standard analyses of eye movements in reading. Journal of Memory and Language, 94(), 119–133. http://dx.doi.org/10.1016/j.jml.2016.10.003
Detailed comments:
- In the discussion, the authors make claims about parafoveal processing. I think any such claims would need support from boundary-paradigm experiments. The present experiment provides only indirect and therefore weak information about parafoveal processing.
- I could not reproduce the descriptive stats in table 3. My own values for first pass reading time and gaze duration are very close but my values for total viewing time are considerably lower than the author’s.
- There were no predictions for duration measures (as far as I can see), but I imagine that the authors were primarily interested in the interactions of predictability and frequency with age. There were three such interactions per each of the three measures, so 9 tests of the general hypothesis (age has an impact in frequency and predictability effects). The freq × group interaction is just borderline significant and does not survive a correction for multiple comparisons. None of the other interactions were significant by conventional standards. So, no age interactions in duration measures. However, most main effects (frequency, predictability, age) do survive a Bonferroni correction.
- It appears that the authors analyzed raw saccade length. Since this variable is strictly positive, some kind of transform is needed. My own investigation of the data shows that a log-transform yields close to normally distributed residuals. My evaluation of saccade length effects in the comments below is based on my own log-transformed analysis which yields considerably more robust results than the analysis of raw saccade length conducted by the authors. Note, though, that due to time limitations I ran intercept-only models which tend to be anticonservative. (I calculated p-values using the lmerTest package.) The authors should redo the analysis with log-transformed saccade length and report only results from those analyses.
- The prediction for incoming saccade length (ISL) was that age may somehow interact with frequency and predictability. There are three concrete ways in which this could pan happen: 1. effect in frequency × age interaction, 2. predictability × age, 3. frequency × predictability × age. A Bonferroni correction is therefore appropriate (new α = 0.05 / 3 ≈ 0.017). The three-way interaction survives this correction but the frequency × group interaction does not.
- The prediction for outgoing saccade length (OSL) was again that age may somehow interact with frequency and predictability. The same interactions are relevant as for ISL. Unfortunately, the observed interaction effects do not survive the Bonferroni correction.
- In OSL, I found a highly reliable predictability effect that the authors didn’t find. When predictability was low, the outgoing saccade was shorter, consistent with the author’s earlier findings (Liu, Guo, Yu, Reichle, 2018). Nice!
- I can’t get the glmer models for skipping and refixation to converge even with intercept-only random effects. How did the authors deal with that? More generally: If a maximal model fails to converge, the authors should specify which simplified model they’re reporting and by which procedure they arrived at this model.
- In the discussion, the authors interpret various null results. However, it is unclear whether these were due to the true absence of an effect or due to a lack of power. While useful, I would tone this discussion down.
Overall a worthwhile contribution that provides some interesting new insights along with replications of effects reported earlier. The statistical analysis needs some (easy) fixes and as a result some results will change and the discussion will have to be adjusted accordingly. However, the key findings will likely hold up and some effects will actually become stronger and clearer. I would happily recommend publication once the statistical issues are addressed.
Comments regarding the supplementary information: I applaud the authors for publishing their data. Having the data really helped me to better understand the results. Some possible improvements: 1. The read-me file describing the variables in the two CSV files is not entirely accurate. Please fix. It’s also not clear why there are two files. Isn’t most of the information redundant? My impression is that one table should be sufficient. If not, please explain why. 2. Comprehension performance not included in the data and should be added. 3. The analysis code is not part of the supplementary information. From the description in the manuscript alone I was not able to fully reconstruct the analysis. Please add the R code in the supplementary section.
Please resubmit this paper with two changes:
- Please provide a README file explaining what each column in your csv file is.
- In the section confirming that all authors have approved the submission, please add the name of the third author.
Once you resubmit, I will send this paper out for review.
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.