Validation of a battery of inhibitory control tasks reveals a multifaceted structure in non-human primates

Subjects

All the adult rhesus macaques (M. mulatta) taking part in this study were from the breeding colony of the Medical Research Council’s Center for Macaques (MRC-CFM) in Porton Down, United Kingdom. Each group had access to an indoor free-roaming room (3.35 × 8.04 × 2.8 m) and an adjacent caged area (1.5 × 6.12 × 2.8 m), with a minimum total space of 3.5 m3/breeding animal in the largest groups. All rooms were temperature controlled (20 °C ± 5) with humidity at 55% +/−10. Each free-roaming area had a large bay window at one end facing outdoors and allowing a natural day-night cycle. At the other end of each room was an internal window fitted with movable mirrors which the monkey could control using a handle, allowing them to view the activities outside their area. Rooms were enriched with climbing structures (platforms, poles, fire hose and ladders) and enrichment devices (food puzzles, boxes, plastic barrels and balls, and small plastic blocks attached to structures or walls). Subjects received a supply of fruit and vegetables, dried forage mix (cereal, peas, beans, lentils etc.), bread and boiled eggs, in the morning and afternoon, with enough food to last for a 24 h period. All subjects had access to water and food prior to and during the experiment. Eighteen of the subjects already participated in a behavioural study involving looking at pictures (Bethell et al., 2019) and all of them were familiar with basic training and clicker procedures. However, none of them had experience with touchscreen experiments. Thirty subjects (14 males, 16 females; aged from 3 to 17 years old, mean age in years M ± SD = 8.10 ± 4.05, N = 30) started the touch screen training phases (see Supplemental materials, Training phases) but only 21 (12 males and nine females, aged from 3 to 17 years old, mean age in years M ± SD = 8.9 ± 4.41) successfully completed the training and were able to take took part in the experiment. The subjects were housed in 14 different social groups with an average of 12 individuals per group. Hierarchy, calculated in each group using David’s scores (David, 1987), was provided by the head of research of the facility (see Supplemental materials, Rank calculations). Agonistic behaviours including threats, displacements, chases, and physical conflict were recorded to assess the hierarchy. The caretakers regularly monitored the groups, and David’s scores were updated accordingly. Using video recordings of training and test sessions, a blind observer coded agonistic interactions between the tested individual and other conspecifics to verify the given ranks for each group (see Supplemental material). If a male never lost, he was considered high ranking. If the female never lost against other females, she was considered high-ranking, otherwise she was considered low-ranking.

Ethics

This study was approved by the Animal Welfare and Ethical Review Body of the University of Portsmouth, AWERB no. 4015B and by the MRC-CFM’s Animal Welfare and Ethical Review Body, ARWEB no. CFM2019E002 and was part of the Macaque Cognition Project. All research was carried out in accordance with ethical guidelines for work with non-human primates (NC3Rs, 2017).

Apparatus

To minimise stress, all tests were conducted in the enclosure, with no isolation from the social group; meeting recent and important ethical considerations on animal welfare and cognition (Cronin et al., 2017; Jacobson et al., 2018). The macaques had free and voluntary access to the apparatus and were never restrained; at any point subjects could leave the experiment and return voluntarily. For the experimental task, the set up was customized to be transported from one cage to another. Outside the cage, a laptop was connected via USB and HDMI cables to a capacitive touchscreen (ELO 1590L, frequency of 60 Hz, 19″ in diagonal, resolution 1,280 × 1,024 pixels). The program Elo touch solution 6.9.20 was used for calibration. The laptop screen duplicated the touchscreen display to be able to follow the experiment’s progress. The touchscreen was attached to the cage bars and the position was adjusted to each individual. All experimental procedures including stimulus presentation and response collection were carried out using MATLAB (version R 2018b, using Psychtoolbox-3.0.15 functions), under Windows 10. The MATLAB scripts were specifically conceived for the needs of this study; an individual progression file allowed the experimenter to abort and come back to the same point of a running session (see Supplemental Material for MATLAB CODE). If a trial was aborted the response latency was not recorded. The computer gave auditory feedback in response to the subject’s performance. All sessions were videotaped with one digital video camera (Sony HDR-CX330EB). The rewards (dry raisins) for each correct answer were given by hand.

General procedure

When the program for a specific task was launched, the experimenter entered the name of the individual. When more than one individual per cage were tested or when other individuals from the group were interacting with the touch screen, a research assistant was distracting the other macaques at the opposite side of the room. Every session was initiated by the subject touching a red cross located in the centre of the screen, starting the time recording. The session was aborted when another individual displaced the tested individual or interacted with the touchscreen. If the subject left the testing area or was not focusing attention on the screen the session was aborted. If the target was not touched within the specified time limit (see task descriptions below for specific time limit), the built-in timer of the program was paused, and a red cross appeared in the centre of the screen until the session was resumed by touching it. The response latencies above the time limits were not used in the analysis. If the subject stayed inactive for more than 5 min the experiment was stopped and continued the next testing day, if the subject did not participate for three testing days in a row the subject was excluded from the task.

Inhibitory control tasks

For the task battery we chose three tasks covering the main domains of inhibitory control: inhibition of distraction, inhibition of action and inhibition of a cognitive set (see Fig. 1). The tasks were conducted in the same order as they were built upon the previous task (see Fig. 1 for a visual presentation of the tasks and timeline): first a Distraction task (a target touching task with pictorial stimuli as distractors). This task was repeated (to assess temporal repeatability), for each subject, 2 weeks apart. Once this task was completed for the second time, the subjects were tested (the next testing day) on the Go/No-go task (a novel unrewarded stimulus was introduced). Once, the subjects were tested a second time on the Go/No-go task, they were tested on the reversal learning task (built upon the previously rewarded and unrewarded stimuli). Finally they finished the experiment by being tested a second time on the Reversal learning task 2 weeks after. As in previous batteries of tasks in animals (Beran & Hopkins, 2018; Herrmann et al., 2007, 2010; Lacreuse et al., 2014; MacLean et al., 2017; Wobber et al., 2014), the order of tasks was the same for all subjects. Although this design cannot eliminate the possibility of order effects (i.e., the participation on a given task affects performance on subsequent measures), it ensures consistency across subjects, permitting direct comparisons of performances across time and tasks. Besides, we wanted our subject to have the same experience with inhibitory control testing as this ability can be learned (Diamond, 2013) and is directly influenced by previous inhibitory control testing (Radel, Gruet & Barzykowski, 2019).

Design

Every session was initiated by the subject touching a red cross in the centre of the screen. Then the timer started, and the subject had to touch a target (a red rectangle of 10 × 13 cm) randomly displayed at the far left or right of the screen. When the subject successfully touched the target, a high-pitched chime was played, the timer was stopped, and the reward was given. After an inter-trial of 2,000 ms with only a white background displayed, the next trial was presented. Such a trial without a distractor was considered as a “Control” trial. Two “Control” trials were followed by a block of four trials with pictures from the same categories (either four pictures of objects, neutral or threatening conspecific faces, see Fig. 1 and online supplemental materials for MATLAB codes and stimuli). Each block of pictures of the same category was seen 2 times per session. From the literature, it appeared that the Stroop effect was more pronounced with this block presentation of pictures (in humans Bar-Haim et al., 2007; McKenna & Sharma, 2004 and in chimpanzees, P. troglodytes, Allritz, Call & Borkenau, 2016). The distractors were displayed at the centre of the screen at the same time as the regular target. The distractors were pictures of 16 × 18.5 cm with matching contrast and luminosity (function ‘Match colour’ in Adobe Photoshop CS6). The category “Object” included a leather ball, a leather bag, a brown stone and a wooden log. The conspecific pictures were chosen to be as realistic as possible, depicting a frontal view of the face and the torso of four unknown adult rhesus macaques. The “Neutral” conspecific included four pictures of individuals with a neutral facial expression. The “Threatening” conspecific included four pictures showing a “open mouth threat” facial expression, frequently displayed by rhesus macaques (Bethell et al., 2016; Hinde & Rowell, 1962). Threatening stimuli (as well as positive stimuli see Hopper et al., 2021) have been shown to have an important distracting effect in macaques (Bethell et al., 2016; Landman et al., 2014). The subjects were not rewarded to touch the distractors and the screen remained the same until the target on the side was touched. During a pilot study (N = 4 subjects, these data were not included in our analysis), we observed that the subjects were, for a long duration, intensely reacting to the pictures of their conspecifics’ faces (lip-smacking, stares and threats) so we set up a maximum response time of 35 s (at the condition that the subject kept looking at the screen). This time period allowed the subject to display a behavioural response, overcome it, and continue the task. If the subject did not touch the target within 35 s, the response latency was not taken into account in the analysis. Each block and trial were counterbalanced across subjects (see Fig. 1). Three sessions of 36 trials were repeated at time point 1 and time point 2 (average days between the time points M ± SD = 12.91 ± 2.84, N = 21; 216 trials total per individual). One male rhesus macaque was not willing to participate in further testing after this task.

Design

The apparatus and the general procedure were identical to the Distraction task. A “Go” (red rectangle of 16 × 18 cm) or a “No-go” stimulus (green circle of 16 × 16 cm) appeared randomly in the centre of the screen. The “Go” stimulus was preceded by a high-pitched sound (0.6 s before the stimulus appeared) and the “NoGo” stimulus a low pitch sound to help the subjects anticipate the next trial. The “Go” stimuli appeared 75% of the 40 trials to elicit a prepotent response toward the screen. If the screen was touched outside the stimulus no sound was produced and the trial continued. The “Go” stimulus stayed on the screen until it was touched. We set up a maximum response time (i.e., touching the “Go” stimuli) of 15 s after this the red cross appeared on the screen and the response latency was not recorded. From a pilot study conducted with a shorter response limit, we observed that the subjects frequently left the testing session as they were not rewarded on each “Go” trial. The “No-go” stimulus disappeared if not touched during 2,000 ms and the subject received a reward. If the “No-go” stimulus was touched during this lapse of time, a blank white background appeared for 3,000 ms (as a time out), an “incorrect” sound (with frequency 800, 1,300, 2,000 Hz) was produced and the reward was not given. At first, we fixed a success criterion for the subject’s performances at 80% of correct trials per session, but four macaques never reached this criterion. The performance was therefore measured after five sessions of 40 trials per time point (200 trials in total) for each monkey. These sessions were repeated at time point 2 (average days between the time points M ± SD = 11.95 ± 2.10, N = 20, see Fig. 1).

Design

At the beginning of the task, two stimuli, a “Go” rewarded stimulus (a red square of 15.34 × 15.34 cm) and a “No-go” unrewarded stimulus (a green circle of 15.34 cm of diameter), were displayed at the same time on the screen at counterbalanced locations (left or right of the screen). When the subject touched the “Go” stimulus, the usual “correct” sound was played, the subject received a reward, and a new trial began. If the subject touched the incorrect stimulus the “incorrect” sound was played, the subject did not receive a reward and the two stimuli stayed on the screen until the correct stimulus was touched. If the background was touched nothing happened. We set up a maximum response time (i.e., touching the “Go” stimuli) of 15 s after this the red cross appeared on the screen and the response variables were not recorded. We set up this response limit to keep the subjects engaged with the task. From a pilot study we observed that this period of time allowed the subject to frequently receive a reward and to keep engaged with the task. A session consisted of 40 trials. Once a criterion of success was achieved (75% of correct trials out of 20 trials, i.e., the subjects touched the correct stimulus from the first attempt), the rule was reversed: the correct stimulus became the incorrect and the incorrect the correct. One male macaque was excluded from the study as he did not reach the first criterion. The reversed session was continued until the success criterion was reached again (75% of success for the whole session). Three sessions of the reversed paradigm were repeated at time point 2 (average days between the time points M ± SD = 12.74 ± 5.06, N = 20). One male subject could not participate in the second time point test as it was permanently removed from the facility.

Analysis

To validate our battery of inhibitory control tasks our first aim was to assess the content validity of these tasks, for this we wanted to demonstrate that a prepotent response had indeed been triggered (Völter et al., 2018). In the Distraction task, we expected a prepotent response to be triggered by the pictorial distractors, which would increase the response latency of the subjects in a trial with pictures. We expected a greater response latency when a picture was presented, particularly in the trials with the negative stimuli, compared to control trials with no picture (as in Allritz, Call & Borkenau, 2016; Bethell et al., 2016, in non-human primates). We also expected to demonstrate that the subjects would still be able to perform the task by overriding their prepotent response to the distractors, keeping a general high rate of success in the task (i.e., successfully touching the target within the time limit).

To assess the content validity of the Go/No-go task, we wanted to demonstrate that a prepotent response was triggered (an incorrect impulsive action toward the No-go stimulus). Therefore, we investigated the difference in response accuracy in the Go and No-go trials. We expected it to be lower on a No-go trial compared to a Go trial. We also expected the subjects to try to override their impulsive response in the No-go trial, with a greater response latency compared to a Go trial.

In the Reversal learning task, we expected an interference from the previously learnt rule while learning the new rule. We expected a lower probability of success in learning the reversed rule (Rule 2) compared to the first acquisition rule (Rule 1). However, we still expected that the subjects override this interference from the first rule with an overall high accuracy for the second rule. We used a Wilcoxon one sample test to check that the previous task did not reinforce the subjects’ responses toward the red stimuli (even though the location and size were different).

All analyses were conducted in the R environment for statistical computing v.3.6.0 (R Core Team, 2019). We used linear mixed models and general linear model using the functions ‘lme’ from the R package ‘nlme’ v3.1-144 to analyse continuous variable (Harrison et al., 2018) and the function ‘glmer’ from the R package ‘lme4’ v1.1-21 to analyse the binary variable (Bates et al., 2015). The dependent variables were the response latency to touch the target on a trial in the Distraction task and in the Go/No-go task (continuous). As the response latency data presented a skewed distribution, they were log transformed to meet assumptions of normality (Cauchoix et al., 2018; Fazio, 1990; Harald Baayen & Milin, 2010). We excluded latencies below 200 ms (time needed for stimulus perception and motor responses to occur, Harald Baayen & Milin, 2010; Whelan, 2008) and above the time limit. We also recorded the successful completion of a trial in the Go/No-go task and in the Reversal learning task (binary). For all tasks, the random factor included the individual’s identity. We controlled for known influencing factors of inhibitory control by including the following explanatory variables into the models: the sex of the subject (Paul, Harding & Mendl, 2005; Sass et al., 2010), the age (in year, Tapp et al., 2003; Bray, MacLean & Hare, 2014), the experience with pictures (for the Distraction task) and the rank of the subject (Johnson-Ulrich & Holekamp, 2020). To increase the power of the analysis we merged middle ranked individuals and low ranked individuals to have two categories of ranks : either low or high. We had a total of 16 individuals from the higher rank and five from the lower rank. We also controlled for the following task influencing factors: trial, session, time point and type of stimulus (type of picture, no picture vs any type of picture for the Distraction task, type of stimulus: Go vs No-go in the Go/No-go task and type of rule: acquisition or reversed rule in the Reversal learning task). The full model contained all probable explanatory variables (demographic and task determinants). Terms were sequentially dropped from the full model, until the best fitted model contained only those terms that could not be removed without significantly reducing explanatory power (Bates et al., 2015; Harrison et al., 2018). We used the function ‘anova’ from the R package ‘car’ v3.0-6 to compare each model by likelihood ratio test (given as : χ² (Degree of freedom, N = sample size)). Our significant threshold was p < 0.05. Visual inspection of residual plots did not reveal any obvious deviations from normality. We presented the mean ± the standard error for each effect of the explanatory variable on the outcome. Data are available on an online repository (see Supplemental Materials, DATA SET).

Content validity, distraction task

There were no significant differences between each type of picture (‘Threat’, ‘Neutral’ or ‘Object’) in the overall performances but there was a significant difference in the mean response latency between the trials without pictures (control trial, M ± SE = 3,961.78 ± 104.52 ms, N = 21) and with any type of picture (test trials, M ± SE = 4,496.52 ± 111.17 ms, N = 21, likelihood-ratio test, χ² (1, N = 21) = 9.98, p = 0.002; see Fig. 2 and Table S1 for more details and for the presentation of the effects of the other explanatory variables). This indicates that the response latency was higher when a picture was present, showing that the subjects were distracted. Nonetheless, when a picture was present the percentage of success on the task (to touch the target within the time limit) was still high (96%) showing that this prepotent response was overridden by the subjects. Thus, the interference from a distractor was inhibited in order to successfully complete the task, these results indicate the content validity of this first task.

Content validity, Go/No-go task

There was a significant difference in the mean successful completion of the task between the trials with the Go signal or the No-go signal (likelihood-ratio test, χ² (1, N = 20) = 3335.6, p < 0.001, see Fig. 2 and Table S2). With only a mean of 47.3 ± 1.09% (N = 20) of correct answers when a No-go signal was presented (i.e., not touching the No-go stimulus), compared to a mean of success (i.e., touch the Go stimulus within the time limit) of 99.8 ± 0.04% (N = 20, see Fig. 2) when it was a Go stimulus presented. This result is showing that a prepotent response, touching the target impulsively was triggered as the subjects, most of the time, made a mistake and touched the unrewarded No-go target associated with a time out. However, there was still a sign that the subjects tried to override their prepotent response, the mean response latency on a No-go trial (M ± SE = 3,123.13 ± 36.68 ms, N = 20) was significantly longer than the one in a Go trial (M ± SE = 2,881.81 ± 91.69 ms, N = 20, see Fig. 2, from the log-transformation of the response latency, likelihood-ratio test, χ² (1, N = 20) = 600.73, p < 0.001, see Table S3). Please refer to Tables S2 and S3 for the effect of the other explanatory variables on the model. These results indicate that the subject had a prepotent response to touch any time of stimulus, but still tried to overcome this dominant response by slowing down their action toward the screen, this task is thus assessing the inhibition of a prepotent action.

Content validity, reversal learning task

First, we found that the previous task did not influence the subjects’ responses as they did not choose the red stimulus for the first trial above the chance level (chance level = 0.5, mean proportion = 0.53, p = 0.84). There was a significant difference in the mean success in a trial if the rule was the first acquisition or the reversed rule (likelihood-ratio test, χ² (1, N = 20) = 5.10, p = 0.02, see Table S4 for more details on the other explanatory variables), the probability of success was significantly a little higher in the first rule (65 ± 0.97% of success, N = 20), compared to the reversed rule (63 ± 0.78% of success, N = 20, see Fig. 2). These results indicate that the acquisition rule was interfering with the learning of the reversed rule, despite this interaction 100% of the remaining subject still managed to pass the 75% criterion of success for the second rule (see Table 1 for a summary of the main findings).

Analysis

We wanted to assess the temporal repeatability (also known as test-retest reliability) of the individual’s inhibitory performances. For this, we computed, from the response variables (response latency and successful completion of the tasks), scores of inhibitory control. For the Distraction task we computed a Distraction control score, which is the standard method in Stroop task paradigms (as in Allritz, Call & Borkenau, 2016; Bethell et al., 2016, 2019). Distraction control score represented, for each trial, the difference between the mean response latency in all trials without pictures minus the response latency in each trial. For a trial with a picture, a higher score would indicate better control of a Distraction (a shorter response latency). For the second task, the Go/No-go task, to quantify the individual’s ability to inhibit its prepotent action, we calculated the Action control score as the mean percentage of trials when a No-Go was present, and the individual didn’t touch it for each session (Verbruggen & Logan, 2008). We took the last three sessions of each time point of each animal to have a comparable number of sessions between tasks. A higher score would indicate an individual is better at inhibiting the action. Finally, we calculated a Rule control score (as in Tapp et al., 2003), as the difference between the number of trials to reach the criterion of success for the first rule (75% of correct trials for the whole session) minus the number of trials to reach the same criterion for the reversed rule. A higher score would indicate that an individual is better at inhibiting a previous rule when learning a new one.

Once we had these inhibitory control scores, we wanted to assess the temporal repeatability (also known as test-retest reliability) of the individual’s inhibitory control scores, i.e., if the rank of the performances of each subjects within the group were consistent over the two time points (as done in the meta-analysis of Cauchoix et al., 2018). We used the repeatability estimates (R) or Intraclass Correlation Coefficient (ICC) which indicates the amount of variation explained by inter-individual variation of performances in the tasks relative to intra-individual variation (developed by Nakagawa & Schielzeth, 2010). This estimate accounts for both consistency of performances from test to retest (within-subject change), as well as change in average performance of participants as a group over time (Vaz et al., 2013). This test thus assesses the repeatability of the rank of the subjects’ performances within the group between the test and the retest and between contexts. We used the function ‘rpt’, from the ‘rptR’ package v.0.9.22 in R (Nakagawa & Schielzeth, 2010; Stoffel, Nakagawa & Schielzeth, 2017; Vaz et al., 2013). We applied a restricted maximum likelihood function (with 1,000 bootstrapping and 1,000 permutations) and the individual identity was specified as a random intercept effect. The appropriate type of data distribution was adjusted in each model depending on the dependent variable under investigation (“Gaussian” for continuous variables, “Binomial” for binomial variables). We checked Gaussian models for normal distribution of the residuals using the function ‘qqnorm’ from the R package ‘car’ v3.0-6. An individual’s performance was considered as repeatable if the p-value from the Likelihood-ratio test was <0.05. The decision for the qualification of the R estimates, either low, moderate, or high, was based on the work of (Cauchoix et al., 2018). In this meta-analysis, regrouping 44 studies across 25 animal species, the authors computed the repeatability of individual cognitive performances. They found out a mean estimate for the temporal repeatability unadjusted of R = 0.18, 95% CI [0.09–0.28], the R adjusted for test order and individual determinants was R_adj = 0.15, 95% CI [0.09–0.21]. We considered that if R ≤ 0.1 the repeatability estimate is low, for 0.1 < R ≤ 0.3, it is moderate and for R > 0.3 it is high.

Once the (R) estimates were calculated, we also needed to consider the influence of confounding factors on the temporal repeatability estimates. For this we calculated adjusted estimates (estimates that adjust for confounding effects which remove fixed effect variance from the estimate, see Nakagawa & Schielzeth, 2010; Stoffel, Nakagawa & Schielzeth, 2017; as done in Cauchoix et al., 2018). Adjusted repeatability can be interpreted as the repeatability given that the level of the confounding factor is known (Nakagawa & Schielzeth, 2010). To calculate these adjusted estimates, we first needed to determine which factors had an effect on the individual’s performances by fitting linear mixed models (LMM) and general linear mixed models (GLMM) (we used the same packages as for the content validity). The outcome variables were either the Distraction control score, the Action control score or the Rule control score. As before, we control for two types of confounding factors: individual determinants (age, sex, rank of the individuals, experience with pictures for the Distraction task), and test determinants (type of stimulus, time point, session and trial). Individual’s identity was included as a random factor. The full model contained all probable explanatory variables (demographic and task determinants). We used the same model selection as before (see content validity) and the function ‘anova’ from the R package ‘car’ v3.0-6 to compare each model and our significant threshold was p < 0.05. Then, we calculated the adjusted repeatability, R_adj, by including confounding effects (identified by comparing GLMMs) into the repeatability function. Uncertainty in estimators was quantified by parametric bootstrapping and significance testing was implemented by likelihood ratio tests with a significance threshold of p < 0.05 (Stoffel, Nakagawa & Schielzeth, 2010). We reported 95% confidence intervals (CIs) for parameter estimates based on 1,000 bootstrapping and 1,000 permutations. We reported the result for the adjusted repeatability from the “Link scale approximation”.

Temporal repeatability, distraction task

For the Distraction task, the Distraction control score per individual was moderately repeatable between sessions and the two time points (R = 0.282 ± 0.095, 95% CI [0.093–0.462], p < 0.001). When testing for explanatory factors using LMMs, the model with the variables session, time-point and sex as fixed terms best explained the Distraction control score of the subjects (likelihood-ratio test comparing the best fitted model with the null model: χ² (5, N = 21) = 119.61, p < 0.001, see Table S5). Males had a lower Distraction control score (likelihood-ratio test, χ² (1, N = 21) = 9.38, p = 0.002), and this score increased as the number of the session increased (likelihood-ratio test, χ² (1, N = 21) = 14.02, p < 0.001) and at time point 2 (likelihood-ratio test, χ² (1, N = 21) = 93.14, p < 0.001). When considering these confounding factors, the adjusted repeatability of the Distraction control score was lower than the unadjusted repeatability (R_adj ± SE = 0.128 ± 0.048, 95% CI [0.041–0.241], p < 0.001) but still repeatable (see Fig. 3). Thus, in this Distraction task, the rank of the subjects’ performances within the group was repeatable over the two time points.

Temporal repeatability, Go/No-go task

For the Go/No-go task, the Action control score per session per individual was highly repeatable between sessions and the two time points (R = 0.338 ± 0.105, 95% CI [0.120–0.514], p < 0.001). When testing for confounding factors using GLMMs, none of the variables had a significant effect on the model of the Action control score of the subjects so the temporal repeatability estimates were not adjusted (likelihood-ratio test comparing the full model with the null model: χ² (7, N = 20) = 6.85, p = 0.44, see Table S6 and Fig. 3). Thus, in this Go/No-go task, the rank of the subjects’ performance within the group was repeatable over the two time points.

Temporal repeatability, reversal learning task

For the Reversal learning task, the Rule control score per session per individual was highly repeatable between the sessions and the two time points (R = 0.944 ± 0.033, 95% CI [0.855–0.981], p < 0.001). When testing for confounding factors using GLMMs, none of the explanatory variables had a significant effect on the models so the temporal repeatability estimates were not adjusted (likelihood-ratio test comparing the full model with the null model: χ² (7, N = 20) = 3.59, p = 0.61, see Table S7 for more details about the other confounding variables). Thus, in this Reversal learning task, the rank of the subjects’ performances, within the group, was repeatable over the two time points.

To summarize, the rank of the individual performances within the group for inhibitory control scores were consistent over time. While the adjusted temporal repeatability estimates were lower compared to the unadjusted ones, they were still repeatable (see Table 1 for a summary of the main findings).

Analysis

Once we had demonstrated the content validity and the temporal repeatability of our measurements, we were able to look at the cross-contextual consistency of the individual’s performances between the different tasks. We estimated contextual repeatability of our tasks by comparing individual performances on different tasks that putatively measure inhibitory control (as done in the meta-analysis of Cauchoix et al., 2018). We thus wanted to look at the consistency between subject’s performance ranks between each pair of tasks. We use the same repeatability test as before to assess the repeatability of the rank of the subjects’ performances within the group between contexts. As before, we used LMMs and GLMMs to look for confounding factors in order to adjust the contextual repeatability when two tasks were analysed. In the models, the inhibitory control score was the dependent variables and one of the four types of tasks, the sex, the age, the rank, the session, and the time point were the independent variables. Each score was centred and scaled using the function ‘scale’ from the package ‘base’, in R v3.6.3 to allow comparison between the scores of different units. As before, the decision for the qualification of the R estimates, was either low (R ≤ 0.1), moderate (0.1 < R ≤ 0.3), or high (R > 0.3).