Associations with monetary values do not influence access to awareness for faces

Participants

Twenty-four participants (13 females; age: 18–35 years, M = 24.54 ± 4.36 standard deviation(SD)) took part in the experiment. Five participants were left-handed, all other participants were right-handed. All participants had normal or corrected-to-normal vision and written informed consent was obtained from each participant prior to their participation in the experiment. The study was approved by the local ethics committee of the Charité –Universitätsmedizin Berlin (EA1/301/13) and performed in accordance with the Declaration of Helsinki. The final sample size was based on a sequential testing approach using Bayes Factors (Schönbrodt et al., 2017). More specifically, we planned to calculate Bayes Factors (BF10) for our main analysis after every new batch of 24 participants, since this sample size is needed to completely counterbalance the association between stimulus exemplars and monetary values across participants. Our aim was to continue data collection until these Bayes Factors would either exceed a threshold of 3, which indicates evidence in favour of the alternative hypothesis, or fall below a threshold of 1/3, indicating evidence in favour of the null hypothesis. Since this condition was already met after the inclusion of 24 participants, we stopped data collection at this stage.

Stimuli and apparatus

The face stimuli used in the study were grey-scale photographs of four different female faces with a neutral expression, taken from the NimStim Set of Facial Expressions (Tottenham et al., 2009; image IDs: 01, 07, 09, 17). All four images were similar in global contrast (root mean square contrast between 0.16 and 0.20) and luminance (mean luminance between 27.49 cd/m² and 31.35 cd/m²). All stimuli were presented on a uniformly grey background (30.28 cd/m²).

Participants viewed the screen through a mirror stereoscope providing separate visual input to the two eyes. Each participant’s head was stabilized by a chin rest at a viewing distance of 60 cm. All stimuli were presented using MATLAB (The MathWorks, Natick, MA, USA) and Psychtoolbox-3 (http://psychtoolbox.org/) on a 19 inch CRT monitor (resolution: 1024 × 768 Px, refresh rate: 60 Hz).

Procedure

The experiment consisted of three phases: (1) a pre-conditioning phase to measure baseline response times, (2) a conditioning phase during which different faces were paired with monetary outcomes, and (3) a post-conditioning phase that was identical to the pre-conditioning phase, intended to assess the change in response times after the conditioning had taken place.

In the initial pre-conditioning phase (Fig. 1A), different face exemplars were presented under continuous flash suppression. At the beginning of each trial, a black rectangle (10° × 10°) and a black fixation cross in its center (0.68° × 0.68°) were presented to each eye. The rectangle and the cross were visible throughout the whole experimental phase. After a fixation duration of 2 s, high-contrast dynamic mask stimuli consisting of circles and squares of various colors and sizes were flashed to a randomly selected eye at a frequency of 10 Hz. Simultaneously, a face image (3.75° × 3.75°) that was located within one of the four quadrants of the black rectangle was presented to the other eye. The contrast of the face stimulus linearly increased from 0% to 100% during the initial 2 s. After that, the face remained at full contrast until the end of the trial. A trial ended when the participant gave a manual response or, if no response was made, after 15 s. Participants’ task was to indicate the location of the face, that is, the quadrant in which the face appeared, as fast and as accurately as possible by pressing one of four designated keys on the keyboard. This part of the experiment comprised 96 trials. The combination of the face exemplar, the location of the face, and the eye to which the face was presented was counterbalanced and randomized across trials.

In the second phase, participants had to learn the association between the face exemplars and monetary outcomes by means of classical conditioning (Fig. 1B). In each trial, one of the faces that were already presented in the first part of the experiment was shown in the center of the screen. Below the face, a positive or negative monetary value was displayed. The face and the value were presented for 5 s. After the offset of the face and the associated monetary value, a blank screen was presented for a randomized interval between 1 s and 1.6 s before the next trial started. Each face was associated with one of four different monetary values: −2 €, −0.1 €, +0.1 €, and +2 €. In 75% of the trials, the face was depicted with its associated monetary value. In the remaining trials, an outcome of 0 € was presented along with the face. Such a probabilistic reinforcement schedule was chosen to maintain participants’ attention to the stimuli and the task, as for classical conditioning with monetary outcomes attention is preferably directed towards partially predictive stimuli in comparison to fully predictive stimuli (Austin & Duka, 2010). Participants were instructed to passively view the stimuli and to memorize the association between the faces and the monetary values as well as possible. They were further informed that the monetary value depicted in a trial would be counted towards their overall payoff. Thus, for a trial, in which a face was presented together with ‘+2 €’, for instance, 2 € would be added to their payoff, while for trials with ‘-2 €’ 2 € would be subtracted from their payoff. Since unbeknown to participants each monetary value was presented equally often, the outcome for this part of the conditioning phase amounted to zero. Thus, participants’ outcome was solely defined by their accuracy in the query trials (see below). This phase of the experiment comprised four blocks, during which each face was presented four times. The order of the face exemplars was randomized and the association between the face exemplars and the monetary values was fully counterbalanced across participants. The association was further kept constant across all four blocks.

To assess whether participants were indeed able to learn these associations, query trials were added at the end of each block. Each face was presented once during these query trials. In contrast to the conditioning trials, a question mark and all four different monetary values were displayed below the face in random order. Participants were required to select the value that was associated with the respective face by pressing one of four designated buttons. The face and the monetary values were presented until a response was made. Participants were informed that monetary reimbursement for their participation in the experiment would depend on the accuracy of their choices during these query trials. After the experiment, participants indeed received the amount of money that they accumulated during the conditioning phase. More specifically, a correct assignment of a monetary reward to the respective face would yield an addition of +2 €or +0.1 €, respectively, to their payoff. A wrong response to faces associated with reward did not result in a monetary gain. For faces associated with a monetary punishment, participants had to assign the correct monetary value to avoid a monetary loss. This means that a correct assignment of −2 € or −0.1 € to the respective face did not yield a monetary gain or loss. However, if a participant assigned a wrong value to a punishment-related face, this resulted in a loss of −2 € or −0.1 €, respectively. Thus, participants expected that their payoff was dependent on both, the passively viewed pairings of monetary values and faces as well as their performance during the query trials. This procedure served two purposes: (1) to increase the relevance of the high-valued compared to the low-valued faces, and (2) to increase participants’ attention to all face stimuli during the query trials so that the accuracies in the query trials would provide an optimal account of participants’ learning progress. Note that in case of negative values participants had to respond accurately to avoid monetary losses during the query trials.

The post-conditioning phase, which followed directly after the conditioning phase, was fully identical to the pre-conditioning phase. After the experiment, participants rated how much they felt motivated by the different monetary values to memorize the faces on a visual analog scale ranging from 0 to 5.

Data analysis

Participants’ learning performance during the conditioning phase was assessed on the basis of their responses in the query trials. For each of the four blocks, each participant’s accuracy in assigning the monetary value to each face exemplar was computed. The chance level for each block was .25. To evaluate participants’ sensitivity for reward and punishment, we computed each participant’s response bias during the query trials. For reward-related faces the response bias is computed on the basis of the z-transformed hits (H_R) and false alarm rates (FA_R) as follows: ${\displaystyle c_R=-\frac{z\left(H_R\right)+z\left(F{A}_R\right)}{2}}$

However, since participants have to assign a positive or negative value to each face, the response biases for reward and punishment are not independent of each other. More specifically, an increase in the false alarm rate for reward leads to a decrease in the hit rate for punishment (H_P) by the same amount, such that: z(FA_R) = −z(H_P). Thus, the formula above can be rewritten as: ${\displaystyle c_R=-\frac{z\left(H_R\right)-z\left(H_P\right)}{2}.}$

Since a bias towards reward (c_R) automatically implies a bias away from punishment (c_P) such that c_R = −c_P, we only report the reward response bias. For c_R, a positive bias denotes a stronger sensitivity for reward, while a negative bias denotes a stronger sensitivity for punishment.

For the bCFS phase before and after the conditioning phase, trials in which participants responded incorrectly (percentage of trials: M = 3.19%, SD = 2.75%) or failed to give a response until the end of a trial (M = 0.61%, SD = 1.26%) were discarded from further analysis. Furthermore, response times below 200 ms were considered anticipatory responses and were also discarded from further analysis (percentage of trials: M = 0.61%, SD = 1.26%). To compare the response times between the different conditions, we followed two approaches. For the first approach, we computed the median response time for each participant and condition before and after conditioning. We used the median instead of the mean to account for the skewness of response time distributions, which is line with other bCFS studies (e.g., Gayet et al., 2016; Gayet & Stein, 2017; Han, Blake & Alais, 2018). We then subtracted the median response times of the pre-conditioning phase from the median response times of the post-conditioning phase, which resulted in a measure for the change of response times for each condition and participant. Finally, we performed two paired t-tests to compare the mean change in response times between high and low reward as well as between high and low punishment. The alpha level of these two t-tests was adjusted to .025 to account for multiple comparisons. For the second approach, we analyzed the response time distributions in more depth by computing hierarchical shift functions (Rousselet & Wilcox, 2019), which can be more sensitive to response time differences, especially when they are restricted to early or late responses. To this end, we computed the deciles of the response time distribution for each condition and participant before and after conditioning. In a next step, we subtracted the deciles of the pre-conditioning phase from the deciles of the post-conditioning phase. The resulting values thus indicate the change in response times for each segment of the whole distribution, where lower deciles reflect faster responses and higher deciles slower responses. For each decile, we then performed a paired t-test to compare the changes in response times of the high reward to the low reward condition and of the high punishment to the low punishment condition. As this amounts to 18 different t-tests in total, we adjusted the alpha level of the t-tests to .003. Thus, we adjusted the significance level for the number of tests across reward and punishment, because our aim was to control the maximum experiment-wise error rate (Bender & Lange, 2001). Our reasoning for applying such a rigorous adjustment of the alpha level was that the underlying hypothesis of an influence of learned values on bCFS response times would be supported by a difference in response times for either reward or punishment.

To probe potential time-dependent effects of learned values, we additionally split the post-conditioning phase into two halves. For statistical inference, we performed repeated-measures ANOVAs with the factors time (first half vs. second half of the post-conditioning phase) and value (high value vs. low value). Separate ANOVAs were performed for reward-related and punishment-related stimuli. The focus of this analysis was on the interaction between time and value, since a statistically significant interaction would signify an influence of the monetary value that was dependent on time. Note that this analysis was performed on the median response time differences between the pre- and post-conditioning phase.

Finally, we performed two generalized linear mixed-effects models on the single trial data, one for reward-related and one for punishment-related stimuli. This analysis was intended to take into account the variability of the effect of interest in dependence on the repetition of the stimuli and the association between the different face exemplars and the monetary values. We defined a Gamma distribution as the distribution of our response variable, since it provides a plausible approximation to the processes reflected in response times (Lo & Andrews, 2015). We included the interaction of face value, bCFS phase, face exemplar, and trial number as well as all other interactions and main effects of these variables as fixed effects in our models, for which we used effects coding. For the random effects term, we defined the theoretically maximal model by including subject as a random factor together with the by-subject random intercept and the by-subject random slopes of all fixed factors (Barr et al., 2013). Both models thus had the following structure: ${\displaystyle RT\sim value\ast phase\ast exemplar\ast trial+\left(1+value\ast phase\ast exemplar\ast trial|subject\right).}$

The estimate of each fixed-effects predictor was then tested against zero by means of individual t-tests.

In addition to frequentist inference statistics, we computed Bayes Factors (BF10) for each t-test, using a default Cauchy prior of 0.707. For ANOVAs we computed BF10 directly from the F-values of the ANOVA statistics (Faulkenberry, 2018). In line with previous suggestions (Wetzels & Wagenmakers, 2012), we interpret BF10 >3 as evidence for the alternative hypothesis and BF10 <1/3 as evidence for the null hypothesis. We also report BF01 to clarify which analyses yielded evidence for the null hypothesis. In this case, BF01 >3 indicates evidence for the null hypothesis.

Classical conditioning

At the end of every block in the conditioning phase, participants had to indicate the associated monetary value for each presented face. Figure 2A depicts the accuracy of these responses for the four different blocks. Since participants had to make four choices in each block, the chance level corresponds to an accuracy of .25 for each block. Binomial tests indicated that the response accuracies across all four blocks exceeded chance level for all participants (p ≤ .002, BF10 ≥ 10.10). Figure 2B shows the response accuracies across all four blocks separately for each value condition. As for each value condition the majority of participants did not commit any mistake during the query trials, the median accuracy for each condition amounted to 1. Thus, participants quickly and successfully learned the associations between the face exemplars and the monetary outcomes for each of the different monetary values. After the experiment, participants rated their motivation for each monetary value on a visual analog scale ranging from 0 to 5. In comparison to a low reward of 0.1 € (M = 1.51 ± 0.28 standard error of the mean [SEM]), participants felt substantially more motivated by the high reward of 2 € (M = 2.98 ± 0.36 SEM; paired t-test: t(23) = 4.99, p <  .001, BF10 = 521.53). The difference in motivation between a low punishment of 0.1 € (M = 1.46 ± 0.30 SEM) and a high punishment of 2 € (M = 2.08 ± 0.37 SEM) was less pronounced compared to reward (t(23) = 2.42, p = .024, BF10 = 2.35), but still statistically significant at a corrected threshold of α = .025. The majority of participants (n = 18) did not exhibit a response bias to either reward or punishment (i.e., c_R = 0). One participant showed a response bias towards monetary punishment (c_R = −0.18) and five participants showed a response bias towards monetary reward (c_R = 0.24–0.97).

Response times during breaking-CFS

Figure 3A shows the change in response times from the pre-conditioning to the post-conditioning phase for the two different reward conditions. For the high reward condition, response times decreased by 501.01 ms (±102.27 ms SEM), on average, while for the low reward condition we observed an average decrease in response times of 412.74 ms (±84.25 ms SEM). The difference between the two conditions was not statistically significant (t(23) = 0.80, p = .43) and the Bayes factor indicated the absence of an effect (BF10 = 0.29, BF01 = 3.48).

The average change in response times for the high and low punishment condition are depicted in Fig. 3B. Numerically, there was a stronger decrease in response times for the low punishment condition (M = −477.85 ± 97.54 ms SEM) in comparison to the high punishment condition (M = −401.97 ± 82.05 ms SEM). However, the difference between the two conditions was again not statistically significant and indicative of an absence of an effect (t(23) = 0.65, p = .52, BF10 = 0.26, BF01 = 3.84).

To rule out baseline differences between conditions that might have masked potential effects of learned values in the post-conditioning phase, we additionally computed response times during the pre-conditioning phase only. Response times for faces that were later paired with a high reward (M = 2275.90 ± 464.57 ms SEM) did not significantly differ from response times for faces that were later paired with a low reward (M = 2254.57 ± 460.21 ms SEM; t(23) = 0.17, p = .87, BF10 = 0.22, BF01 = 4.60). Similarly, the time to respond to faces later associated with high punishment (M = 2224.52 ± 454.08 ms SEM) was not significantly different from faces later associated with low punishment (M = 2315.77 ± 472.71 ms SEM; t(23) = 0.85, p = .40, BF10 = 0.30, BF01 = 3.36).

Differences between high and low values, however, might be limited to a specific range of the whole response time distributions, that is, the learned values could specifically affect fast or slow responses, which would not necessarily be captured by measures of central tendencies. We therefore explored the response time distributions in more depth by computing the change between the pre- and the post-conditioning phase for each decile of the whole response time distribution for each condition. For reward-related stimuli (Fig. 4A) as well as well as for punishment-related stimuli (Fig. 4B), the decrease in response times was more pronounced for slower responses. However, there was no significant difference between high and low values for any of the deciles (p ≥ .12 and BF10 ≤ 0.65 for reward, p ≥ .12 and BF10 ≤ 1.66 for punishment). Of note, the differences between high and low reward and punishment, respectively, were close to an uncorrected significance level of .05 for the last deciles. The response time decreases in this decile, however, were numerically stronger for low values compared to high values and as such contrary to our a priori hypothesis.

Due to extinction, potential influences of the monetary associations on response times could have quickly decayed during the post-conditioning phase. Thus, averaging across all responses in the post-conditioning phase might have masked the effects of learned values. To identify potential time-dependent effects, we split the post-conditioning phase into two halves. We performed two repeated-measures ANOVAs with the factors time (first half vs. second half of the post-conditioning phase) and value (low value vs. high value). If the influence of learned values was indeed dependent on time such that it quickly faded after the conditioning block, this should be reflected in the interaction between time and value. However, neither for faces previously associated with reward (F(1,23) = 0.60, p = .44, BF10 = 0.28, BF01 = 3.59) nor for faces associated with punishment (F(1,23) = 0.76, p = .39, BF10 = 0.30, BF01 = 3.31) we observed an interaction between time and value.

In line with our main analysis, the parameter estimates for the interaction of the face value and the bCFS phase in the generalized linear mixed-effects models were not statistically significant from zero, neither for reward (β = −8.49*10⁻⁶, t(2194) = -1.87*10⁻⁵, p >.99) nor for punishment (β = −4.26*10⁻⁵, t(2193) = −8.62*10⁻⁵, p > .99). There were also no other statistically significant interactions with trial number or face exemplar (all p > .99).

Exploratory analyses

While our a priori hypotheses focused on the comparison of the change in response times for high vs. low monetary values, we performed two additional exploratory analyses to examine whether the learned values might have influenced the response times during the bCFS phase in other ways.

For the first analysis, we collapsed our data across high and low values and compared response time changes for all reward-related faces to all punishment-related faces. For reward-related faces, response times decreased by 428.13 ms (±87.39 SEM), while for punishment-related faces response times decreased by 443.98 ms (±90.63 SEM), on average. The difference between reward and punishment was not statistically significant (t(23) = 0.19, p = .85, BF10 = 0.22, BF01 = 4.59). Even when only faces associated with a high value (i.e., high reward vs. high punishment) were considered, the difference between reward and punishment did not reach statistical significance (t(23) = 1.15, p = .26, BF10 = 0.39, BF01 = 2.58).

In a second exploratory analysis we examined whether the influence of learned values on response times might have been dependent on the learning performance during the conditioning phase. To this end, we computed each participant’s accuracy in the query trials across all four conditioning blocks. We then correlated these accuracies against the response time differences (i.e., post- vs. pre-conditioning) during the bCFS blocks, separately for reward- and punishment-related faces. The bCFS response time differences were computed by subtracting the change in response times for the low value condition from the change in response times for the high value condition as follows: ${\displaystyle \left({\left[{\mathrm{RT}}_{\mathrm{post}}-{\mathrm{RT}}_{\mathrm{pre}}\right]}_{\mathrm{high}\mathrm{reward}}-{\left[{\mathrm{RT}}_{\mathrm{post}}-{\mathrm{RT}}_{\mathrm{pre}}\right]}_{\mathrm{low}\mathrm{reward}}\right)\mathrm{and}\left({\left[{\mathrm{RT}}_{\mathrm{post}}-{\mathrm{RT}}_{\mathrm{pre}}\right]}_{\mathrm{high}\mathrm{punishment}}-{\left[{\mathrm{RT}}_{\mathrm{post}}-{\mathrm{RT}}_{\mathrm{pre}}\right]}_{\mathrm{low}\mathrm{punishment}}\right).}$ As such, negative values indicate a change in response times in line with our a priori hypotheses, that is, a stronger decrease in response times for the high compared to the low value condition. Since only few participants exhibited low accuracies during the query trials, the distribution of participants’ accuracies was heavily skewed. To account for this, we computed Spearman’s ρ. Neither for reward (ρ = .30, p = .16, BF10 = 0.43, BF01 = 2.35) nor for punishment (ρ = .13, p = .56, BF10 = 0.19, BF01 = 5.39), accuracies were related to response time differences between conditions. Note that if higher accuracies in the query trials were related to a greater influence of high monetary values on response times, this would be indicated by a negative correlation. Due to the high accuracy during the query trials in the majority of participants, this analysis should be interpreted with caution.