Evaluating portable EEG: a comparison between two wireless systems (EPOC Flex and LiveAmp) and the wired BrainAmp system

Participants

Twenty-one French volunteers, free from neurological or psychiatric conditions and in good health, participated in the study. Data from two participants were excluded due to technical issues and experimenter error, resulting in a final sample of 19 individuals (15 females, four males), aged 19 to 56 years (M = 27.6; SD = 9.3). All participants provided written informed consent, and the study was approved by the University of Lyon ethics committee (CER-UdL n° 2023-06-15-003).

General design of the study

The study employed a within-subjects design, divided into three sessions, each featuring a different EEG system. Each participant carried out the three sessions (one or two per day). Each session included five tasks administered in the same order for all EEG systems to minimize variability between sessions: (1) SSVEP, (2) active auditory oddball (standard vs. deviant stimuli), (3) passive auditory oddball, (4) face perception, and (5) resting state. Alternating between active and passive tasks aimed at preventing boredom and maintaining participant attention during the approximately 35-min session. In contrast, the order of the EEG systems was counterbalanced to ensure that the three systems were used similarly in the first, second, and third sessions. The total duration of EEG acquisition for one participant, excluding installation time, lasted around 105 min.

Stimuli

The experiment was conducted in a dedicated sound-attenuated room. Participants were comfortably seated in a chair, with the ventilation system and lights turned off, the curtains drawn, and their phones set to airplane mode throughout the EEG recordings. Stimuli and instructions were delivered using Presentation software (version 23.1) on a laptop with a 15.6″ Full HD (1,920 × 1,080) screen (60 Hz refresh rate). Participants were seated 60 cm from the screen, and auditory stimuli were played through Sennheiser IE 100 Pro Clear in-ear headphones at a comfortable listening level. We used the ERP CORE (Compendium of Open Resources and Experiments) resource material as a starting point to create our variant of the standard ERP paradigms (Kappenman et al., 2021).

EEG systems

Table 1 presents the characteristics of each system, including channels, electrodes, headset, amplifier resolution, sampling rate, and connectivity.

Table 1:

Features and comparison of the three EEG systems.

	EPOC Flex	LiveAmp	BrainAmp
Company	EMOTIV	Brain products	Brain products
Features system	Wireless consumer-grade	Wireless research-grade	Wired research-grade
Channels	32 passive electrodes	32 passive electrodes	32 active electrodes
Ag/AgCl electrodes	Sponge with KCl electrolyte solution	Sponge with NaCl electrolyte solution	Gel-based active electrode
Headset	EasyCap (flexible sensor placement)	R-NET (International 10-10 system)	actiCap slim (International 10-20 system)
Amplifier resolution	14-bit	24-bit	16-bit
Sampling rate	128 Hz	500 Hz	250 Hz
Wireless connectivity	Yes (Bluetooth)	Yes (Wifi)	No
Signal quality check	Yes	Yes	Yes
Impedance check	Yes	Yes	Yes
Setup time (min)	20	20	40
References	Ref: CMS, Gnd: DRL	Ref: FCz, Gnd: Fpz	Ref: FCz, Gnd: AFz

DOI: 10.7717/peerj.20416/table-1

Procedure

Electrode positions

EEG recordings for each system were acquired on a Windows 11 Pro Dell Precision 3571 laptop, separate from the stimulus presentation computer, using the respective native software (BrainVision Recorder for Brain Products, GmbH and EmotivPRO for EM). Continuous EEG was recorded from 32 scalp sites for the EM and LA systems, with F9, P9, F10, and P10 replaced by FT9, TP9, FT10, and TP10 for the BA system (Fig. 1). In the EM layout, reference electrodes were CMS (Fpz) and DRL (FCz), while LA and BA used Fpz as the ground and FCz as the reference. Impedance was maintained below 50 kΩ for LA and 20 kΩ for BA. Following Williams et al. (2020), Flex electrodes were adjusted until impedance values turned ‘green’, indicating levels below 20 kΩ.

Resting state

The difference alpha power between EO and EC conditions was computed for each participant and compared to zero using a two-tailed t-test. An ANOVA was conducted on this difference with system (BA, EM, LA) as within-subject factor.

SSVEP

The SNR at the fundamental components and their first two harmonics (12, 18 Hz for the 6 Hz condition; 20, 30 Hz for the 10 Hz condition; 30, 45 Hz for the 15 Hz condition) were averaged. The presence of harmonic responses was assessed using t-tests against zero for the pooled SNR values. A two-way ANOVA was performed on the SNR using system (EM, LA, BA) and condition (6, 10, 15 Hz) as within-subject factors.

N170

The P100-N170 difference in peak amplitude was compared to zero by a t-test at P7 and P8. ANOVAs using the EEG system (EM, LA, BA) as within-subject factor were conducted on N170 peak latency and P100-N170 peak amplitude difference, at P7 and P8.

N200 and P300

The presence of significant N200 and P300 waves was assessed using t-tests against zero for peak amplitude at Fz, Cz and Pz. ANOVAs were performed on N200 and P300 latencies and amplitudes at Fz, Cz and Pz with EEG system as within-subject factor (three levels).

MMN

Significant MMN was assessed with one-tailed t-tests comparing the AUC and mean peak amplitude to zero at Fz, Cz and Pz. ANOVAs were performed on latency, amplitude, and AUC of the MMN at these electrode sites using EEG system (three levels) as within-subject factor.

Supplementary analyses

We conducted a series of additional analyses to further assess the reliability and precision of the ERP measures obtained across the three EEG systems. First, Pearson correlations were calculated to assess subject-by-subject measurement consistency across the three systems. Second, we computed the standardized measurement error (SME) for each ERP component, following the procedure described by Luck et al. (2021). The SME provides an estimate of the trial-level variability in amplitude and latency measures, and thus complements the correlation analysis by quantifying measurement precision at the individual level. The SME for each system was compared with the benchmark from Zhang & Luck (2023) (when available at the same electrode sites) using Mann-Whitney tests. The systems were also pitted against each other using Friedman tests.

Signal quality

As can be seen in Table 3, the EM system yielded less exploitable data than LA and BA systems. The EM headset had to be changed in the middle of a session for two participants due to a lost connection between the amplifier and the acquisition computer. More triggers were lost and more electrodes were declared faulty within the region of interest with EM than the other systems, leading to exclude one participant from the analysis of N200, P300 and MMN for every system. Another participant has been excluded for N170 for every system, because of a high percentage of rejected epochs with EM system.

Resting state results

When compared to zero, the difference in alpha power between the EO and EC conditions (Berger Effect) was significant and large-sized for each EEG system whether calculated in Pack-100, 75, or 50 (Cohen’s d_s > 1.03, Table 4, left).

The significant main effect of system was large in size for Pack-100 (F(2, 36) = 7.04, p = 0.007, η_p² = 0.28), Pack-75 (F(2, 36) = 6.62, p = 0.008, η_p² = 0.27), and Pack-50 (F(2, 36) = 6.36, p = 0.011, η_p² = 0.26), reflecting a lower Berger Effect with EM than LA and BA (Table 4, right).

SSVEP results

The SNR was significantly higher than zero in the 6, 10, and 15 Hz conditions for every system (Fig. 2, Table 5), and the effect sizes were large (d_s > 1.25) for every system and Pack.

In Pack-100, the significant main effect of system (F(2, 36) = 23.93, p < 0.001, η_p² = 0.57) was due to lower SNR with EM than LA (t(18) = 8.17, p < 0.001, d = 0.97) and BA (t(18) = 5.45, p < 0.001, d = 1.11). There was no main effect of condition neither Condition × System interaction. However, the SNR was very low with EM at 10 Hz condition for the fundamental component and its harmonics (Fig. 2).

In Pack-75, there was only a significant main effect of system (F(2, 36) = 18.15, p < 0.001, η_p² = 0.50), reflecting lower SNR with EM than LA (t(18) = 6.71, p < 0.001, d = 0.93) and BA (t(18) = 4.94, p < 0.001, d = 1.05). Similarly, in Pack-50, the main effect of system (F(2, 36) = 16.44, p < 0.001, η_p² = 0.48) reflected lower SNR with EM than LA (t(18) = 6.53, p < 0.001, d = 0.98) and BA (t(18) = 4.77, p < 0.001, d = 1.03).

N170 results

To assess the absence of any bias due to the EEG system on overall response speed, mean response times and SD were submitted to Wilcoxon tests, corrected for multiple comparisons. No significant difference was observed neither for response speed (EM-LA, V = 82, p = 0.615; EM-BA, V = 111, p = 0.841; LA-BA, V = 107, p = 0.956) nor speed variability (EM-LA, V = 108, p = 0.927; EM-BA, V = 139, p = 0.216; LA-BA, V = 145, p = 0.143).

In face condition and Pack-100 with native sampling rate (Pack-100a for EM, Pack-100b for LA and BA), the P2P amplitude difference (P100-N170, hereafter, N170) significantly differed from zero at P7 and P8, with large effect sizes for every system (1.22 < all d_s < 2.06) (Table 6). N170 remained large-sized in Packs-75 and 50 for the three systems (1.12 < all d_s < 2.00), and with lower sampling rate (Pack-100a) for LA and BA (1.22 < all d_s < 1.93).

For Pack-100 and native sampling rate, the significant main effect of system on P2P amplitude at P7 (F(2, 34) = 14.81, p < 0.001, η_p² = 0.47) and P8 (F(2,34) = 14.57, p < 0.001, η_p² = 0.46) reflected higher N170 for BA than EM and LA, but no EM-LA difference (Fig. 3, Table 6). This pattern of results was confirmed by the main effect of system with a lower sampling rate (Pack-100a: P7, F(2, 34) = 12.36, p < 0.001, η_p² = 0.42; P8, F(2, 34) = 13.29, p < 0.001, η_p² = 0.44), and Pack-75 (P7, F(2, 34) = 13.30, p < 0.001, η_p² = 0.44; P8, F(2, 34) = 13.73, p < 0.001, η_p² = 0.45). For Pack-50, the significant main effect of system at P7 (F(2, 34) = 13.82, p < 0.001, η_p² = 0.45) reflected higher N170 for BA than EM and LA, but at P8 (F(2, 34) = 13.89, p < 0.001, η_p² = 0.45) it reflected higher N170 with BA than LA and higher N170 with LA than EM (Table 6).

Regarding N170 latency for face condition, the significant main effect of system in Pack-100 at native sampling rate (P7, F(2, 34) = 76.08, p < 0.001, η_p² = 0.82; P8, F(2, 34) = 71.62, p < 0.001, η_p² = 0.81) reflected shorter latencies for EM than LA and BA (Table 6, Fig. 3). This pattern remained in Pack-100a, according to the main system effect (P7, F(2, 34) = 77.45, p < 0.001, η_p² = 0.82; P8, F(2, 34) = 66.64, p < 0.001, η_p² = 0.80), as well as in Pack-75 (P7, F(2, 34) = 60.14, p < 0.001, η_p² = 0.78; P8, F(2, 34) = 52.32, p < 0.001, η_p² = 0.75), and Pack-50 (P7, F(2, 34) = 41.64, p < 0.001, η_p² = 0.71; P8, F(2, 34) = 39.34, p < 0.001, η_p² = 0.70).

The typical parietal N170 saliency for faces was observed with LA and B—less with EM—, at least when the face-texture difference was represented (Fig. 4).

In texture condition, N170 for Pack-100a significantly exceeded zero at P7 and P8 (Table 7), and the effect size was large for LA and BA, and medium for EM. The examination of Fig. 3 suggested the classical decrease in amplitude and increase in latency of the N170 at posterior electrodes in texture condition as compared with face condition (Rossion, 2014), especially for LA and BA.

The main effect of system for Pack-100a was significant at P8 (F(2, 34) = 8.38, p < 0.001, η_p² = 0.33), reflecting lower N170 amplitude for EM than BA (t(17) = 3.49, p = 0.008, d = 0.79) and LA (t(17) = 4.51, p < 0.001, d = 0.65). No difference reached significance at P7.

N170 latency was modulated by the main effect of system (P7, F(2, 34) = 26.12, p < 0.001, η_p² = 0.61; P8, F(2, 34) = 30.12, p < 0.001, η_p² = 0.64). It was shorter for EM than LA (P7, t(17) = 6.29, p < 0.001, d = 1.85; P8, t(17) = 6.06, p < 0.001, d = 1.83) and BA (P7, t(17) = 6.15, p < 0.001, d = 1.79; P8, t(17) = 5.36, p < 0.001, d = 1.69) without LA-BA difference.

N200 & P300 results

Like the face perception task, the active auditory oddball task did not give rise to differences between the systems regarding response speed (EM-LA, V = 89, p = 0.571; EM-BA, V = 116, p = 0.701; LA-BA, V = 100, p = 0.869) nor speed variability (EM-LA, V = 83, p = 0.430; EM-BA, V = 105, p = 0.999; LA-BA, V = 101, p = 0.898).

The N200 amplitude in Pack-100 and native sampling rate significantly differed from zero for all three systems (Table 8). It was large in size for BA (medium in size at Pz) and LA, and medium in size for EM, with prominence at Fz for every system (Fig. 5). Despite reduced datasets (Packs-75, 50) for every system or lowered sampling rate (LA and BA), the N200 effect size remained large and prominent at Fz. N200 amplitude was not significantly affected by a main effect of system in Packs-100a, 100b, 75 and 50, at Fz, Cz or Pz (all p_s > 0.415).

Regarding N200 latency, there was no significant main effect of system at the native sampling rate (Pack-100b) (Fz, F(2, 34) = 3.50, p = 0.064, η_p² = 0.17; Cz, F(2, 34) = 0.51, p = 0.606; Pz, F(2, 34) = 0.90, p = 0.416). The marginal effect at Fz site reflected shorter latency for EM than BA (t(17) = 4.67, p < 0.001, d = 1.12). A main effect of system appeared with a lower sampling rate (Pack-100a: Fz, F(2, 34) = 7.66, p = 0.006, η_p² = 0.31; Cz, F(2, 34) = 5.73, p = 0.016, η_p² = 0.25; Pz, F(2, 34) = 3.51, p = 0.041, η_p² = 0.17), reflecting significantly shorter latency with EM than BA (Fz, t(17) = 6.51, p < 0.001, d = 1.44; Cz, t(17) = 5.25, p < 0.001, d = 1.04; Pz, t(17) = 3.07, p = 0.021, d = 0.77). For Pack-75, the system effect was significant at Fz site (F(2, 34) = 4.77, p = 0.031, η_p² = 0.22) reflecting the EM < BA pattern (t(17) = 5.69, p < 0.001, d = 1.40). For Pack-50, there was no significant main effect of system, but the EM < BA pattern remained at Fz (t(17) = 2.84, p = 0.034, d = 0.60).

P300 amplitude significantly exceeded zero at Fz and Cz, with a large effect size for every pack and system (Table 9). It was prominent at Cz (except with LA in Packs-75 and 50) and overall smaller in size, but still significant, at Pz.

These was no main effect of system on P300 amplitude in all Packs and electrodes.

In Pack-100b, the main effect of system was significant on P300 latency (Fz, F(2, 34) = 3.90, p = 0.030, η_p² = 0.19; Cz, F(2, 34) = 4.57, p = 0.018, η_p² = 0.21), reflecting shorter latency for EM than BA (Fz, t(17) = 2.75, p = 0.041, d = 0.63; Cz, t(17) = 3.18, p = 0.016, d = 0.81) and LA at Fz (t(17) = 2.71, p = 0.045, d = 0.66). In Pack-100a, the main effect of system at Fz (F(2, 34) = 5.57, p = 0.008, η_p² = 0.25) was explained by a shorter latency for EM than LA (t(17) = 2.82, p = 0.035, d = 0.76). The system effect at Fz was replicated in Pack-75 (F(2, 34) = 7.87, p = 0.002, η_p² = 0.32), with an EM < BA pattern (t(17) = 5.29, p < 0.001, d = 1.03), but there was no system effect in Pack-50.

MMN results

The AUC of the MMN in Pack-100 at native sampling rate significantly exceeded zero for every system at Fz (Table 10); the effect size was large (LA, BA), or medium (EM). It also exceeded zero significantly at Cz, medium-sized for LA and BA and small for EM (Fig. 6), while it was not significant at Pz. It remained the highest at Fz, even in Pack-100a, for all three systems.

For BA, over Packs-100b, 100a, 75 and 50, the significant MMN amplitude was medium in size at Cz, and large at Fz. For LA, the significant MMN was medium in size over every pack at Cz, whereas it decreased sharply from large over Pack-100b, 100a and 75 towards medium size in Pack-50 at Fz. For EM, at Cz, the significant MMN was small in Pack-100a and -50, but medium in Pack-75, while it increased from medium size in Pack-100a to large size in Pack-75 and -50 at Fz. No main effect of system, and no two-by-two system comparison reached significance, whatever the sampling rate or the pack, at any electrode site.

There was no system effect on the MMN latency, peaking around 180–200 ms for every dataset (Table 10). Detailed two-by-two comparisons did not reveal any significant difference.

Between-system correlations

Correlations calculated to assess measurement consistency across participants between the three systems (LA/BA, EM/LA, and EM/BA) are detailed in Table 11. Theses correlations reached significance (p < 0.05) when exceeding r = 0.47. With respect to time/frequency measures (alpha-power and SSVEP), most correlations were significant or near-significant (even with reduced data-sets), with the notable exception of the EM-BA relation for 15 Hz condition. With respect to ERP components, the systems were highly correlated for both the N170 amplitude and latency at P7, but only for latency at P8. For the N200 as P300, few correlations were significant for latencies, but correlations were largely significant for amplitude. Taken together, various indices supported relatively high measurement consistency between participants for time/frequency measures across the systems, and moderate consistency for ERP components with the exception of the MMN, (see Figs. S1 to S6).

Standardized measurement error

Data are shown in Table 12. Key findings were (1) systematically lower SME for the LA system measuring MMN, N170, N2 and P3 amplitude (LA < BA < EM), (2) no difference between systems when measuring latencies except the EM > LA = BA pattern at P8 and the EM > BA at P7 for the N170, (3) regarding the N170, each system had a lower SME than the ERP CORE dataset for both amplitude and latency, whereas EM exhibited a higher SME that the ERP CORE, (4) as to the MMN latency, higher SME was observed for each of the three systems as compared with the ERP CORE dataset (see Tables S1, S2 and Fig. S7).

Effect sizes for every system, according to task duration/number of trials

Table 13 provides an overview of the effect sizes for each task, system and Pack (expressed in seconds or number of trials). This synthesized presentation meets our objective regarding guidelines for researchers.

Spectral analyses

Consistent with our hypothesis, all three EEG systems captured the occipital Berger Effect already observed in adults (Johnstone, Blackman & Bruggemann, 2012; Cannard, Wahbeh & Delorme, 2021; An et al., 2022) and children (Barry et al., 2009; Bhavnani et al., 2022; Edgar et al., 2023). The effect was stable and large in size even in the 30-s Pack-50 with every system, though it was of larger size with LA and BA compared to the consumer-grade EM system. These differences in spectral responses could not be attributed to discrepancy between recording resolution, as the three systems were equated in sampling rate. Though the Berger Effect has previously been reported with very minimalist devices (Johnstone, Blackman & Bruggemann, 2012; Debener et al., 2015; Shivaraja et al., 2023), mixed results were obtained from a 14-electrode EM system (Grummett et al., 2015; Mahdid, Lee & Blain-Moraes, 2020; Kounios et al., 2024). In the current study, the 32-electrode mobile EM system proved effective in assessing oscillations and succeeded in demonstrating the Berger Effect.

We also found that flickering stimuli evoked oscillatory brain responses exactly at the periodic stimulation frequency and its two first harmonics (SSVEP) across every EEG system. It should be noted that, with LA, these effects were large in size even on a short recording (15-s), highlighting the efficiency of this portable research-grade system in assessing oscillations, but a drop in effect size was however observed in Packs-75 and -50 compared to Pack-100 for LA, not for BA. Consequently, using only 15- or 22.5-s recording, comparable effect sizes were obtained with LA and EM systems, in line with similar spectral effects shown using EM and a medical-grade EEG system during emotional clips perception (Katsigiannis & Ramzan, 2018).

Despite showing lower SNR responses than LA and BA, the EM system revealed effective for spectral analyses, probably due to the choice of wet electrodes for EM and LA. Indeed, systems with dry electrodes have been found less efficient (Xing et al., 2018; Ladouce et al., 2022). However, there are two restrictions. Firstly, for SSVEP, increasing the dataset duration up to 30 s led to higher effect size with LA, but not EM. Therefore, longer stimulation did not compensate for less exploitable signal from EM. Secondly, EM captured the expected SNR increase at the 6 or 15 Hz fundamental frequency of a periodic visual stimulation and its harmonics, but it failed to do so in response to 10 Hz-stimulation, contrary to LA and BA. This may come from a temporal instability of SSVEP responses, as a decline of amplitude specifically after 10 and 15 Hz has been documented (Łabęcki et al., 2024). Another possibility is that 10 Hz is part of the alpha range (8–12 Hz), which is vulnerable to changes in endogenous attentional states. Endogenous attention has been shown to interfere with SNR variations and decrease their amplitude in response to flickers at alpha rhythm (e.g., 10 Hz) (Gulbinaite, Roozendaal & VanRullen, 2019). To avoid alpha wave contamination, slightly higher rates (12–13 Hz) are preferred in SSVEP experiments (Yang, Paller & Van Vugt, 2022). In sum, EM was found to record satisfactory SSVEP signature, but caution is advised when EM is used to assess the brain’s ability to discriminate images below the threshold of consciousness.

ERP analyses

Compared to spectral aspects, ERPs require time/amplitude analyses on very short signal durations, and the ability of consumer-grade systems to do so has been rarely questioned. Below, we discuss our results with respect to the N170, N200, P300, and MMN.

First, all three EEG systems showed a large N170, peaking at its typical posterior site (P7 and P8) in the face condition, with a much smaller amplitude, texture conditions, consistent with its well-known association with face perception (Bentin et al., 1996). This N170 was robust enough to remain large after LA and BA downsampling, or dataset reduction (e.g., Pack-50, 150 trials). The EM system detected an N170 that was only slightly smaller than with BA (but comparable to the LA system, except in the very short 150-trials dataset). Its efficiency in detecting the N170 was in line with previous studies examining similar systems (De Lissa et al., 2015; Soto et al., 2018; Wehrman et al., 2021; Sagehorn et al., 2024). Taken together, the results contribute to validate the use of mobile, wireless and low-cost EEG systems for research about N170. Note, however, that the posterior N170 prominence was less typical from EM than from LA and BA. Additionally, an overall reduction in latency occurred using EM for most of the ERP components, including N170. EM users are therefore encouraged to focus on ERP amplitude, and should be aware of shortcomings regarding spatial distribution and temporal precision.

Second, although the N200 has not always been obtained in auditory oddball studies (Falkenstein, Hoormann & Hohnsbein, 1999, 2002; Smith, Johnstone & Barry, 2004), all three EEG systems were able to detect it as a medium-to-large cerebral response with consistent frontal prominence in our auditory oddball task, despite dataset reduction for every system and sampling rate lowering (LA and BA). N200 amplitude did not significantly differ between systems. This effectiveness of the EM system was consistent with a previous study (Barham et al., 2017). Like the N170, the N200 peaked earlier using the EM than the BA system, reflecting lower time precision for EM.

Third, the P300 also appeared as large-sized with the three systems at the expected fronto-central site, replicating (Badcock et al., 2015). LA contrasted with EM and BA by failing to show Cz prominence for P300 in short datasets (Packs-75 and -50). EM and BA systems succeeded in recording P300 of similar size and were unaffected by dataset reductions. The efficiency of the EM system here contrasts with a previous study comparing EM with a standard system (Duvinage et al., 2013). However, it is worth noting that this previous study, which evaluated EM for medical diagnosis, differed in several aspects from ours (e.g., participants were walking, deviants were frequent, and EM was compared to a medical-grade EEG device). Nevertheless, like the N200 and N170, the P300 appeared earlier with EM than LA and BA.

Fourth, in line with our hypothesis, a significant fronto-central MMN was observed across all systems and all sampling rates, with stable prominence in effect size at Fz over Cz and Pz. At Fz site, the MMN was stable in amplitude (medium to large) over all systems, in line with MMN comparison with another research-grade system (Neuroscan) (Williams et al., 2020). Mixed results were previously obtained using EM to assess the MMN (Badcock et al., 2013, 2015). Note the decrease in the MMN size with the dataset reduction from Pack-75 (505 trials) to Pack-50 (337 trials) using EM and LA, contrary to BA. As only LA also exhibited sensitivity to the number of trials for P300, users should be aware of the risk of employing few trials to assess ERPs with this system. In contrast with the other investigated components, the MMN was not observed earlier using EM than the other systems. Therefore, the MMN was of satisfactory amplitude, latency, and topography even when recorded with a wireless and consumer grade EEG system.

Overall, one key finding is that the EM system recorded ERPs of reasonable quality. Each targeted ERP component had satisfactory amplitude and intra-individual variability measured by SME reached an appropriate level as compared with standards (Zhang & Luck, 2023). This is in relative contrast with previous reports (Grummett et al., 2015). We speculate that this might be due to the use of a custom acquisition software to improve synchronization between stimuli presentation and EEG recording (Tahmasebzadeh, Bahrani & Setarehdan, 2013). Note, however, that shorter latencies were observed for three of the four assessed ERPs using EM compared to both LA and BA. The observed lack of time precision replicates previous results obtained when the EPOC-Flex was compared to the Biosemi System (Ries et al., 2014). More generally, discrepancies related to latencies have been shown with portable EEG systems (e.g., Nautilus) compared with literature data (Melnik et al., 2017), though reports are inconsistent (David Hairston et al., 2014; De Lissa et al., 2015; Williams et al., 2020). Here, we can rule out that this limitation is due to a difference in electrolyte (Ries et al., 2014), because saline solution was used with EM as well as with LA. As a tentative explanation, it can be suggested that time imprecision could be partly due to either the method used to synchronize the EEG data with the events of interest or the internal filters of the EM amplifier. Furthermore, we could find no indication in our results to attribute this difference to a participant effect or task order. In addition, we used robust paradigms adapted from ERP CORE (Kappenman et al., 2021).

In line with Williams, McArthur & Badcock (2021), the low-cost mobile EM system can be recommended for spectrum analyses of EEG signal recorded over relatively long intervals, but it is less precise for time and has to be used with caution to assess some ERPs.

Interestingly, as a portable research-grade system, LA provided robust results with respect to ERP detection. Only minor limitations have been observed regarding LA, (1) for N170, it was less effective than BA in the smallest dataset, and it did not outperform the consumer-grade EM system in terms of effect sizes in larger datasets, (2) slight topographical variability was noticed for P300 in small datasets, (3) the MMN amplitude dropped with the LA and EM systems in the smallest dataset. Some of these weaknesses could be attributed to impedance maintained below 20 kΩ for BA, but only 50 kΩ for LA. Interestingly, no latency difference was observed between ERPs recorded with LA and BA, which suggests a higher temporal precision than with EM.

Taken together, the LA system appears to record reliable signal even with lowered sampling rate, saving energy and memory space, and to be effective except with very few trials.

General considerations regarding the mobile consumer-grade EM and the portable research-grade LA

Over the last few years, EM has been the prevalent choice for cognitive scientists using consumer-based EEG systems (Sawangjai et al., 2020; Sabio et al., 2024; Gramann, 2024), probably because of its large number of electrodes ensuring wider scalp coverage, and its attractive price. In our study, its signal quality yielded less exploitable data than research-grade systems. Additionally, a major incident (lost amplifier/acquisition computer connection leading to change the headset) occurred twice only with EM. However, with 90% of participants in the final dataset and enough epochs to conduct analyses, EM exhibited overall satisfactory signal quality and can be seen as a reliable tool for researchers, as emphasized in reviews (Lau-Zhu, Lau & McLoughlin, 2019; Niso et al., 2023; Sabio et al., 2024; Jacobsen et al., 2024; Gramann, 2024) and validation studies (Maskeliunas et al., 2016; Melnik et al., 2017; Sawangjai et al., 2020; Khng & Mane, 2020; Williams et al., 2020; Browarska et al., 2021; Williams, McArthur & Badcock, 2021).

In a previous study, Niso et al. (2023) investigated every characteristic of 48 portable systems, highlighting notable advantages of LA over EM system, justifying a price ten times higher for LA than EM. This is consistent with the score for system specifications (CoME S) proposed by Bateson et al. (2017), which considers electrode type, bit resolution, sampling rate, and battery life, on a 4 to 20 scale. According to this scale, LA obtained a higher score with 15-10 than 8-7 for EM. Furthermore, SME measurements have provided evidence for fewer variability in LA data than in the other systems data (often including the BA system). Some higher performance observed from LA over EM system can be attributed to transmission mode (Wi-Fi used for LA is better than Bluetooth used for EM), discrepancies in hardware filter responses, and in amplifiers. Regarding the latter, the bandwidth (Hz) was higher for LA than for EM (131 Hz vs 0.16–43 Hz), and the common mode rejection ratio (CMRR) is 80 dB for LA, but it is unknown for EM. Both characteristics could impact the signal amplitude and SNR, which was lower in EM than in LA. The impact of these features, emphasized by Gramann (2024), increases in brief experiments.

More generally, although EM can certainly be used as a research tool (Maskeliunas et al., 2016), it is important to keep in mind that, as well as other consumer-grade systems, EM does not provide raw data and the signal is already preprocessed in untraceable way (Browarska et al., 2021). Despite the exhaustive study by Niso et al. (2023), some technical information is not known and could explain temporal imprecision and slightly lower amplitude of the ERP.

In summary, the portable LA system meets the requirements of a research-grade system with good signal quality and temporal accuracy, but it still underperforms its wired counterpart (BA). Furthermore, although the EM system is definitely more affordable and offers good scalp coverage, as well as a portability score equivalent to that of the LA (CoME D = 3, Niso et al., 2023), it only provides preprocessed data, has a lower SNR, and has shown technical problems such as amplifier-computer disconnections, which need to be taken into account, particularly for time-sensitive measures such as ERPs.

Limitations

This study contributes to the growing body of research assessing the validity of portable EEG systems to measure neural oscillations and ERPs, and is notable for its analysis of the minimum test duration required to obtain usable data. However, the current findings should be interpreted in light of some limitations.

First, while the sample size was rather large in the context of studies comparing the validity of wireless vs wired EEG systems, it remained limited. It might have prevented the detection of small differences between the systems due to insufficient power. Nevertheless, Melnik et al. (2017) calculated that 8% and 9% of variance in ERP measures can be respectively explained by variability between subjects and EEG systems if 16 participants were tested. Thus, our 19-participant group accounted for even smaller part of ERP variance in the present study. It is possible that inter-individual variability may come from contextual differences, since participants were tested at different times of the day, and were not forbidden from drinking coffee or smoking before EEG recordings. For instance, Chen et al. (2020) showed that caffeine intake may enhance the efficiency of individual automatic responses and early cognitive processes. Cui et al. (2020) also showed that abstinence in young smokers may have an impact on the effect of neurophysiological indicators of performance control. However, as shown by the relatively high between participants correlation values observed across systems for time/frequency measures and most ERPs, measures were relatively stable between systems. The lack of correlation for the MMN can tentatively be explained by the specificity of the electrode reference (the common average) used to calculate this ERP component, because of the differential changes in frontal and sub-temporal components of mismatch negativity (Baldeweg, Williams & Gruzelier, 1999).

Second, another limitation concerns comparisons of the SME data for the three systems to the ERP CORE dataset due to few compatibilities between electrode sites of interest. This calculation conveys mitigated conclusions since variability was overall lower than the ERP CORE dataset for the N170 amplitude and latency, while variability was higher than this reference for the MMN latency.

Third, due to a technical problem, the BA system was not used with its optimal sampling rate, which was paradoxically lower than for the LA system. This problem was indirectly solved by analyses conducted with an equalized 128 Hz-sampling rate, initially designed to rule out an interpretation of between-system differences based solely on recording resolution.

Fourth, although researchers may be inclined to use mobile systems, particularly for real-life studies, our study focused on immobile participants. Future research could explore the suitability of these EEG systems in real-world settings, which would involve addressing artifacts caused by large gestures and making informed decisions about preprocessing (Jacobsen et al., 2024). Guidelines on the acceptability of these systems (Bhavnani et al., 2022) and the selection of hardware are available elsewhere (Sawangjai et al., 2020; Niso et al., 2023; Gramann, 2024).

Finally, future studies might extend the comparison of these systems towards tasks involving later cognitive processing, such as sensitivity to semantic or syntactic violations through N400 and P600 components, or addressing spectral power in other bands.

Practical guidelines

Based on the present study, the following points can be offered as guidelines to researchers wishing to use a portable EEG system to address scientific issues. First, not only the LA, but also the EM consumer-grade EEG system can be recommended if time/frequency analyses on brain oscillations are planned, and no increase in the recording duration is needed. For instance, large changes in the alpha-band, and typical responses to the frequency of flickering stimuli and its harmonics can be observed from 30- and 15-s recordings. Second, whether the system was wired or portable, ERP amplitude did not significantly decrease with the reduction of the number of trials (150, 200 and 300 trials for the N170, N200 and P300, respectively). Except for N200 (and N170 in the texture condition), which can be expected to be of medium size from EM, large-sized ERPs can be recorded using wireless as wired systems. Third, studying ERP latencies is possible with LA, but is compromised by temporal imprecision of EM. Fourth, capturing the MMN with the EM or LA portable systems proved possible, but to a lesser extent than other ERPs, and the number of trials should not be lower than 505 to expect a medium-sized MMN using EM system, while it can be reduced towards 337 using LA or BA.