The Sensed Presence Questionnaire (SenPQ): initial psychometric validation of a measure of the “Sensed Presence” experience

Design of the Sensed Presence Questionnaire (SenPQ)

The Sensed Presence Questionnaire (SenPQ) is comprised of 16 questions. These were derived from a literature review of the sensed presence phenomenon spanning studies from sleep paralysis, epilepsy and other neurological disorders, psychosis, stress and anxiety, ritual, drug induced experiences, and the general population. As well as covering a range of typical sensed presence experiences from the scientific literature, the scale also includes items that are positively and negatively valenced, as well as neutrally valenced in their presentation.

The questionnaire requests that respondents refer to experiences from the last month only when rating the items, and not to record any perceptions associated with drug-induced experiences. Respondents are asked to indicate the frequency with which the experience has occurred using a Likert-like scale: ‘Never’, ‘Occasionally’, ‘Sometimes’, ‘Very Often’, ‘Always’. The questionnaire is freely available online and has been released under a Creative Commons license at the following link: https://osf.io/fecgz/.

Participants

Participants were recruited via two methods: (i) online via the http://proflic.ac online study recruitment platform that has diverse participant base and where we received 101 completed responses from separate individuals (Sample S1) from 135 responses in total including incomplete responses. In addition, social media advertisements were sent from the authors’ personal accounts and accounts associated with the authors’ university department (with notices that did not refer to anything spiritual or religious) and emails were sent to religious groups including university religious societies (Hindu society, Islamic Society, Christian Union, Sikh Society, Buddhist Society, and the Jewish Society) and local churches requesting participants (Sample S2). Recruitment for both samples was started in parallel. The questionnaire took approximately 25 min to complete. Participants recruited via the online recruitment service were paid £4 upon questionnaire completion. All participants were directed to the same online questionnaires.

Individuals who clicked on the link to the survey and began to fill in the survey were considered to have been recruited into the study, regardless of whether the scales were fully completed, although only complete questionnaire sets were entered into the analysis. Questionnaires were preceded by a page requesting demographic data that required age, gender, religious belief, average meditation practice, ethnic group, and education level.

Both samples were combined for the final analysis which consisted of a total of 191 participants included (see Fig. 1). Analysis scripts that conduct a separate analysis for each sample (minus the principal components analysis which is underpowered when not conducted on the whole sample) are available on the Open Science Framework page for this study (https://osf.io/fecgz/), which demonstrate the same pattern of results in each sample, supporting the reliability and validity of the scale.

Additional measures

Other Experiences Questionnaire—social imagery subscale (OEQ7) (Solomonova et al., 2008): a validated subscale designed to capture social imagery, and has previously been established as reliable and valid in the context of sleep paralysis and anxiety. The frequency of each item is rated on a 4-point Likert scale. This measure has been previously used to measure an aspect of the sensed presence experience, and we predicted that it would correlate with SenPQ scores, indicating convergent validity. The internal reliability for the scale in this study, measured using Cronbach’s Alpha, was 0.706.

Brief Oxford-Liverpool Inventory of Feeling and Experiences (O-LIFE) (Mason, Linney & Claridge, 2005): a briefer, validated version of the O-LIFE schizotypy scale (Claridge et al., 1996) that includes four subscales that measure different aspects of schizotypy: unusual experiences (UE), cognitive disorganisation (CD), introvertive anhedonia (IA), and impulsive non-conformity (IN). Each item is rated on a binary scale, with participants either affirming or disagreeing with each statement. We predicted a selective association between the SenPQ and the unusual experiences subscale of the O-LIFE, indicating that the sensed presence experience reflected a form of reality distortion experience but no other cognitive or behavioural aspects of schizotypy. Internal reliability of the scale in this study was 0.863.

Daily Spiritual Experience Scale (DSES) (Underwood & Teresi, 2002) is originally a 16-item validated scale designed to assess spiritual experiences in daily life. This scale was based on qualitative research and has been tested for reliability and validity in a range of populations (Underwood, 2011). The first 15 items are answered on a 6-point Likert scale that ask about frequency of particular spiritual experiences from ‘many times a day’ to ‘never or almost never’. The final item is a general question about ‘how close do you feel to God’ and is answered on a four point Likert scale from ‘not close at all’ to ‘as close as possible’. This item was omitted as it has been found not to be reliable for some Hindu and Muslim respondents (Husain et al., 2016) who were groups we specifically invited to participate. However, the DSES has been shown to be reliable and valid in this 15-item form Kim, Martin & Nolty, (2016) and Kurian et al. (2016). Considering that the sensed presence experience has been reportedly more frequently in spiritual and religious contexts, we predicted a positive correlation with the SenPQ. Internal reliability of the scale in this study was 0.936.

We also included two scales that measured general psychological well-being and social interaction anxiety for an exploratory analysis and use as potential covariates, given that sensed presence experience has been previously associated with psychopathology in some populations.

WHO-5 Well-Being Index (World Health Organization, 1998): a brief, validated well-being questionnaire, shown to capture well-being as accurately as its extended version (Topp et al. , 2015). Each item describes an aspect of well-being and the respondent is asked rate how present it has been during the last two on a 6-point Likert scale ranging from ‘all of the time’ to ‘at no time’. Internal reliability of the scale in this study was 0.844.

The Social Interaction Anxiety Scale (SIAS) (Mattick & Clarke, 1998): a validated scale designed to capture anxiety associated with meeting, conversing, or mixing with others. Each item describes anxiety associated with a particular social interaction scenario and the respondent is asked to rate each item on a 5-point Likert scale from ‘not at all’ to ‘extremely’. Internal reliability of the scale in this study was 0.950.

Participants were also given the option of a free text box at the end of the questionnaire series to describe a sensed presence experience in their own words if they indicated they had experienced one. This information was not used in the current study.

Analysis

Internal reliability of the SenPQ was tested with Cronbach’s Alpha (Cronbach, 1951). Validity was determined firstly by whether SenPQ score distinguished between religious and non-religious groups by using an independent samples t-test, and we predicted a significant difference between SenPQ in religious and non-religious groups, indicating discriminant validity. Secondly we examined the association between SenPQ score and additional measures, predicting that the SenPQ would selectively correlate with the O-LIFE unusual experiences subscale, the OEQ7 measure of social imagery and the DSES measure of spiritual experience, indicating convergent and divergent validity. We had no specific predictions about associations with the measures of anxiety and well-being, as sensed presence experiences have been previously associated with a range of affective states (Alderson-Day, 2016).

In addition, we completed an exploratory principal components analysis of the SenPQ items on the entire sample using direct oblimin rotation based on the assumption that underlying factors would not be independent. The mean item scores for each individual factor loading were used for all subsequent analysis. Parallel analysis for principle components (95% confidence interval, 1,000 random correlation matrices) (Horn, 1965; O’Connor, 2000) and observation of the scree plot (Cattell, 1966) were used to select retained factors. Subsequent factor analyses were completed to specifically extract models for specific numbers of retained factors.

An Item Response Theory (IRT) analysis was performed on the entire dataset for scores on the SenPQ. IRT is a model-based theory that measures the responses between items and the trait that each item is supposedly measuring (Emberston & Reise, 2000)—in this case, the sensitivity to sensed presence experiences. A Graded Response Model (GRM; Samejima, 1969) was used in this particular IRT analysis—this is due to the SenPQ using a polytomous scoring system. The GRM model attributes each item as a series of response dichotomies, or thresholds (β) compared to the discrimination parameter, or slope (α)—because the SenPQ items comprised 5 graded choices (1- Never to 5- Always) this represents four β: 1 vs 2–5 (β₁), 1–2 vs. 3–5 (β₂), 1–3 vs. 4–5 (β₃), and 1–4 vs. 5 (β₄). These thresholds correspond to the trait level (θ) at which a new individual answering the questionnaire has a 50% chance of endorsing the relevant or higher response. These thresholds are not the same for each item, and each item will have its own set of thresholds.

Graphical illustrations of this relationship were also created. An Item Information Function (IIF), Test Information Function (TIF), and Test Characteristic Curve (TCC) was generated for the SenPQ. An IIF displays at which locations along the trait scale each item is able to be accurate about a new individual using the questionnaire. A TIF reports the level of precision of the entire measure at different points along the trait scale. A TCC shows the expected score of the measure at different points along the trait scale.

We also completed a receiver operating characteristic (ROC) analysis to examine how specific scale scores distinguished between religious and non-religious groups to additionally test discriminant validity across the range of the measure.

Demographics

The total sample consisted of 89 males, 99 females, 2 who selected ‘Other’ and 1 who selected ‘Prefer not to say’. The mean age of the sample was 36.23 (SD = 13.4; Range 17–73). Religious affiliation, ethnicity and level of education are reported in Table 1. For the purposes of classifying people into religious and non-religious groups for further analysis, people who selected ‘No Religion’ or ‘Agnostic’ were considered non-religious, as were individuals who recorded themselves as ‘Atheist’ under the ‘Other’ option. All others were considered religious.

20 participants reported daily meditation practice, 10 weekly practice, 23 monthly practice, and 138 no practice. Self-reported ethnicity of the sample was “English / Welsh / Scottish / Northern Irish / British” (N = 131), “White other” (N = 25), “Indian” (N = 8), “White and Asian” (N = 5), “Chinese” (N = 4), “White and Black African” (N = 3), “Pakistani” (N = 2), “Other” (N = 2), “Irish” (N = 2), “White and Black Caribbean” (N = 2), “Other Mixed / Multiple ethnic background” (N = 2), “Bangladeshi” (N = 1), “African” (N = 1), “Caribbean” (N = 1), “Arab” (N = 1), “Gypsy or Irish Traveller” (N = 1).

Statistical analysis

Due to the sampling distribution of mean scores on the SenPQ violating the assumption of normal distribution, all analyses were conducted using a simple bootstrap re-sampling method (1,000 samples, 95% CI) (Bland & Altman, 2015). All data analysis used SPSS v.22 (SPSS Inc.), except for the IRT analysis which used STATA v.14 (Stata Corp). The raw data and analysis scripts for this study are freely available online at the Open Science Framework at the following link: https://osf.io/fecgz/.

Internal reliability

All SenPQ items where entered into internal reliability analysis and the scale demonstrated very high internal consistency (Cronbach’s alpha = 0.951).

Validity

Means and standard deviations for the scale scores are displayed in Table 2. Discriminant validity of the SenPQ was demonstrated by conducting an independent samples t-test (two-tailed) between mean scores from religious (N = 71) and non-religious groups (N = 120). The religious group had a higher mean score than the non-religious group (see Table 2), a difference which was significant when tested with an independent samples t-test (t =  − 3.592, p = 0.002, mean difference = −5.208, 95% CI [−8.098–−2.156]; Cohen’s d = 0.51), indicating good discriminant validity.

As can be seen in Table 3, the SenPQ demonstrated a strong significant correlation with the OEQ-7 social imagery scale and a moderate significant correlation with the DSES daily spiritual experiences scale. There was a strong significant correlation with the unusual experiences subscale of the O-LIFE schizotypy scale, a weak correlation with the impulsive non-conformity subscale, and no significant correlation with the cognitive disorganisation or introvertive anhedonia subscales, indicating good convergent and divergent validity.

Demographic analysis

No significant correlation between the age and SenPQ score (Pearson r = 0.026, p = 0.723) was found and no significant difference between genders when tested with an independent samples t-test (t = 1.268, p = 0.206). There was no significant effect of education level on SenPQ score when tested with a one-way between subject ANOVA (F_(3,187) = 1.100, p = 0.350). Because so few people reported meditation practice at the more frequent end of practice, this was collapsed into a binary ‘yes/no’ variable. People who reported engaging with any sort of meditation practice were significantly more likely to score higher on the SenPQ when tested with an independent samples t-test (t = 3.222, p = 0.001).

Measures of anxiety and well-being

No significant associations were found between the SenPQ and WHO-5 score (r = .56, p = .443) and between the SenPQ and SIAS score (r = .025, p = .773).

Factor analysis of the SenPQ

To investigate the factor loadings of the Sensed Presence Questionnaire, a factor analysis was run on all 16 items.

To test assumptions, a Kaiser–Meyer–Olkin measure of sampling adequacy (0.925) and Bartlett’s test of sphericity (χ2(120) = 2467.009, p < 0.001) were run and considered adequate, with all items significantly correlating by at least 0.3 (p < 0.001).

An initial factor analysis suggested a two factor solution based on inspection of the scree plot that indicated a clear break after two components (Cattell, 1966). The parallel analysis conducted using O’Connor’s (2000) method suggested a one factor solution, based on the fact that only one eigenvalue from the study data set was greater than the simulated equivalent for randomly generated correlation matrices, with the second factor being marginally below the cut-off. We subsequently conducted two separate factor analyses that specifically extracted one and two component solutions and subsequently judged the two component solution to be more interpretable. The first factor was interpreted as ‘benign presence’ and the second factor was interpreted as ‘malign presence’. The first component explained 52.28% of the variance, and the second component explained an additional 7.66% of the variance in the sample. The pattern matrix can be found in Table 4.

Item-response theory graded response modelling

The SenPQ was submitted to an IRT GRM bootstrap analysis (1,000 replications) to understand the relationship between difficulty of items and sensitivity to sensed presence experiences.

The analysis reported that all items had moderate to large slopes (α = 1.76–4.51) with a model log likelihood value of −1947.92. All β values were evenly spaced and ascending from β ₁ to β₄. Two β₄ thresholds were not available because no participant answered ‘5-Always’ on item 3 and 7: ‘When I was under a lot of pressure, I felt someone or something was accompanying me’, and ‘When I have visited specific locations, I felt I was in the presence of an unseen being or beings’, respectively.

Table 5 shows the results for the slopes (α) and threshold parameters (β1–β4) for all items on the SenPQ.

Observing the slopes and the residuals suggests that all items were providing high item level and test level information, with item 9 being the ‘easiest’, and item 5 the ‘hardest’.

Residuals at the test and item level were also analysed graphically. Figure 2 displays the IIF, TIF, and TCC for the SenPQ.

Because a large portion of the curves are above 0θ in Fig. 2, this suggests that the SenPQ is better designed for respondents with higher sensitivity to sensed presence experiences.

ROC analysis

Data from the psychometric measures was entered into a ROC analysis to show they discriminated religious from non-religious groups over the extent of their score range. Results are displayed in Table 6 and Fig. 3.

The most efficient total discriminator of religious and non-religious groups was the DSES. The SenPQ and OEQ7 were highly discriminant and perform almost identically. The Unusual Experiences subscale of the O-LIFE schizotypy scale discriminates between groups to a lesser extent, and all other scales show no significant discriminant ability.