Personality Assessment Inventory (PAI): obsolete norms identify psychopathology in nearly everyone

Participants

The participants were 207 undergraduate students enrolled in introductory psychology classes in a mid-size university in Canada who participated in the study for a course credit. Students were administered the PAI following the PAI Manual instructions (paper & pencil administration) as part of a 2-hour long study on individual differences in cognition and personality. Participants were tested in small groups ranging from 1 to 10 students. The study was approved by the Mount Royal University Human Research Ethics Board (HREB #103697). Written informed consent was obtained from all participants.

Participants’ data were excluded if they (a) left more than 17 items missing or (b) produced invalid profiles defined as ICN ≥ 13 (ICN ≥ 73T) or INF ≥ 9 (INF ≥ 75T) (Morey, 2007). In total, one participant was excluded due to having more than 17 missing item responses and two participants were excluded due to invalid profiles. Following these exclusions, 204 participants were left in the analysis (160 women, 38 men, one transgender female, zero transgender male, two gender variant, two prefer not to answer, 1 other; age in years: M = 21.2 years, SD = 4.96; race/ethnicity: 49.0% Caucasian, 13.7% Asian, 7.8% African, 7.4% Indian/Brown, 4.9% Hispanic/Latino, 5.9% Middle Eastern, 4.9% Native American, 5.4 Other, 0.9% Prefer not to say).

Measures

The Personality Assessment Inventory (Morey, 1991; Morey, 2007) is a 344-item self-report measure of personality. It includes 4 validity scales, 11 clinical scales, 5 treatment scales, 2 interpersonal scales, numerous subscales and numerous other indexes. All items are answered on a 4-point scale: 0 = False, not at all true, 1 = Slightly true, 2 = Mainly true, 3 = Very true. Tables 1 and 2 shows the scales and subscales and their abbreviations. The PAI scales, subscales, and supplementary scales are calculated as described in the PAI manual, and the resulting T scores (M = 50, SD = 10) reflect a comparison to the US 1990 adult community norms (Morey, 2007; p. 2). The authors have permission to use this instrument from the copyright holders (Morey, 2007).

Statistical analyses

All statistical analyses were performed using R (R Core Team, 2021). We examined reliability of scales, subscales and supplementary scales using Cronbach’s alpha to ensure comparability of our findings with those reported in the PAI Manual (Morey, 2007) and other published research reviewed in the introduction. However, we also provide McDonald’s omega designed to address some of the limitations of Cronbach’s alpha. Importantly, we note that McDonald’s omega has problems of its own (e.g., estimates fail to compute), and, more importantly, the simulation studies show that McDonald’s omega and Cronbach’s alpha result in nearly identical estimates when sample sizes are reasonably large, reliabilities moderate to high, and scales have more than a few items (Edwards, Joyner & Schatschneider, 2021).

Previous “abnormal” has become new normal

The large shifts in how today’s Canadian university students score on the PAI are consistent with the previous studies of US university students and with a number of studies recently conducted with both university students and community adults in the US and in other countries, especially post COVID-19 pandemic detailed in the introduction. The large shifts in the PAI scores are also consistent with the wealth of independent evidence demonstrating large increases in mental health issues and psychopathology, especially in depression, anxiety, and other anxiety related disorders. For example, Statistics Canada reported that the 12-month prevalence of depression, anxiety, and anxiety related disorders increased substantially between 2012 and 2022: nearly doubled for Major Depressive Episode (4.7 vs. 7.6%), doubled for Generalized Anxiety Disorder (2.6 to 5.2%), and more than doubled for Social Phobia (3.0 to 7.1%). In contrast, Statistics Canada reported no increase in Alcohol Use Disorder (3.2% to 2.2%), Cannabis Use Disorder (1.3% to 1.4%), and Other Drug Use Disorder (0.7% to 0.5%) over the same time period (Stephenson, 2023). Similar doubling of rates of depression have been reported in US university students by Lipson, Lattie & Eisenberg (2019) as well as other countries (Buizza, Bazzoli & Ghilardi, 2022). Thus, higher scores on anxiety and depression scales are consistent with increases in diagnosed anxiety and depression disorders among university students.

Moreover, some studies suggest that stigma surrounding some common mental disorders, for example, Major Depression has decreased substantially. To illustrate, Pescosolido et al. (2021) found that the percentage of US respondents who expressed unwillingness to work closely with, have as a neighbor, socialize with, make friends with, and marry into family of persons with Major Depression all decreased. Similarly, numerous studies have documented increases in the percentage of students asking for and receiving various accommodations due to various disabilities including test anxiety (Suhr & Johnson, 2022). Accordingly, one possible reason for increases in the PAI scores is greater willingness of respondents to disclose aspects of their mental challenges independently of the increased documented prevalence of various mental disorders and disabilities.

The large shifts in the PAI scales scores relative to the US 1990 adult as well as college students normative samples are likely due to both temporal/cohort changes and cultural differences between the US and Canada. The temporal/cohort changes likely include changes in psychopathology, changes in Canadian culture itself, and changes in respondents’ openness about mental health, that is, reporting style. The world was a very different place in 1990 when the PAI was normed than today. For example, in 1990, the Soviet Empire was collapsing, the Cold War was nearly over, and the prospect of nuclear armageddon was reduced. In contrast, today, Russia is in its fourth year of its “Special Military Operation”—otherwise known as the Russian war to occupy Ukraine, the World War III is regularly discussed in the news and social media, and, similarly, the threats of using nuclear arms by Russia appear regularly in the news. The way people obtain information, socialize, communicate, etc. has also changed massively with unprecedented changes in technology, such as an introduction of personal computers, world wide web, smart phones, tablets, and social media. Similarly, the composition of undergraduate students has also changed in the intervening 35 years years with a far greater proportion of population attending universities and graduating with university degrees (Uttl, Violo & Gibson, 2024). The COVID-19 pandemic has also resulted in unprecedented changes in society and mental health of populations in general. Critically, one has to keep in mind that personality questionnaires such as the PAI are composed of items, that meaning of the items changes, and that the endorsement of items changes with external conditions/situations independently of persons’ psychopathologies. For example, one can hardly consider greater endorsement of the PAI items asking respondents how much they worry about things today vs. 35 years ago as an indication of greater psychopathology. Critically, regardless of what has caused the changes in the PAI responses, the PAI norms are no longer valid because what once was “abnormal” is now the new normal.

PAI norms are outdated and obsolete

The PAI and many other psychological tests define what is normal in reference to a bell-shaped distribution of scores obtained by a representative sample of individuals from the populations of interest. Accordingly, when a population’s characteristics change over time due to a variety of changes and historical events, for example, due to changes in culture, education attainment, technology, and due to historical events and crises such as COVID-19 pandemic, the previously established norms become outdated, obsolete, and invalid. Accordingly, the tests have to be renormed to capture changes in the population’s characteristics and to establish the new norms, the new normal.

The PAI US norms (Morey, 1991) are 35 years outdated and obsolete. Our data show that they do not describe Canadian university students and the data we reviewed demonstrate that they no longer describe even the US university students or the US adult non-clinical population. In clinical settings, any interpretations of the PAI scores based on the US outdated norms are likely wrong, misclassifying large proportions of examinees as having elevated or high scores on various clinical and other scales when in fact they are perfectly normal, average, relative to their current peers. It is now nearly guaranteed that if one is assessed with the PAI US norms one is likely to have one or more elevated or high PAI scores. For example, using the obsolete PAI norms in fitness for duty assessment, an employee scoring 65T on the Anxiety scale would be flagged as unusually anxious, and possibly unfit for duty even though the employee’s Anxiety score was average and no different from other employees.

Accordingly, the PAI outdated US norms ought not to be used in Canada nor elsewhere and ought to be withdrawn by the test developer and the test publisher. The continued use of the US norms has likely caused harm to countless examinees, especially in forensic, employment, fitness-for-duty, child custody, and other high stakes assessment settings where obtaining PAI scores ≥ 60T may result in ineligibility for or loss of employment, loss of child custody, and other serious adverse effects.

Importantly, a number of the PAI validity indexes (e.g., PIM, CDF) featured in the PAI Manual are based on comparing very small groups of examinees asked to complete the PAI under various conditions. For example, the PIM index was developed by asking a small sample of 45 university students “to portray a very favorable self-impression, as if they were attempting to maximally impress a potential employer” (Morey, 2007, p. 162). Morey then compared the distribution of the PIM scores of these 45 students to the US 1990 Adult Normal sample and chose to set the PIM cutoff ≥ 57T, “the point at which the actual and simulated distributions cross.” (p. 161). Because of the large overlap in the distributions, this 57T cutoff correctly identified 82% of the 45 students as “faking good” and incorrectly classified 30% of the US 1990 Adults Normative sample as “faking good”. Similarly, the Cashel Discriminant Function (CDF) was developed on a small sample of 44 offenders and then cross-validated on a small sample of 36 university students (Cashel et al., 1995). In the Faking condition (23 students), 20 (87.0%) were identified as faking and 3 were identified as honest. In the Honest condition (13 students), 7 (53.8%) were identified as Honest and 6 (46.2%) were identified as “Faking”. The CDF is presented in the PAI Manual as “designed to optimally distinguish between defensive vs. honest responding” (p. 32) even though it misclassifies nearly 1/2 of honest examinees as “Faking”. Thus, these studies are not only tied to the outdated obsolete norms but they are also outdated and obsolete themselves since today’s university students respond differently under standard conditions and may also respond differently under the “fake good” conditions. Moreover, they are often based on such small samples that their findings are neither reasonably accurate nor generalizable beyond a study sample itself. No one should be labelled as “faking good” or dishonest in any high stakes assessment based on indexes that are obsolete and misclassify up to nearly 1/2 of the respondents as “faking good”.

Reliability of many PAI scales and subscales is inadequate for high stakes assessments

Previously published data as well as our data also show that the reliability of nearly all of the PAI scales and subscales is inadequate for use in high-stakes assessments. Although reliabilities in the .70 to .80 range may be acceptable for conducting research and reliabilities in .80 to .90 range may be acceptable for using tests in low stakes situations such as vocational counselling, a reliability of .90 is a bare minimum and a reliability over .95 is desirable for using tests in high stakes decisions such as eligibility for insurance benefits, fitness for duty assessments, child custody evaluations, etc. (Kaplan & Saccuzzo, 2013; Nunnally & Bernstein, 1994; Reynolds, Altman & Allen, 2021). For example, Kaplan & Saccuzzo (2013, p. 124-125) wrote:

... It has been suggested that reliability estimates in the range of .70 to .80 are good enough for most purposes in basic research... In clinical settings, high reliability is extremely important. When tests are used to make important decisions about someone’s future, evaluators must be certain to minimize any error in classification. Thus, a test with a reliability of .90 might not be good enough. For a test used to make a decision that affects some person’s future, evaluators should attempt to find a test with a reliability greater than .95.

As can be seen in Tables 1 and 2, only one out of 44 reliability coefficients reported in the PAI manual (Morey, 2007) for the PAI Scales for the college student sample reached the desirable bare minimum of .90 in terms of Cronbach’s alpha or test-retest reliability. In our sample, only five out of 22 Cronbach’s alpha coefficients reached or exceeded .90. For the PAI subscales, none of the reliabilities reached .90 in the US college sample nor in our sample. Low reliabilities result in large standard errors of measurements (SEm) and wide 95% confidence intervals. Whereas a clinician may think that an examinee’s PIM = 57T indicates some attempt to present themselves in a positive light, the 95% CI ranges from 47T to 67T indicating that the clinician has no evidence whatsoever the examinee engaged in any positive impression management.

Disappointingly, the PAI manual (Morey, 2007) does not mention the terms “error of measurement” or “confidence interval” even once in “Chapter 2: Administration and Scoring” or “Chapter 3: Interpretation”, the two chapters psychologists are most likely to read, if any. In the entire 385 page long manual (Morey, 2007), “confidence interval” is mentioned once, on page 135:

The standard error of measurement, expressed in points on the T-score scale, represent the standard deviation of the hypothesized error term as a contribution to observed PAI T-scores. Thus, one can be 95% certain that a participant’s true score on a PAI scale falls within 1.96 standard errors of measurement of his or her observed score. This confidence interval is particularly useful in attempting to evaluate the meaning of change over time in a person’s score–changes that do not exceed this interval are not likely to be clinically meaningful.

Notably, immediately after explaining what the 95% confidence interval is, the manual directs the reader’s attention to evaluation of “the meaning of change over time in a person’s score” rather than on the more immediate and more serious problem that every psychologist ought to know and consider when attempting to interpret test scores: what is the range of true scores that may have produced this observed score? Equally notably, the PAI interpretive software, including the latest version—PAI Plus (published in 2020 but using the original 1990 US norms), gives no indication of the SEms, 95% CIs, and no indication at all that the PAI’s scale and subscale scores and indexes are observed scores and merely estimates of the examinees’ true scores.

The problem of overpathologizing normal results and ignoring poor reliability of the PAI scales is illustrated by the PAI Manual’s exposition of the PIM scale’s rather low abnormality cut offs. The manual states (Morey, 2007, p. 31):

Moderate elevations (i.e., 57T to 67T) suggest that the individual responded in a manner to portray himself or herself as relatively free of the common shortcomings to which most individuals will admit. This response style could be overt, but it could also involve a covert, automatic defensive process. With PIM in this range, the accuracy of interpretations based on the PAI clinical scales profile may be distorted, and interpretive hypotheses should be reviewed with caution. It is likely that the PAI profile will underrepresent the extent and degree of significant test findings, although the influence on the PAI scales may be scale specific. For example, a respondent may be minimizing a drug problems but accurately describing problems with aggression.

Accordingly, if an examinee scores 57T on the PIM (i.e., 2/3 standard deviation above the mean of the normative sample), a psychologist who takes the manual for its words (and is unaware of what the observed scores actually represent, unaware of SEm, unaware of 95% confidence intervals) may report that the examinee tends “to portray himself as relatively free of the common shortcomings to which most individuals will admit”—copying the phrase directly from the test manual or the PAI computerized report—and decide not to interpret the PAI profiles because of the examinee’s “defensiveness.” The psychologist may not even bother to quote the statements nor cite the source of these boiler plate, stock phrases lifted directly from the manual or from the computerized report. A reader, an adjudicator, a court, or a tribunal assumes that the lifted phrase was actually the psychologist’s original work, presumably arrived to after thorough assessment and after some independent thought about the assessment, its circumstances, etc. by the psychologist/assessor. Given no quotation and no citation, the reader, the adjudicator, the court or the tribunal has no indication that the phrase was lifted straight from the manual or from the interpretive report. The examinee is forever labelled as a faker, or even worse, “defensive”.

The PAI manual’s approach of classifying examinees based on the PIM of 57T as portraying themselves “as relatively free of the common shortcomings to which most individuals will admit” is accusing the examinee of providing dishonest responses or lacking insight into their faults. This accusation is unscientific, inappropriate, unconscionable, and dishonest in and of itself. First, the PIM of 57T classifies 25% of the normative sample as being dishonest or lacking insight by definition, without any evidence whatsoever that they were dishonest or lacking insight.

Second, the SEm of the PIM is approximately 5T, and –quoting the manual’s interpretation of the 95% CI—“thus, one can be 95% certain that a participant’s true score on a PAI scale [PIM] falls within 1.96 standard errors of measurement of his or her observed score [PIM = 57T]”, that is, 47T to 67T. Since 47T is below 50T, the average of the PIM’s distribution in normative sample, the psychologist has no evidence whatsoever that the examinee’s true score is at or above 57T, and thus, no evidence that the examinee is dishonest or lacking insight, no evidence that “the PAI clinical scales profile may be distorted”, and no evidence that “it is likely that the PAI profile will underrepresent the extent and degree of significant test findings...”.

Third, Morey (2007) himself acknowledged the Positive Impression Management does not measure only the Positive Impression Management; Morey wrote (p. 30):

... There are a number of reasons why people completing a self-report instrument might not report negative characteristics. One possibility is that the respondent indeed does not have negative characteristics [the instrument asked about], or at least has fewer than most individuals. A second possibility is that they are not telling the truth—that they are trying to deceive... A third possibility is that they are simply not aware of certain faults that they may have... It is these later two characteristics that PIM was designed to measure.

What the PAI “was designed to measure” is irrelevant; the question is what it actually measures. The PIM measures not only the “later two characteristics” but also the first one, and thus, the PIM’s interpretation offered in the manual is completely unreasonable, and ultimately causing harm. Morey (2007) himself admits that a large proportion of the normal population is identified as “faking good” or being dishonest by indexes such as the PIM (p. 30):

It should be recognized that the tendency for favorable self-presentation appears to be fairly common in the normal population. Typically, most cutoff scores on indexes of social desirability that were derived from clinical studies will identify 30% to 40% of the general population as “faking good.” Such results underscore the difficulty of distinguishing defensive responding from normality with respect to clinical instruments...

If the index identifies honest examinees as “faking good”, the index is wrong, inappropriate, and its use in any high or even low stakes assessments is unconscionable and unethical. Labelling an examinee as dishonest, as “faking good”, is a severely defamatory accusation. No psychologist should ever make a claim that the examinee is dishonest when the psychologist has no idea what caused the particular score, has no idea about the errors of measurement, and has no idea how to interpret psychological scores. The severity of this relevant non-disclosure by psychologists is further compounded by the fact that a reader of a psychological report who is not a psychologist and not an expert (a) has no indication that they were misled, and (b) has no way to access the test manual to find out that they were misled about the PIM since only qualified psychologists can access the PAI manual but the public can not.

One may argue that an examinee has no need to access the PAI manual because the examinee can request that the examinee’s PAI data and materials be sent to another psychologist for a review or the examinee can apply to the Court for access to the raw scores, interpretive reports and even the PAI manual. This argument ignores the fact that neither option is accessible to ordinary people who do not have extra $5,000 to $10,000 to hire another psychologist or a lawyer to represent them in the court system to obtain access to their test scores, interpretive reports and the test manual.

PAI multivariate base rates of elevated/abnormal scores have not been reported

The PAI manual, the PAI computerized interpretive reports, and the PAI training materials are also silent on the problem of multivariate base rates of abnormal scores, that is, percentage of examinees within the normative samples with one or more elevated scores across 22 primary scales. When healthy normal people complete a battery of tests or a test with multiple measures such as the PAI, a large proportion of them will obtain one or more “abnormal” or “elevated” scores. When one focuses on one measure or one scale in isolation, a cutoff for elevated scores set at 60 T will identify approximately 16% of normative sample as having elevated scores. However, when one applies this same cut off to 22 primary scales simultaneously, a large proportion of the normative sample will have one or more elevated scores. If one rolls a die once, a probability of getting 1 is only about .17. However, more times one rolls a die, greater the probability that one rolls 1 at least once. Rolling a die 22 times is likely to result in at least one 1 with probability nearly 1.00. Accordingly, when a clinical psychologist administers the PAI and looks for elevated scores, the clinician is likely to find at least one elevated score among the 22 primary scales. To put it bluntly, an ignorant or unscrupulous forensic psychologist has 22 ways to find some problematic psychopathology in each examinee based in part on random chance due to low reliability of the PAI scales and large number of measures considered. Moreover, the clinician will find more elevated scores the lower “elevation” cut off the clinician chooses.

Although there appears to be no awareness of this problem in the personality assessment literature, the problem of the multivariate base rates has been recognized, acknowledged, and widely discussed in intelligence testing for decades (Crawford, Garthwaite & Gault, 2007; Karr et al., 2017; Karr et al., 2018; Karr et al., 2020). For example, in one of the extensive reviews of the problems of multivariate base rates, Brooks, Iverson & Holdnack (2013) identified five key psychometric principles that need to be considered when interpreting multiple test scores on intelligence tests. The first three of those principles apply, by definition, to any kind of assessments including personality assessments: (1) abnormal (e.g., low or high) scores are common across multiple measures, (2) the number of abnormal scores depend on the cutoff score used, (3) the number of abnormal scores increases with the number of scores considered.

Our research shows that using liberal cutoff for abnormality (60T or 84th percentile) and outdated obsolete US 1990 norms identifies nearly all of our students (95%) as “abnormal”, having one or more elevated scores across the 22 primary scales. Similarly, using the same liberal cutoff for abnormality (60T or 84th percentile) with T scores calculated from within our Canadian sample (i.e., not relative to the US norms) for each of the 22 PAI primary scales, only 10.3% of our sample showed no elevated scores, nearly 90% had one or more “abnormal” scores, and over 1/2 had 3 or more abnormal scores. Thus, the presence of a number of elevated scores is normal and not indicative of any kind of abnormality. Moreover, when abnormal scores are defined by deviation from the average, presence of abnormal scores across multiple measures is mathematical necessity. Accordingly, any interpretation of abnormal scores must take into account prevalence of abnormal scores in normal healthy normative samples. Clinicians who do not consider multivariate base rates are likely to misclassify large number of examinees as having some kind of psychopathology when they do not.

First, do no harm! Ethics of using outdated, obsolete and invalid tests

The Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association & National Council on Measurement in Education, 2014) are clear that it is test users’ responsibility to periodically verify that continued use of tests, norms, and data is still appropriate, and explains in a commentary why this periodic verification is necessary. The Standard 9.7 states:

Test users should verify periodically that their interpretations of test data continue to be appropriate, given any significant changes in the population of test takers, the mode(s) of test administration, or the purposes in testing.

Comment: Over time, a gradual change in the characteristics of an examinee population may significantly affect the accuracy of inferences drawn from group averages. Modifications in test administration in response to unforeseen circumstances also may affect interpretations.

The American Psychological Association’s Ethical Principles of Psychologists and Code of Conduct (American Psychological Association, 2017) includes several relevant principles that bar psychologists from using outdated and obsolete tests and from using tests whose norms, validity and reliability were not established for the target population. The Principle 9.08 states:

9.08 Obsolete Tests and Outdated Test Results (a) Psychologists do not base their assessment or intervention decisions or recommendations on data or test results that are outdated for the current purpose. (b) Psychologists do not base such decisions or recommendations on tests and measures that are obsolete and not useful for the current purpose. The principle 9.02 (a) and (b) states:

9.02 Use of Assessments (a) Psychologists administer, adapt, score, interpret, or use assessment techniques, interviews, tests, or instruments in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques. (b) Psychologists use assessment instruments whose validity and reliability have been established for use with members of the population tested. When such validity or reliability has not been established, psychologists describe the strengths and limitations of test results and interpretation.

Unfortunately, the APA’s and other code of ethics provide no guidelines on how to determine whether tests“are obsolete and not useful for the current purpose”; it presumes psychologists know that tests are obsolete when, for example, norms are no longer valid, test items are outdated, and continued use of tests would lead to incorrect decisions and misdiagnoses. The International Test Commission’s Guidelines for Practitioner Use of Test Revision, Obsolete Tests, and Test Disposal (International Test Commission, 2015) come closest in defining test obsolescence, at least “generally speaking”:

... Generally speaking, a test is obsolete when its underlying theory, item content, norms [emphasis added], or technical adequacy no longer meet the needs for its intended purpose, professional standards, or when its continued use would lead to inappropriate or inaccurate decisions or diagnoses.

In contrast, other standards and ethics codes are silent and do not even mention the problems of outdated and obsolete tests. Moreover, more general dicta that psychologists ought to be familiar with and rely on current science are often ignored, not only by psychologists but also by some of the regulatory bodies, members of the Association of State and Provincial Psychology Boards (ASPPB) (Uttl, Violo & Gibson, 2024; Uttl, Sikma & Tat, 2025). For example, the College of Alberta Psychologists (http://www.cap.ab.ca/) considers use of even 80 year old tests, norms, and data as well as comparison of an examinee’s scores on a new edition of the test (WAIS-IV CDN) (Wechsler, 2008) to norms for the earlier (53 years earlier) edition of the same named test (WAIS) (Wechsler, 1955) as meeting their minimum professional standards (Uttl, Violo & Gibson, 2024), even though such uses amount to the practise of junk science by psychologists (Lilienfeld, Lynn & Lohr, 2014; Uttl, Sikma & Tat, 2025). In contrast, the North Carolina Psychology Board considered such uses inappropriate and unprofessional conduct, with an appropriate penalty requiring a psychologist to relinquish their license to practice (North Carolina Psychology Board, 2022a; North Carolina Psychology Board, 2008; North Carolina Psychology Board, 2018; North Carolina Psychology Board, 2022b). The North Carolina Psychology Board even issued an official advisory on “Using Current Tests and Norms” (North Carolina Psychology Board, 2018) and another reminder four years later (North Carolina Psychology Board, 2022b).

Psychologists’ ignorance of general dicta to use current, established science to support their clinical opinions and to avoid obsolete and outdated tests, norms, and data has been causing widespread harm to examinees and to the public who incorrectly believe that professionals use current scientifically based methods to assess personality and other psychological constructs (Uttl, Violo & Gibson, 2024). Examinees, readers of psychological assessment reports, adjudicators, judges, and the public have no idea that the regulated professionals are routinely using decades outdated, obsolete, and irrelevant tests, norms, and data even in high stakes assessments, and that at least some psychology boards condone such unscientific practices and consider them fine minimally competent conduct (Uttl, Violo & Gibson, 2024; Uttl, Sikma & Tat, 2025).

With respect to the PAI (and other widely used psychological tests), the problem is compounded by failure to recognize low reliabilities of the PAI scales and subscales, failure to present 95% confidence intervals for the PAI scores, failure to advise users on the folly of interpreting obtained scores as if they were true scores, and failure to take into account multivariate base rates of low scores.

Various reviews of the PAI, often authored or co-authored by researchers with financial stakes and other conflicts of interest in the PAI’s sales and success, continue to exalt the PAI as a modern reliable and validated instrument widely used in forensic and other settings. These same reviewers do not mention massive shifts in what is normal today vs. what was normal 35 years ago and captured at that time by the US outdated and obsolete norms. For example, Paulino and his colleagues (Paulino et al., 2024b) exalted the qualities of the PAI in various superlatives. To illustrate, the first sentences of paragraphs in their discussion section start as follows: “The PAI is a comprehensive, objective, and direct measure that allows for the evaluation of several characteristics of personality with special utility in forensic settings...”, “Its applicability in offender and inmate populations is of particular interest...”, “In other forensic settings, such as intimate partner violence cases, it is also a powerful tool...”, “The PAI is also a useful measure in family law cases, where it is mainly used in the evaluation of positive distortion responding...”, “The PAI is a good predictor of misconduct, insubordination, and abuse of power in police officers...”, and the final paragraph reads: “Overall, the utility of the PAI in forensic settings is generally accepted and its growing recognition is evident in the exponential dissemination to other countries that, in the last couple of years, have adapted it to their populations and forensic realities.”

However, Paulino et al. (2024b) did not mention low reliabilities, did not mention the wide confidence intervals for the PAI scales including the PIM scale, did not mention the norms are 35 years outdated and obsolete, and did not mention any of the research demonstrating that the normative responses have shifted substantially in the US as well as elsewhere. This blindness to the PAI’s fatal flaws may be explained by the competing interests of several of the authors. The declaration of competing interest states that “Dr. Edens is the lead author of the Personality Assessment Inventory Interpretive Report for Correctional Settings (PAI-CS) and receives royalties from its publisher, Psychological Assessment Resources, Inc... The remaining authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.” (p. 8). However, the Conflict of Interest declaration in a different article (Paulino et al., 2024a) states that Paulino, Moniz, Moura, Rijo, Simoes (i.e., every author except Leslie Morey) are “part of the team responsible for adapting the Personality Assessment Inventory, in Portugal” conducted by the PAI’s Portugal’s publisher Hogrefe Publishing “which approved the current adaptation study (e.g., translation, back translation, preliminary version, and normative data).” It appears that, contrary to the declaration in Paulino, Edens at al., the authors (a) have had “personal relationships” with their co-author, Leslie Morey, who “is the author of the Personality Assessment Inventory and receives royalties from its sale.” (p. 14), and (b) were part of the team adapting the PAI to a Portuguese population.

Limitations

Our data examined the PAI performance of university students only and in one Canadian university only. However, our data are consistent with the large shifts in the PAI performance of the US college students, and these changes are even more pronounced in our sample. Accordingly, although it is clear that the US norms (Morey, 1991) are outdated, obsolete, irrelevant, and clearly inappropriate for use with both the US and Canadian university students today, our data do not constitute new norms for a non-clinical Canadian general population nor for clinical Canadian samples. Our data can be used as norms only for Canadian university students and with awareness and explicit acknowledgement that (a) our data were obtained only in one university and (b) various scales and indexes may no longer be valid measures of constructs they were intended to measure since we have not examined the validity of the PAI scales.

In 1990, the US college students norms were similar to the US census matched norms, especially to the US census matched young adults, but it is unclear whether this similarity continued and whether this similarity would be observed in Canada. Comparison of Nails et al. (2023) and McCredie & Morey’s (2019) data suggest that both the US university students and the US community adult samples score much higher than the PAI 1990 US norms, and that the PAI US norms are outdated, obsolete and invalid for both the college and the census matched samples. However, no comparable studies exist in Canada and it is unclear whether Canadian norms for the university students would differ from those for normal adults. Furthermore, it is unclear whether the PAI scales, subscales, and various supplementary indexes developed using data gathered decades ago are still valid measures of what they were intended to measure.

In any case, the PAI 1990 US norms ought not to be used with any population until researchers establish current norms that accurately describe what is normal in the current US as well as the Canadian population. Comparison between Nails et al. (2023) and our data suggests that applicability of the US data to the Canadian populations cannot be assumed but must be empirically established.

Recommendations

The problem of psychologists using outdated and obsolete tests and norms as well as norms developed in another country or culture is not limited to the PAI; it is widespread (Uttl, Sikma & Tat, 2025). For example, an alternative to the PAI, Minnesota Multiphasic Personality Inventory- 3 (MMPI-3) also suffers from lack of Canadian norms, validation in Canada, as well as many other issues (e.g., low reliabilities of validity scales, inadequate reliabilities of substantive scales with 95% confidence intervals for most of the scales spanning approximately 20T scores), although its US norms are more recent, collected in 2017 to 2018, in pre-COVID-19 era (Ben-Porath & Tellegen, 2020). As another example, the D-KEFS (Delis-Kaplan Executive Function System) popular test battery at least among neuropsychologists was normed on US sample sometime prior to 2001 (the test manual is silent on when the normative sample was actually tested), that is, nearly 25 years ago or more (Delis, Kaplan & Kramer, 2001). Just like the PAI, the D-KEFS has never been normed on Canadians. Moreover, the D-KEFS has been heavily criticized for decades for having substandard inadequate reliabilities ranging from .13 (Design Fluency Test, Condition 3: Correct Designs Switching, Ages 8–19, p. 26) and with reliabilities .90 or higher being rare (Crawford, Sutherland & Garthwaite, 2008; Suchy & Brothers, 2022). Yet, Monette et al.’s (2023) survey shows that the D-KEFS is used by 82.6% of Canadian neuropsychologists even though it is obsolete, developed and normed in another country, and lacks reliabilities necessary to support clinical decisions. More generally, Monette et al. (2023) also reported that 80% of Canadian neuropsychologists responded that they use “normative data from another country”, 55.5% responded that one of the challenges was that tests “lack normative data for my country”, 29.1% responded that tests were “not adapted to my cultural setting”, and 28.3 noted that tests “do not have good psychometric properties.”

What can be done to ensure that psychologists do not use outdated obsolete tests and norms? One of the obvious recommendations arising from ours as well as from the previous studies with the US adult and US university students is for the author and the publisher of the PAI to withdraw the PAI and to issue the public advisory that the PAI test score interpretations are no longer valid with neither US nor Canadian populations. According to the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association & National Council on Measurement in Education, 2014) these are test authors’ and publishers’ obligations. For example, the Standard 4.24:

Test specifications should be amended or revised when new research data, significant changes in the domain represented, or newly recommended conditions of test use may reduce the validity of test score interpretations. Although a test that remains useful need not be withdrawn or revised simply because of the passage of time, test developers and test publishers are responsible for monitoring changing conditions and for amending, revising, or withdrawing the test as indicated.

However, despite the published research (some co-authored by the PAI author himself) showing that both the US adult and US college norms are outdated, obsolete, and no longer valid (Maffly-Kipp & Morey, 2024; McCredie & Morey, 2019; Nails et al., 2023), Morey and the test publisher continue to inform the public and psychologists that the PAI 1990 adult norms continue to be valid. For example, Morey (2021) informed attendees of a workshop at the 2021 meeting of the Society for Personality Assessment that the 1990 norms were “pretty much holding up”; Morey stated:

The PAI, you should know this but make sure you do, the PAI is standardized using a linear transformation to have a mean of 50 and a standard deviation of 10 in the US community sample that was basically, um, census matched on age, race, and gender, um, when the PAI was developed. And, uh, ah, by the way, we’ve, we’ve been continuing to do research on those norms, including looking at, uh, online samples using Amazon MTurk and things like that and published, um, recently that really suggests that those norms are pretty much holding up. It’s not as if the US is really changing, um, with respect to a lot of the questions the PAI is asking. Those norms are holding up pretty well.

The four hour long recordings of Morey’s workshop has been posted on the PAR Inc. (parinc.com) website on August 23, 2022 as one of the PAI training materials accessible to psychologists with the PAR Inc. account (Morey, 2021). It appears that the PAI author and the publisher continue to believe that the US 1990 adult/community norms continue to be valid, and that today’s US adults are responding the same way on the PAI even though this belief is objectively false and demonstrated to be false by the PAI’s author’s own research and numerous other studies. This belief may not be surprising given that test publishers are corporations established to make profits and that test authors collect royalties from tests and related merchandise sales. Accordingly, test authors and test publishers are unlikely to periodically re-examine continued reliability, validity, and currency of their tests, and to renorm them unless they are forced to do so by external forces, for example, overwhelming research demonstrations that their tests are no longer valid for their stated purposes, test users refusals to buy and to use their outdated and obsolete tests, and test information consumers (e.g., employers, courts) refusing to consider the PAI assessment results.

Although ethics codes make test users, for example, clinical, vocational, and forensic psychologists, ultimately responsible for using current, reliable, and valid tests, test users often obtain their information about tests from their graduate school training, test manuals, and from test publishers’ sponsored talks, training videos, and webinars such as the one noted above. Most clinical psychologists do not do relevant research and are unlikely to go and read primary sources (often published behind paywalls) and examine scientific evidence of test norm currency and test reliability and validity for themselves. One way forward would be to require all clinical psychologists who wish to conduct formal assessment using psychological tests to be periodically required to pass an exam on reliability, validity, currency and interpretation of psychological tests including exams on reliability, validity and currency of specific tests such as the PAI. Such requirements would likely be very unpopular among clinical psychologists and other test users.

Yet, importantly, clinical psychologists have a duty to obtain “informed consent” for assessment from all examinees. As detailed in one ethics code, Canadian Code of Ethics for Psychologists, this requires psychologists to do the following (Principle I.23) (Canadian Psychological Association, 2017):

Provide, in obtaining informed consent, as much information as reasonable or prudent individuals and groups .... would want to know before making a decision or consenting to the activity. Typically,…, this would include: purpose and nature of the activity; ... likely risks and benefits of the activity...

Thus, an informed consent form for administration of the PAI to Canadian adult as part of a fitness for duty assessment ought to disclose to an examinee that they would be tested with the 35 years old tests, that their scores will be compared to scores of examinees tested in a different country 35 years ago, that a psychologist’s opinion of the examinee’s personality based on the PAI will have no scientific basis, and that one risk of this activity is that the examinee may lose their job for no reason whatsoever. We propose the following text to be included in an informed consent for assessment of adults in Canada with the PAI at this time:

I will administer to you a personality questionnaire called the Personality Assessment Inventory (Morey, 1991; Morey, 2007). To interpret your scores, I will compare them to scores of US examinees tested 35 years ago in several selected US states because no one has established how Canadians such as yourself score on this test. To be clear, although I will use performance of US examinees 35 years ago, I have no scientific evidence how people such as yourself score on this test today here in Canada. Accordingly, my opinions about your personality will have no scientific basis but they may be used by your employer to terminate your employment. Please initial here __ to indicate your informed consent for me to interpret your scores using the 35 years old test norms from a different country.

We believe that no adult examinee competent to give informed consent would consent to such an unscientific assessment if these facts were disclosed to them in the informed consent form prior to the assessment. Ethics codes and standards of practice ought to make these disclosures—tests to be used, their age, origins of test norms, etc.—mandatory, in writing, as part of informed consent process. Examinees have a right to know this essential information prior to proceeding with the assessment.

Another recommendation is for the regulatory bodies established to protect the public (in the US and Canada provincial and state psychology boards) to do what the test publishers ought to do and to set up standing committees to be “responsible for monitoring changing conditions” and issuing the public advisories on currency, reliability, and validity of various psychological tests, especially on tests most frequently used in high stakes decisions such as employment selection, fitness for duty assessment and child custody evaluations. The problem with this recommendation is that the regulatory bodies are not necessarily any more knowledgeable than an average clinical psychologist who obtains their knowledge about tests’ currency, reliability, validity, and wider scientific underpinning, if any, from test publishers and test publishers’ sponsored talks, training videos, and webinars such as the one the quote about the PAI’s 1990 adult norms continued validity comes from. As detailed above, the College of Alberta Psychologists considers use of even 80 years old intelligence tests, norms, and data at least minimally competent practice despite massive amounts of research evidence (some summarized in test manuals themselves) that such use of 80 years obsolete intelligence tests and data amounts to practice of junk science (Uttl, Violo & Gibson, 2024).

Apparently, some psychologists believe, incorrectly, that tests and norms are valid forever or until someone demonstrates that the tests and norms are invalid. For example, Russell (2010, p. 66) wrote:

... neuropsychology should recognize that the research, including norming, based on a validated test or battery remains valid until it has been demonstrated to be invalid. Regardless of how long ago a validated test was published, its results are still sound unless research has demonstrated its lack of validity.

This view is wrong and misguided (Uttl, Sikma & Tat, 2025); it assumes that the world, populations, cultures, education, technology, climate, etc. are not changing but remain static. Since the world, population, cultures, etc. change, and sometimes very rapidly (e.g., COVID-19 pandemic), one cannot assume that tests and norms remain valid but one has to assume that tests and norms become invalid not because of simple passage of time per se but because of the changes in the world, populations, etc. that occur with time. Thus, reliability, validity, and currency of tests and norms have to be periodically re-examined to determine whether or not changes in the world, populations, etc. changed how current samples from those populations perform on psychological tests including how they answer questions on personality tests such as the PAI today vs. 35 years ago, or before vs. after extraordinary events such as pandemics.

Critically, reliability, validity, and currency of psychological tests needs to be periodically verified and re-established by researchers with no conflict of interest, and no financial and reputational benefits tied to outcomes of such research. Any such research ought to be transparent, published in open access journals, published with accompanying data, and widely available to the public including to examinees affected by use of these tests, their lawyers, and other consumers of the PAI assessments and opinions based on those assessments. Establishing reliability, validity, and currency of psychological tests in original as well as new contexts (e.g., countries, cultures, situations) are projects imminently suitable for undergraduate honours as well as Master’s theses projects. When such research yields evidence of poor reliability, lack of validity, and/or shifting norms, it would be incumbent on researchers to notify regulatory bodies as well as provincial, state, and national psychological associations to issue advisories to its members alerting them to new data on reliability, validity, and currency of such psychologist tests and norms. If the College of Alberta Psychologists’ acceptance of 80 years obsolete tests and norms as minimally competent ethical practice is any guideline, regulatory bodies and their members themselves may not become aware of outdated nature of many tests and norms as well as resulting harm for centuries to come if researchers themselves do not take time to alert them.

Finally, as a rule of thumb, we suggest that if a test or test norms are older than 10 years, test users ought to investigate scientific evidence of their continued reliability, validity and currency prior to using the test and/or norms. If the users cannot find and cite scientific evidence of continued reliability, validity and currency, the users ought to stop using the test and norms to avoid harming examinees and society, especially in high stakes, life altering assessments. We note that a widespread use of a test and norms is not the scientific evidence of the test and norms’ reliability, validity and currency. Only scientifically sound research studies, with adequate samples representing populations of interest, can establish continued reliability, validity and currency of tests and norms in particular contexts (e.g., Canada vs. US, younger vs. older examinees).