35 Years Revisited

35 Years Revisited: Conceptual Errors in Scientific Inquiry, a Case Study

Richard M. Gray

School of Criminal Justice and Legal Studies

Fairleigh Dickinson University

Excerpted for the NLP Wiki group largely Gray, R., Liotta, R., Wake, L. & Cheal, J. (2012). Research and the History of Methodological Flaws. In Lisa Wake, Richard Gray & Frank Bourke (Eds.), The Clinical Efficacy of NLP: A critical appraisal (194-216). London, Routledge.

An example of how misrepresentation occurs comes from the oft repeated story of Little Albert and John Watson. This narrative is derived from Harris (1979).

In 1920, Watson and Rayner reported a conditioning experiment with “Little Albert”. The experiment was designed to determine whether a child who was unafraid of small animals could be conditioned to fear a white rat when its presence was followed by the loud startling sound of a metal pipe being hit with a hammer and whether that fear would generalize to other stimuli. The original literature gives a very clear and concise description of the process that Watson and Rayner followed. The descriptions included the specifics of the items that Albert was placed in contact with, the length of time between the stimulus presentations and follow-up regarding Albert’s long-term adjustment.

Harris reports that a survey of Introductory Psychology texts found that a majority misrepresented the experiment on some level. Some texts exhibited minor errors including the child’s name, and age and whether he was initially conditioned to a rat or a rabbit. More significantly, many texts claimed that the fear generalized to a fur pelt, a man’s beard, a cat, a pup, a fur muff, a white furry glove, various relatives who wore fur, and a teddy bear. Some failed to include a second conditioning event that was part of the original experiment. Yet other texts indicated that at the end of the experiment Watson extinguished the fear responses when, in fact, Albert was removed from the hospital where the experiments were performed and was never seen again (Harris, 1979).

Harris supposes that there are two main reasons for these inaccuracies, a desire to portray Watson positively and, more importantly for our purposes, an over reliance on secondary texts. A further implication of Harris’ observations is that these errors are promulgated as authors continue to rely on secondary sources without checking their accuracy.

In the case of NLP, while there have been ill-intentioned popular reviews, many of the myths have been perpetuated by the continual reliance on a series of ill-informed studies that proceeded on the belief that the preferred representation system (PRS) was some kind of theoretical foundation upon which the rest of NLP depended. Anyone who has read the literature beyond a few early texts (Bandler & Grinder 1975, 1979; Lewis & Pucelik, 1980), or who had carefully read the early texts for themselves, would have discovered that the concept was more narrowly conceived than most researchers have understood and would also discover that it fell quickly from use. Insofar as the PRS represented an observation whose currency in NLP was sufficiently brief that Connierae and Steve Andreas never included it in their authoritative descriptions of the field, the premise of most of these studies was spurious. Nevertheless, because of their presence in the peer reviewed literature they continue to exert an inordinate influence on researchers who are otherwise ignorant of the field. Despite a fairly steady stream of research that supports many of the basic concepts of NLP, researchers return to the flawed data from 1980s. The root sources of the supposed experimental refutation of NLP come from two reviews by one author (Andreas, C. & Andreas, S., 1989; Andreas, S., & Andreas, C., 1987; Gray, Wake, Andreas, & Bolsted, 2012)

Examination of Reviews

In 1984, after noting the repeated appearance of NLP in popular literature, Sharpley reviewed the extant research on what he understood to be one of the central tenets of NLP, the preferred representational system (PRS). According to Sharpley (1984), the PRS, the idea that everyone encounters the world mainly through one preferred sensory system–usually visual, auditory or kinaesthetic–was the key to understanding NLP. It was his understanding that accurately evaluating and making use of this preference was foundational to all that NLP had to offer. This assessment could be done by examining eye accessing cues (EACs) and descriptive predicates (Sharpley, 1984).

Sharpley evaluated 15 published studies focusing on this phenomenon for design, method and dependent measures. He divided the studies into four categories: those that tested for the presence of the PRS as an actual phenomenon, those examining the validity of the construct using multiple measures, those examining the use of PRS matching in non-counseling situations and those testing whether PRS matching is effective in counseling situations. He concluded, accurately—in harmony with the main point made by NLP regarding eye accessing cues (EACs)–that matching the dynamic range of predicate responses in a clinical conversation is valuable for enhancing client empathy. Because, however, he had gone in search of the elusive or non-existent PRS with the assumption that it is somehow foundational to NLP, he rejected (correctly) the dynamic use of EACs as not relevant to the PRS issue and warned that the research that he had reviewed did not support the existence of the PRS.

This was a reasonable analysis of the research to date (1985). From the perspective of NLP, however, it highlights several serious problems with that research. These problems are: the assumed ubiquity of the PRS; its presumed centrality to NLP; the presumption that the indicators, eye accessing cues and predicates, always agree as to the conscious perceptual system; and that rapport implies something other than a set of observable changes in interpersonal responses. Having already examined the first two problems, the focus now turns to the others.

Sharpley, and most of the studies that he examined, appears to have believed that the analysis of sensory modalities is a reductive exercise. He surmised that it is related to one or two indicators (EACs and predicate usage) and that those elements will reveal the active sensory modality as either independent or co-varying dependent variables. Sharpley and many of the authors that he cites also believed that the sensory modality revealed by these cues must necessarily arise on demand. That is, if we instruct an individual to think about a sound or a picture, they must immediately and singularly access the auditory, visual or kinaesthetic centers in the brain and that this access must be reflected in the predicates or the eye accessing cues or both. What this assumption misses is that the NLP sensory modalities are, for the most part, not demand characteristics. This means that in their most salient aspects, their identification in predicates or eye movements, they are not simply evoked by instruction but they are observed in the course of naturalistic conversation. Moreover, rather than being an absolutely invariant reflection of neural processing that must also be observable under easily specifiable circumstances, they are more often highlighted against the background of or inferred from an entire gestalt that includes breathing rate, body posture, speaking tone, speech rate and the sensory context. This gestalt provides a perceptual frame that allows the trained practitioner to discern what the simple procedure cannot evoke. Beyond this, the ‘simple’ EACs are complicated by multiple levels of concurrent eye movements.

Bandler and Grinder (1979) point to the currently active representational system as being identified with a pattern that recurs more often than other concurrent eye movements. They suggest that the proper way of learning how to evaluate representational systems is by asking multiple questions, one system at a time, and noticing which pattern appears most frequently. They also indicate that these vary as specific personal patterns and therefore must be detected by observing each person as an individual.

Complicating the issue is their statement that eye accessing cues reflect the entire process of making the content conscious. This typically involves multiple systems. They note:

We’ve got to make a distinction now. The predicates, the words a person chooses to describe their situation—when they are specified by representational system—let you know what their consciousness is. The predicates indicate what portion of this complex internal cognitive process they bring into awareness. The visual accessing cues, eye-scanning patterns, will tell you literally the whole sequence of accessing, which we call a strategy. What we call the “lead system” is the system that you use to go after some information. The “representational system” is what’s in consciousness, indicated by predicates. The “reference system” is how you decide whether what you now know—having already accessed it and knowing it in conscious­ness—is true or not. (1979, p.28)

This means that, inter alia, eye accessing patterns and predicates will only match some of the time. So, the lead system need not match the EACs as it may reflect unconscious process. After the perception enters consciousness, it may be represented by a different system which is used to process conscious content.

Further, Bandler and Grinder emphasize the contextual nature of any such determination in the following passage:

Our claim is that you are using all systems all the time. In a particular context you will be aware of one system more than another. I assume that when you play athletics or make love, you have a lot of kinesthetic sensitivity. When you are reading or watching a movie, you have a lot of visual consciousness. You can shift from one to the other. There are contextual markers that allow you to shift from one strategy to another and use different sequences. (p.36)

The other conceptual error by Sharpley and the studies he examines is the definition of rapport against which various tests of the PRS and EACs are made. As is often done in poorly designed research, the term is defined using a standard definition, a dictionary definition or a definition that is current in the researcher’s sub-discipline. For the most part this leads to false negatives. Unless the phenomenon under question is tested as defined by its proponents, the test is invalid. In general, the tests of rapport, except those that define it in terms of increased empathy are bound to fail. These kinds of errors are just those suggested by Ioannides (2005) and documented by Martinson, Anderson and Devries (2005).

These consistent errors regarding the overemphasis on the PRS appears to come from readings of the early chapters of The Structure of Magic, Volume II (1976). According to Andreas (Gray, Wake, et al., 2012).), who was there, the PRS was introduced as a teaching tool. This statement becomes obvious and makes sense with a close reading of Magic II.

In the first several sections of the text, Grinder and Bandler do indeed talk about the utility of the PRS as a means of establishing rapport and as an important tool in therapy. They talk about how all of us have the capacity to experience the world in terms of all five senses and that for each of us one or more of these is more fully developed or preferred than the others. This is described as a tendency; it is an observed pattern, not a truth. What often passes unnoticed in this discussion is that it appears to progress from the description of a general tendency to the discussion of a problem and it ends with several means of solving the problem. The early discussion of detecting the PRS develops into a discussion of how the highly preferred sensory system can limit peoples’ maps of the world, impair their capacity to communicate with others and impoverish their capacity to enjoy the full range of human experience. In such cases, the therapist first paces the limitation and then seeks ways to open the client to new possibilities of perception and action. They note:

As we repeatedly pointed out in Magic I, when people come to us in therapy with pain, feeling that they are stuck in that they don’t have enough choices, we find that their world is rich and varied enough for them to get what they want, but that the way which they use to represent the world to themselves is not rich and varied enough for them to obtain it. In other words, the way that each of us represents our experience will either cause us pain or allow us an exciting, living and growing process in our lives. More specifically, if we choose (consciously or not) to represent certain kinds of experience in one or another of our representational systems, we will succeed either in causing ourselves pain or in giving ourselves new choices. (P.28)

Other than in this therapeutic context, as Andreas notes, the PRS was introduced as a means of focusing attention on representational systems more generally.

Despite these errors, Sharpley is perceptive and often gets it right. Constrained, however, by the hypotheses he is testing, he has to pass the observations by. One of the more striking and accurate statements that he makes is that

However, the identification of this PRS (if it is a PRS and not merely current language style) by either eye movements or self-report is not supported by the research data. The cuing effect of client verbalizations is valuable, not to identify PRS but to alert counselors to phrase their responses in such a way as to maximize empathy within the interview. The existence or stability of the PRS is irrelevant to predicate matching as a counseling process, and parsimony argues for the process rather than the as yet unverified theory. (p. 247)

Had it not been for his early preconceptions about the PRS as central to the technique and had he not taken his study-authors at their word, this might have been a much more valuable paper. Somewhat earlier, he says, correctly, that:

… if NLP is suggesting that counselors who demonstrate high levels of reflection and empathy will be more effective than those who do not, then little new is being said. If NLP seeks to promote empathic responses from counselors, then scales designed to measure empathy ought to, and do, show this (e.g., Hammer, 1983). Although this is a worthwhile procedure for counselors, it does not justify NLP as a separate theoretical position (nor as the “magic” its proponents quote). (p.246)

This appears to be another important part of the problem. NLP clearly indicates that its aim is to discover patterns that provide results in treatment that have been part of the characteristic repertoires first of the exemplars that they modelled and then others. It did not claim that these elements of technique in and of themselves would be transformative—they were a set of techniques that could be used for various purposes. There was no claim that its techniques were, ‘something new’ or that it was espousing a specific theory (Bandler & Grinder, 1975, 1979; Dilts, Grinder, Bandler, Bandler, & Delozier, 1980).

Sharpley is often unfairly criticized for taking a position on NLP that was wrong. This is done from a perspective many years in the future when several consensus models of NLP have arisen, none of which hold forth the proposition that either the PRS or EACs are central tenets upon which NLP must stand or fall. It should further be recognized that Sharpley depended upon his sources, which, preceding him, made their own false assumptions about NLP. The problem is not Sharpley, but the uncritical acceptance of his findings by researchers who have not examined the status of NLP since the mid-1980s. As Bradbury has rightly observed: all roads point to Sharpley (Bradbury, 2011a, 2011b).

Shortly after Sharpley’s publication, Einspruch and Forman (1985) published their reply: Observations concerning research literature on neuro-linguistic programming. These authors systematically criticized the 15 studies used by Sharpley and extended their analysis to another 24 not reviewed by Sharpley. They began by pointing out the errors concerning NLP made by Sharpley and his authors. Among the errors noted were the following:

  • They ignored the models specified by NLP for defining patterns (and their highly individualistic nature) and the nature of therapeutic communication and interventions.
  • There were failures to recognize the impact of context and that information may coexist in multiple sensory systems, on conscious and unconscious levels simultaneously.
  • They focused upon the PRS and reified it as discrete object rather than a partial expression of an ongoing process.
  • They mistakenly assume that both the PRS and representational systems more generally only apply to right handed people.
  • Contrary to the representations made in the literature of NLP, they identify matching the PRS as the key to effective counseling.
  • None of the studies took into account the Meta Model of communication as a central means for parsing conscious and unconscious process.

Procedural errors detailed by Einspruch and Forman included a lack of adequate training in the NLP techniques being tested and a failure to understand the NLP position that words are conditioned stimuli that evoke multi-level responses.

It should be remembered that patterns, as understood in NLP have a special status (Grinder & Bandler, 1980). Patterns involve the perspective of the observer and other participants. They are dynamic sequences of perceptions and actions that are identified by the flow of sensory information through the procedure. As a result, no pattern has a single, static formulation. Like statements in mathematics and symbolic logic, they begin with a set of common elements (VAKOG and submodalities). These individual elements may vary in order and value but the more general pattern of their interactions remains. Again, Grinder and Bandler make the following clarification:

Notice that since patterns must be represented in sensory grounded terms, available through practice to the user, a pattern will typically have multiple representation[s]—each tailored for the differing sensory capabilities of individual users. I point out in passing that this requirement immediately excludes statistical statements about patterning as being well-formed in NLP as statistical statements are not user oriented (1980a, p.6.).

In the end, Einspruch and Forman make several recommendations for further research. These include that researchers studying NLP should be trained by competent trainers for an adequate time and to a certain level of competency. Their curriculum should include training in the recognition of patterns and a serious appreciation of the underlying presuppositions of NLP. Only this level of training, they suggested would provide an adequate basis for testing the application of NLP to therapy.

One of the significant problems with many of the studies reviewed by Sharpley was that they attempted to study rapport as a matter of paper-and-pencil tests. These authors suggest that any test of rapport must be made using objective, sensory-based criteria. They also indicated that the procedures tested should be scored individually and later combined for statistical analysis. Finally they recommended that tests of therapeutic procedures, whether NLP-based or in a comparison treatment, should only be performed by thoroughly trained therapists who have illustrated mastery of the procedures.

In 1987, Sharpley responded to Einspruch and Forman in the Journal of Counseling Psychology. Echoing Einspruch and Forman’s critique of the methodology used in the studies to date, he complains that: “There is little more that a researcher can do, however, to evaluate a theory than to test the veracity and strength of those principles of behavior that are held by the proponents of that theory“ (1987, p.103). Yet it is precisely here where the studies fail. Because the researchers lacked adequate training in NLP they misunderstood the nature of the PRS and EACs as patterns and overvalued their importance to NLP.

Sharpley then seeks to correct Einspruch and Forman for critiquing the studies that claimed that eye accessing cues and the elusive PRS only work for persons who are right handed. In his earlier paper, Sharpley (1984) claimed that all of the patterns only apply to right handed persons. In fact it is only the left-right distribution of the EACs to which this caution was applied (Bandler and Grinder, 1979). In a further indication of the superficiality of his reading, Sharpley points to the Eye Acessing Cues diagram (the NLP homunculus) on page 29 of Frogs as applying only to the PRS. The most casual reading of the passage, however, places the chart in the context of an exercise identifying EACs in a general sense, not the PRS.

Two other critiques offered by Einspruch and Forman, that the means of accessing the PRS in several studies was inadequate and that the therapists used were inadequately trained are actually closely related. As already noted, EACs and the PRS, where present, are patterns. They are not simple responses like raising your right hand but they are relatively complex responses that require some subtlety on the part of the therapist for their discernment. Einspruch and Forman’s critique, that training in pattern recognition, not just watching for eye movements, seems more than reasonable.

Having gone through Einspruch and Forman’s critique of the research, and rejecting it, Sharpley proceeds to show that the vast preponderance of research to that point did not support NLP. As a matter of simple fact, that observation is true—as far as it goes. Importantly, it misses the point that few if any of the studies were reflective of the tenets or practices of NLP. Most of the studies seemed to rely on a superficial reading of the materials and a misunderstanding of the central idea of patterns as noted above. As Einspruch and Forman note, the findings are largely irrelevant.

At last, Sharpley turns to the fact that if NLP has any value, its value is derived from techniques garnered from other sources. In terms of technique, he finally gets close to the point of NLP and its founders. As primarily a technique for modeling excellence, NLP makes no claim to originality in process, only originality in modeling and in making the processes transportable.

In summary, Sharpley’s understanding of NLP is flawed, as were the understandings of the researchers he cites. Had any of them had a more complete understanding of NLP, it might have been valuable research but as the bulk is based on false understandings and total, if innocent, distortions, the conclusions reached are without value.

In 1987, shortly after Sharpley’s (1987) second article, Heap (1987) published his review, covering much the same ground and making all of the same errors. Like Sharpley and his forebears, he takes the PRS as a central tenet of NLP theory and praxis and proceeds to illustrate its near total lack of support. While fairly exhaustive, Heap breaks no new ground and continues to test a false conception of NLP.

This review began with the observation by Harris that psychological research is often flawed by its reliance on inaccurate reports of previous research that become immortalized in the peer reviewed milieu. It also noted how that kind of error gives rise to biased findings (Ioannides, 2005; Simmons, Nelson, & Simonsohn, 2011). We have already seen how Sharpley, as early as 1985, suffered from just this problem in taking at their word the false understandings of his informants; this tendency has been reinforced by Heap.

A significant and more recent case in point is the relatively objective attempt by Witkowski (2010) to assess the state of NLP by examining thirty-five years of published research. But it too suffers from the intellectual game of telephone.

The author presents, as the central concept of NLP, the presupposition that the map is not the territory and that individual perceptions are formulated in terms of personal interpretations of sensory data using the five basic sensory systems (VAKOG). He then states:

As they suggested, each of us processes the majority of information using one primary representational system (PRS). Following the example of the most outstanding therapists, to work effectively with a patient one should necessarily match the patient’s PRS so as to be able to use their “map” (2010, p.4).

Witkowski then asserts, that not only is the PRS central but that a second crucial theoretical element of NLP is the observation that eye accessing cues are used most importantly to access the PRS so as to effectively guide all other interventions.

Another discovery of which the NLP originators were particularly proud of was to realize that access to the representational systems is possible through the so-called accessing cues that is precisely specified eye movements. Careful observation of these movements should enable the NLP therapist to unequivocally identify the PRS of the patient, interlocutor etc., and, in consequence, facilitate matching their PRS. All other hypotheses of the NLP system related to the arising of mental disorders, the type of therapy and communication, etc. stem from these basic assertions. (2010, pp 4-5)

It is important to note that here Witkowski has turned the PRS into a typology, an error that even its earliest promulgators were careful to avoid (Bandler and Grinder 1975, 1979; Lewis & Pucelik, 1980). Einspruch and Forman (1985) characterize this error as a reification of the idea, turning its measurement into a meaningless exercise.

Proceeding from these presuppositions, Witkowski embarks upon a review of the 315 articles published in the Neuro-Linguistic Programming Research Data Base (http://www.nlp.de/cgi-bin/research/nlp-rdb.cgi) which was compiled over a period of 18 years by an international body of NLP researchers. His choice was made to ensure that his own biases did not affect the choice of articles and because he believed that this database would provide a more comprehensive collection of such articles than could be found in a search of PsychLit, PsychInfo, MEDLINE or other such academic collections.

Continuing with his qualitative analysis, Witkowski then limited his analysis to studies that had been published by the most reputable of scientific journals. This was done by excluding any publication that did not appear in the Master Journal List of the Institute for Scientific Information in Philadelphia. Using this filter the list was whittled down to 63, or 20% of the original list.

This remaining 20% was further subdivided into three groups: those testing what he believed to be the basic tenets of NLP or hypotheses derived from those tenets—33 studies; those focused on “polemics, discussions, case analyses” or other elements that he determined were irrelevant to his main research goal—14 studies; and studies that he determined to be irrelevant—16 studies.

Of the original 315 studies, Witkowski only examined the thirty-three from category one. From this sample, he found nine works supporting what he believed to be the central tenets of NLP or hypotheses derived from them; eighteen that were non-supportive and six that were indeterminate.

With the single exception, that his understanding of the main tenets of NLP were wrong, Witkowski’s methodology in reviewing the literature is fairly exemplary, however, he does it from a fatally flawed understanding of the main tenets of NLP; this makes his work invalid. He states criteria to be evaluated, for fairness’ sake he goes to a source that can be construed as being favourable to NLP (the Neuro-Linguistic Programming Research Data Base), evaluates the source journals for their academic integrity by a well-respected source, and then evaluates the remainder for relevance to his research criteria.

On a qualitative level, three of the supportive works were viewed as methodologically acceptable, These were Kinsbourne’s (1974) study of eye movements and brain lateralization, Yapko’s (1981) investigation of the PRS and its relationship to hypnotic depth and Dooley and Farmer’s (1988) study of eye movement differences in aphasic and normal controls. The remaining studies were reasonably adjudged to be of lesser value because they reported general outcomes based on 21–day intensive NLP trainings and/or lacked controls.

Of the eighteen articles reporting non-supportive results Witkowski goes through them systematically. He begins with Thomason, Arbuckle & Cady (1980); Farmer, Rooney & Cunningham (1985), Poffel & Cross (1985) and Burke et al. (2003), all of whom examined the correlation between eye movement and assumed sensory access. Witkowski reports that “They all provided unequivocally negative results” (p. 13).

A review of those studies finds that for the most part there is an assumption that external instructions or specific tasks designed for the purpose will, of necessity, produce the effects observed by NLP. The NLP EACs, as understood by the literature of NLP, are typically reported as expressions of a larger communications gestalt in which they are embedded. Because those negative findings do not reflect the context in which NLP predicts their appearance, the conclusions are dubious at best. Moreover, as no one has ever claimed that these were foundational to NLP in any manner, the research could only possibly impact this one set of observations.

An interesting twist that arises from this first batch of studies is the observation by Burke et al. (2003) that although they could find no evidence of the PRS, there was evidence that eye accessing cues changed with the subject matter. Although they were not testing the actual position of NLP, they appear to have found confirmatory evidence. Two other studies (Gumm, Walker & Day, 1982; Coe & Scharcoff, 1985) that set out to test the presumed validity of the PRS failed to confirm its existence. Witkowski again mistakenly characterizes these studies as not supporting the theory of NLP.

Passing on to more complex research, Witkowski presents the findings of Fromme and Daniell (1984); Elich, Thompson and Miller (1985) and Graunke and Roberts (1985).

Fromme and Daniell (1984) design a set of tasks with which to evaluate the PRS which are so laden with self contradictory language that their results are as unsurprising as their methods naïve. In a series of experiments that depended heavily upon visual information processing, they discovered that none of their predicted relationships held. In general, it may be observed that despite intending to test multiple sensory systems, they succeed in unintentionally reducing every task to a visual task. Needless to say, they found that their research did not support the existence of the PRS and as it was supposedly foundational to the field, they rejected NLP as well.

In reviewing their results, it is once more plain that the theory under examination was wrong: NLP posits no foundational principle regarding the PRS. Reviewing the experimental tasks, it becomes immediately obvious that the root task is a highly complex visual exercise. In each subset of the task one might reasonably expect that the dominant sensory modality must be visual, despite the authors’ efforts to evoke other modalities.

In a subsequent experiment, the authors purportedly determined a PRS for each subject and based on these findings subjected their subjects to two other tests to determine whether fast visualizers would communicate visual information more effectively than slow vizualizers (they did) and whether subjects would choose verbal expressions that matched their supposed PRS (they didn’t). A full methodological critique is presented in Gray, Liotta, Wake, & Cheal (2012).

Witkowski then turns to Elich, Thompson and Miller (1985). From the outset, although the study is presented by Witkowski as a test of the idea that eye movements and spoken predicates reflect the sensory modality of imagery, a review of the article (Elich et al., 1985) finds that the aim of the assessment was to test the PRS hypothesis. The authors sought to determine PRS by matching eye movements with predicates using trained raters (perhaps mis-trained) to assess eye movement recordings and standardized assessments of the sensory modality of the spoken elements of the experiment. The experiment appeared to be well designed despite some serious conceptual and procedural issues (smell is identified as a kinesthetic sense and although the raters were trained to identify EACs, there is little information about how they were trained.). The authors acknowledge that use of the word imagery throughout may have biased the results towards the visual modality and that the difficulty of differentiating between conversational eye fixation and staring as an EAC made scoring problematic. They failed to confirm either the relationship between eye movements and predicates and further found that any such matching did not reliably identify the PRS. What is striking is that although the study fails to confirm that matched eye movements and predicates can validate or reliably assess the PRS (as it should have), the authors provide the following conclusion which is much more in line with the actual position of NLP than the reviewer might have understood:

Much of the problem is with the concept of the PRS. It is pointedly apparent from the research of the last few years that the concept of PRS is as slippery and elusive as a greased pig at a country fair. Dorn, Brunson, and Atwater (1983) and Sharpley (1984) concluded that (a) there is no reliable method for assessing PRS, (b) PRS may change over time, (c) it is not certain that PRS even exists, and (d) if PRS does exist it may merely reflect current language style, and we would add that PRS may be heavily influenced by language. (1985, pp. 624-5)

Witkwoski then examines Graunke and Roberts’ (1985) study of the PRS and appears to misrepresent the findings in a significant fashion. Graunke and Roberts set out to test whether sensory modalities changed with context rather than persisting as a trait variable. While it is true that they rejected the PRS, their focus was on the more consistent claim of NLP that the use of sensory modalities would shift with context. In their own words:

The major purpose of this study was to examine the impact of varied imagery tasks on individuals’ usage of sensory predicates. The specific purpose was to test whether female volunteers significantly altered their use of sensory predicates across imaging tasks. The present study’s results indicated that most subjects were auditory types during auditory imaging tasks and kinesthetic types during kinesthetic imaging tasks. Thus, the participants in this study were able to vary their use of sensory predicates according to the situational context or task demands. (1985, p. 529)

Witkowski finally turns his attention to several previous reviews of the literature. He cites Sharpley’s two reviews (1984, 1987) and Einspruch and Forman’s (1985) reply to Sharpley. We have already examined Sharpley at length and discovered his studies to be ill informed and based on presuppositions that do not exist within NLP.

In all, the serious, published research into the validity of NLP seems to be characterized by a series of false understandings dating from the early days and the early misconceptions of those writers. The preponderance of the published literature largely comes down on the side arguing against NLP but for entirely the wrong reasons, they never tested NLP, only their own maps of what they thought it was. As Witowski adeptly argues: “Argumenta ponderantur, non numerantur – the force of the arguments lies in their weight, not numbers” (p.12). We might suggest that arguments are to be weighed, not counted.

This review has focused on serious scholarly critiques of NLP. The most cited of these studies, were flawed by consistent errors of fact and interpretation. The first of the errors is an assumption about the basic tenets of NLP. Most of the studies were based upon the idea NLP stands or falls on the validity of the PRS—which is assumed to be a foundational construct of the field–and its assessment by EACS and conversational predicates. These interpretations were wrong at the time of the research and remain wrong today. These publications have resulted in the accumulation of false findings regarding the theoretical validity of NLP. The second error appears to be an over reliance on the historical research with the assumption that that research tested the actual claims of NLP. Many of the studies either did not investigate NLP in sufficient depth to understand what was and was not central to it while others simply relied on the conclusions of previous researchers. These problems might have been solved by adequate training in NLP or if the authors had reviewed their understandings of the field with its originators and major interpreters. A third error, made in particular by Witkowski, is the assumption that reputable journals are always good sources of valid information. In the case of NLP, otherwise reputable and reliable publishers have allowed studies that were well-designed but based on false premises about the field under investigation to be published. The scientific method has great value but only when it is based on an accurate representation of the premises under investigation.

This review has suggested that responsible journals may institute editorial policies requiring the recruitment of experts from the relevant fields, with special attention given to obtaining qualified reviewers for those fields that are new or outside of the main stream.

It is the responsibility of every investigator to confirm the value of their assumptions and findings about any research that they may cite. As noted, this is a major failing in all of the above studies: nobody checked with a knowledgeable source to determine whether their assumptions were valid. It is, however a failing in many studies. As noted by Anderson and Devries (2005) 12.8% of all researchers interviewed acknowledged the use of questionable interpretations of data.


Altman, L. (2006). For Science Gatekeepers, a Credibility Gap. The New York Times. New York, The New York Times Corporation.

Andreas, C. & Andreas, S. (1989) Heart of the Mind. Moab, UT: Real People Press.

Andreas, S. & Andreas, C. (1987) Change Your Mind—and Keep the Change. Moab, UT: Real People Press.

Andreas, S. & Faulkner, C. (1994) NLP: The New Technology of Achievement. New York: Morrow.

Bandler, R. & Grinder, J. (1975a) The Structure of Magic I. Cupertino, CA: Science and Behavior Books.

Bandler, R. & Grinder, J. (1979) Frogs into Princes. Moab, UT: Real People Press.

Bodenhamer, B.G. & Hall, L.M. (2001) The Users Manual for the Brain. Carmarthen: Crown House.

Bradbury, A. C. (2011a) All Roads Lead to Sharpley – Part 1 (The First Journey to the Emerald City). Available online: http://www.bradburyac.mistral.co.uk/sharpley1.html.

Bradbury, A. C. (2011b) All Roads Lead to Sharpley – Part 2 (Second Journey to the Emerald City). Available online: http://www.bradburyac.mistral.co.uk/sharpley2.html.

Burke, D. T., Meleger, A., Schneider, J. C., Snyder, J., Dorvlo, A. S., & Al-Adawi, S. (2003). Eye-movements and ongoing task processing. Perceptual and Motor Skills, 96, 1330- 1338.

Coe, W. C. and Scharcoff, J. A. (1985) An empirical evaluation of the neurolinguistic programming model. International Journal of Clinical and Experimental Hypnosis, 33(4): 310–18.

Dilts, R. B., Grinder, J., Bandler, R. & DeLozier, J. (1980) Neuro-Linguistic Programming: Volume I. The Structure of Subjective Experience. Cupertino, CA: Meta Publications.

Dooley, K. O. & Farmer, A. (1988) Comparison for aphasic and control subjects of eye movements hypothesized in neurolinguistic programming. Perceptual and Motor Skills, 67(1): 233–4.

Einspruch, E. L. & Forman, B. D. (1985) Observations Concerning Research Literature on Neuro-Linguistic Programming. Journal of Counseling Psychology, 32(4): 589–96.

Elich, M., Thompson, R. W. et al. (1985) Mental imagery as revealed by eye movements and spoken predicates: A test of neurolinguistic programming. Journal of Counseling Psychology, 32(4): 622–5.

Fromme, D. K. & Daniell, J. (1984) Neurolinguistic programming examined: Imagery, sensory mode and communication. Journal of Counseling Psychology, 31(3): 387–90.

Graunke, B. & Roberts, T. K. (1985) Neurolinguistic programming: The impact of imagery tasks on sensory predicate usage. Journal of Counseling Psychology, 32(4): 525–30.

Gray, R., Liotta, R., Wake, L. & Cheal, J. (2012). Research and the History of Methodological Flaws. In Lisa Wake, Richard Gray & Frank Bourke (Eds.), The Clinical Efficacy of NLP: A critical appraisal (194-216). London, Routledge.

Gray, R., Wake, L., Andreas, S. & Bolsted, R. (2012). Indirect Research into the Applications of NLP. In Lisa Wake, Richard Gray & Frank Bourke (Eds.), The Clinical Efficacy of NLP: A critical appraisal (153-193). London, Routledge.

Grinder, J. & Bandler, R. (1976) The Structure of Magic II. Cupertino, CA: Science and Behavior Books.

Grinder, J. & Bandler, R. (1980) Forward. In Robert Dilts, John Grinder, Richard Bandler, and Judith DeLozier. (1980) Neuro-linguistic Programming: Volume I. The Structure of Subjective Experience. Cupertino, CA: Meta Publications.

Gumm, W. B., Walker, M. K. et al. (1982) Neurolinguistic programming: Method or myth? Journal of Counseling Psychology, 29(3): 327–30.

Hall, L. M. (2010). Meta reflections 2010: The history of NLP. Neurons–The International egroup of Neuro-Semantics.

Harris, B. (1979) Whatever Happened to Little Albert? American Psychologist, 34: 151–60.

Haser, V. (2002). “Review of ‘The Whorf Theory Complex. A critical reconstruction’.” Cognitive Linguistics 13(2): 187-194.

Heap, M. (1988) Neurolinguistic programming: An interim verdict. In M. Heap (ed.), Hypnosis: Current Clinical, Experimental and Forensic Practices. London: Croom Helm, pp. 268–80.

Ioannidis, J. P. A. (2005). “Why Most Published Research Findings Are False.” PLoS Med 2(8): e124.

Kinsbourne, M. (1974) Direction of gaze and distribution of cerebral thought processes. Neuropsychologia, 12(2): 279–81.

Lewis, B. & Pucelik, F. (1990) Magic of NLP Demystified. Portland, OR: Metamorphous Press.

Linder-Pelz, S. & L. M. Hall (2007). “The theoretical roots of NLP-based coaching.” The Coaching Psychologist, 3(1): 12-17.

Martinson, B. C., M. S. Anderson, et al. (2005). “Scientists behaving badly.” Nature 435: 737-738.

Poffel, S. A. & H. J. Cross (1985) Neurolinguistic programming: A test of the eye movement hypothesis. Perceptual and Motor Skills, 61(3, Pt 2).

Sharpley, C. (1984) Predicate matching in NLP: A review of research on the preferred representational system. Journal of Counseling Psychology, 31(2): 238–48.

Sharpley, C. F. (1987) Research findings on neurolinguistic programming: Nonsupportive data or an untestable theory? Journal of Counseling Psychology, 34(1): 103–7.

Simmons, J. P., L. D. Nelson, et al. (2011). “False-Positive Psychology.” Psychological Science, 22(11): 1359-1366.

Thomason, T. C., Arbuckle, T. et al. (1980) Test of the eye-movement hypothesis of neurolinguistic programming. Perceptual and Motor Skills, 51(1).

Wake, L. (2008) Neurolinguistic Psychotherapy: A Postmodern Approach. London: Routledge.

Wiseman. R., Watt, C., ten Brinke, L., Porter, S., Couper S-L., et al. (2012). The eyes don’t have it: Lie detection and Neuro-Linguistic Programming. PLoS ONE, 7(7): e40259. doi:10.1371/journal.pone.0040259

Witkowski, T. (2010) Thirty-five years of research on Neuro-Linguistic Programming. NLP research data base. State of the art or pseudoscientific decoration? Polish Psychological Bulletin, 41(2): 58–66.

Yapko, M. D. (1981). The effect of matching primary representational system predicates on hypnotic relaxation. American Journal of Clinical Hypnosis, 23, 169-175.