Reliability and validity in diagnosis of schizophrenia

Specification: Reliability and validity in diagnosis and classification of schizophrenia, including reference to co-morbidity, culture and gender bias and symptom overlap. 

Reliability is the extent to which a finding is consistent. It is the extent to which psychiatrists can agree on the same diagnosis when independently assessing patients (inter-rater reliability). In order for a classification system to be reliable, the same diagnosis should be made each time. Therefore different psychiatrists should reach the same decision when assessing a patient.

Validity is the extent to which we are measuring what we are intending to measure. In the case of an illness like schizophrenia we have to consider the validity of the diagnostic tools; for example, do different assessment systems arrive at the same diagnosis for the same patient?

Key Study: Rosenhan (1973)

Aim: To investigate how situational factors affect a diagnosis of schizophrenia. Sane confederates went into psychiatric hospitals and told medical health professionals they had a hallucination, and observed whether staff would realise that they were sane. If staff did not detect their sanity, it would have implications for methods of diagnosing mental illness and show that situational factors affect diagnosis.

Method: 8 confederates acted as pseudopatients, going to 12 different hospitals. The real participants were the hospital staff who did not know about the experiment. The pseudopatient called the hospital for an appointment. When they arrived they complained of hearing voices saying “empty”, “hollow” and “thud”. They said that the voices were unclear, unfamiliar and of the same sex as the pseudopatient. Pseudopatients gave false names, occupations and symptoms, but gave real life histories. Once on the ward, the pseudopatients stopped pretending symptoms, behaved normally and wrote observations. Pseudopatients were discharged only when they convinced staff that they were sane.

Results: On admission, staff diagnosed 11 pseudopatients with schizophrenia, and one with manic-depression. Staff never detected their sanity. Nurses reported their behaviour as showing “no abnormal indications”, but did interpret their behaviour in the context of their diagnosis (see conclusion). The average hospital stay was 19 days. All pseudopatients were discharged with diagnosis of schizophrenia ‘in remission’. 35 real patients detected sanity (e.g., saying “You’re not crazy”).

Conclusion: Psychiatric staff cannot always distinguish sanity from insanity. Any diagnostic method that makes such errors cannot be very reliable or valid. However, physicians may not identify sanity because it is less risky to diagnose a healthy person as sick than vice versa. So therefore situational factors do affect diagnosis. Normal behaviour was interpreted in the context of illness (e.g., nursing records suggest writing is pathological). Staff reversed some diagnoses due to the situation (expecting pseudopatients). Staff may be more likely to reverse diagnoses when risks are high (e.g., loss of professional esteem). Essentially Rosenhan’s research showed that psychiatrists cannot reliably tell the difference between an insane and sane person, calling into question the reliability of a schizophrenia diagnosis. ‘Normal’ behaviour was misinterpreted as ‘abnormal’ to support their idea that the pseudopatients had a mental illness. This suggests the validity of psychiatric diagnoses was low and the DSM was flawed.

Co-morbidity

Comorbidity refers to more than one disorders or diseases that exist alongside a primary diagnosis, which is the reason a patient gets referred and/or treated. So in this context it is the occurrence of two illnesses or conditions together; for example, a person has both schizophrenia and a personality disorder. Where two conditions are frequently diagnosed together it calls into question the validity of the classification of both illnesses. It could be that the findings of research are due to psychiatrists not being able to tell the difference between the two conditions. In terms of classification, it may be that, if very severe depression looks a lot like schizophrenia and vice versa, then they might be better seen as a single condition.

Culture bias

Research suggests there is a significant variation between countries when it comes to diagnosing schizophrenia. For example, Harrison et al.'s (1984) research suggested that those of West Indian origin were over-diagnosed with schizophrenia, by white doctors in Bristol, because of their ethnic background. In addition, Copeland et al. (1971) gave a description of a patient to 134 US and 194 British psychiatrists. 69% of the US psychiatrists diagnosed schizophrenia but only 2% of the British gave the diagnosis of schizophrenia. No research has found any cause for this, so it suggests that the symptoms of ethnic minorities are misinterpreted. This calls into question the reliability of the diagnosis of schizophrenia as it suggests that patients can display the same symptoms but receive different diagnoses because of their ethnic background; i.e. a patient’s ethnicity makes it more or less likely that they will be diagnosed with schizophrenia.

One issue is that positive symptoms such as the hallucination or hearing voices may be more acceptable in African cultures because of cultural beliefs in communication with ancestors, and therefore people are more ready to acknowledge such experiences. When reported to a psychiatrist from a different culture these experiences might be seen as bizarre and irrational as the psychiatrists are culturally biased towards what is ‘normal’ in their culture and therefore are ethnocentric unknowingly; i.e. any deviation from what is normal in their culture is misinterpreted and therefore mislabelled as a symptom of schizophrenia.

Escobar (2012) has pointed out that White psychiatrists may tend to over-interpret the symptoms of Black people during diagnosis. Such factors as cultural differences in language and mannerisms, difficulties in relating between black patients and white therapists, and the myth that black people rarely suffer from affective disorders may be causing this problem. Therefore clinicians and researchers must pay more attention to the effects of cultural differences on diagnosis

Gender bias

Some critics of the DSM diagnostic criteria argue that some diagnostic categories are biased towards pathologising one gender rather than the other. For example, Broverman et al. (1970) found that clinicians in the US equated mentally healthy ‘adult’ behaviour with mentally healthy ‘male’ behaviour, illustrating a form of androcentrism. As a result there was a tendency for women to be perceived as less mentally healthy when they do not show ‘male’ behaviour. Also interestingly, some research has indicated that a psychiatrist’s gender might affect their ability to diagnose.

Loring and Powell (1988) randomly selected 290 male and female psychiatrists to read two case articles of patients’ behaviour and then asked them to offer their judgment on these individuals using standard diagnostic criteria. When the patients were described as ‘male’ or no information was given about their gender, 56% were given a diagnosis of schizophrenia. However, when the patients were described as ‘female’, only 20% were given a diagnosis of schizophrenia. This gender bias did not appear to be evident amongst the female psychiatrists. This suggests that diagnosis is influenced not only by the gender of the patient but also the gender of the clinician.

Gender bias also occurs due to clinicians failing to consider that males tend to suffer more negative symptoms than women and have higher levels of substance abuse, or that females have better recovery rates and lower relapse rates. These misconceptions could be affecting the validity of a diagnosis as clinicians are not considering all symptoms.

Clinicians also ignore that there are different predisposing/risk factors between males and females, which give them different vulnerability levels at different points in life. This can possibly explain the gender difference in the onset of schizophrenia.

Symptom overlap

There is considerable overlap between the symptoms of schizophrenia and other conditions, despite the claim that the classification of positive and negative symptoms would make for more valid diagnoses. For example, schizophrenia and bipolar disorder both share positive symptoms like delusions and negative symptoms like avolition. This lack of distinction calls into question the validity of both the classification and diagnosis of schizophrenia. For example, Ellason and Ross (1995) point out that people with dissociative identity disorder (DID) actually have more schizophrenic symptoms than people diagnosed as being schizophrenic! Using the International Classification for Disease (ICD) a patient might be diagnosed as a schizophrenic; however, many of the same patients would receive a diagnosis of bipolar disorder when using DSM criteria. This is not surprising given the symptom overlap: Most people who are diagnosed with schizophrenia have sufficient symptoms of other disorders that they could also receive at least one other diagnosis. Some would argue it suggests that schizophrenia and bipolar disorder may not be two different conditions but one.

Serper et al. (1999) assessed patients with co-morbid schizophrenia and cocaine abuse, cocaine intoxication on its own and schizophrenia on its own. They found that despite there being considerable symptom overlap in patients with schizophrenia and cocaine abuse, it was actually possible to make accurate diagnoses, showing that symptom overlap did not affect the validity of a diagnosis and clinicians can tell the difference between the illnesses.

Ketter (2005) points out that misdiagnosis due to symptom overlap can lead to years of delay in receiving relevant treatment, during which time suffering and further degeneration can occur, as well as high levels of suicide- so symptom overlap can have serious consequences. Focusing on fixing this issue could save money and lives.

Ophoff et al. (2011) assessed genetic material from 50,000 participants to find that of seven gene locations on the genome associated with schizophrenia, three of them were also associated with bipolar disorder, which suggests a genetic overlap between the two disorders and a reason for the symptom overlap. The fact that there is this genetic overlap between the two disorders suggests that gene therapies might be developed which simultaneously treat different illnesses.

Possible exam questions