Specification: Types of validity across all methods of investigation: face validity, concurrent validity, ecological validity and temporal validity. Assessment of validity. Improving validity.

Validity refers to whether something is true or legitimate. Internal validity is a measure of whether results obtained are solely affected by changes in the variable being manipulated (i.e. by the independent variable) in a cause and effect relationship. External validity is a measure of whether data can be generalised to other situations outside of the research environment. 

Ecological validity

Ecological validity is a type of external validity, and refers to the extent to which psychologists can apply their findings to other settings – predominantly to everyday life. A lack of ecological validity is typically a point made when discussing weaknesses of laboratory based studies. Due to the artificial and contrived setting of a laboratory, it stands to reason that it is difficult to generalise the findings to a more natural situation since behaviour may be very different as a result. Realistically, it is a multitude of variables that make a laboratory experiment low in ecological validity, including the use of artificial stimulus materials.

Exam Hint: If you are suggesting that research results are low in ecological validity as part of your evaluation, make sure that you justify this point with specific examples relating to that individual study. Avoid writing sentences which could be ‘copy and pasted’ into another essay and still make complete sense, as this means you have not tied the commentary closely enough to the question at hand. 

Temporal validity

Temporal validity is another form of external validity, which refers to the extent to which research findings can be applied across time. For example, Asch’s research into conformity is often said to be lacking temporal validity because the study was a ‘child of its time’, that is, the findings were a product of the fact that the study was conducted in a conformist era, and thus the findings might not be as applicable in today’s society.

Assessment of validity

The validity of a psychological test or experiment can be assessed in two main ways. Firstly, the face validity can be considered, that is, does the test appear to measure what it says it measures? For example, if there is a questionnaire that is designed to measure depression, do the items all look like they are going to represent what it is like to have depression? If not, it is not likely to have face validity. A test of face validity is most likely to be conducted by a specialist in the given area, which in the example above could be a clinical psychologist, doctor or other mental health specialist familiar with the assessment of depression. If the specialist believes that the instrument or measure is valid, this is often seen as a good indication of validity.

A second way of assessing whether or not a measure is valid is to use concurrent validity. This is where the performance of the test in question is compared to a test that is already recognised and trusted within the same field. For example, if psychologists are wanting to introduce a new measure of depression, they might compare their results to the data obtained from a measure that is very similar, such as Beck’s depression inventory. As both measures are looking to do the same thing, it would be expected for participants to score relatively similarly on each questionnaire. Statistically, a correlation of +0.80 or higher would assume that there is high concurrent validity.

There are several things psychologists can do to try to improve the validity of their research. The ways in which it can be improved will very much depend on the methods that have been used in the study.

Improving validity: experiments

In experimental research, a control group is often used. This allows psychologists to see whether the independent variable influences the dependent variable. For example, if researchers are testing the efficacy of a new anti‐depressant drug, they will often have an experimental group (who receive the true medication) and a control group (who receive a placebo). In this case, using a control group would allow a comparison to see whether the medication was truly effective, thus giving greater confidence in the validity of the research.

Research can also include single‐blind or double‐blind procedures as a means to improve validity. The use of a single‐blind procedure means that participants do not know which condition they are in, whilst double‐blind means that neither the participant nor the researcher knows which condition participants are in. This ensures that the knowledge of the conditions does not result in demand characteristics on behalf of the participants or investigator effects from the direct or indirect behaviour of the experimenter.

Another way of improving validity in experimental research is to use standardised instructions, which involves giving all participants the same instructions in exactly the same format. By ensuring that participants receive identical information, psychologists can minimise investigator effects. In this way participants are less likely to have a different interpretation of what they are required to do, whilst the researcher is at less risk of giving a higher level of information to some participants compared to others. 

Improving validity: questionnaires

The validity of questionnaires can also be improved using relatively straightforward techniques. Often, researchers will include a lie scale to check the consistency of participants’ responses. One way in which this can be done is by having two items that are asking the same thing, but in opposite ways. For example, on a scale measuring depression imagine that each item asks participants to rate from 1 to 5, with one being ‘completely disagree’ and 5 being ‘completely agree’. There might be one item in the scale that says, ‘I generally sleep well at night’ and another that says, ‘my sleeping has become worse’. A participant could not respond to both items honestly with a rating of with 5, ‘strongly agree’, because they contradict each other. Such items are then used to check the validity of an individual participant’s scores.

Another way of improving validity in a questionnaire is to ensure that participants know that their responses are going to be kept anonymous because, by remaining unidentifiable, participants are less likely to give answers that are socially desirable. 

Improving validity: observations

When it comes to observations, psychologists can improve validity, in particular ecological validity, by making sure that the researchers have minimal impact on the behaviour that they are observing. One way of doing this is to conduct a covert observation, where the researcher is not seen. By doing this, researchers increase the likelihood that the behaviour observed is natural, as participants will not be acting in a way that they deem correct or desirable for the sake of the study.

Another way of improving validity in observations is the use of behavioural categories. In this instance, researchers will tick off behaviours when they are seen which helps to improve validity by reducing the chance of researcher subjectivity. Ensuring that the categories are clearly defined, and do not overlap, would also further improve validity in observations.

Research that employs qualitative methodology as opposed to quantitative methodology is often regarded as having higher ecological validity due to the depth of data that is collected, often through the use of case studies or interviews. However, validity can be lowered because analysis is more subjective and open to the investigator’s interpretation. To strengthen the validity here, there are several things that can be done. First of all, simply including direct quotes from participants can help to improve validity, as it provides evidence that what was being inferred from the data is accurate. Also, validity can be improved by collecting data from a variety of sources; for example, having data that has come from interviews, observations and written reports from participants which is a process called triangulation. 

Possible exam questions

1. What is meant by validity? (1 mark)

2. Briefly explain one way a psychologist could check the validity of the data they have collected in a questionnaire assessing obsessive compulsive disorder (OCD). (2 marks)

3. Explain how ecological validity and temporal validity differ in psychological research. (4 marks)

4. Describe two ways of assessing validity. (4 marks)

5. A psychologist was interested in studying student stress levels in their third year of their degree course. She asked an academic colleague for feedback on her method who reported concern that the psychologist had not checked the reliability and validity of the questionnaire used to measure the level of stress.

Explain how the psychologist could check the reliability and the validity of the stress questionnaire. (5 marks)