of Experimental Research in Education
Written by Daniel Muijs in Doing Quantitative Research in Education with SPSS. 2004. London. SAGE Publications Ltd. pp 22-26.
I. Advantages
One of the things we are often trying to do in quantitative research is determine what causes what – what is cause and what is effect. Often when talking about the results of research, the term ‘cause’ is used both frequently and loosely, e.g. ‘an overly academic curriculum is a cause of pupil disaffection’.Many studies want to determine causes, and policy frequently wants to address causes of perceived problems (e.g. ‘the causes of crime’).
In experimental studies the researcher is manipulating the treatment so we can be certain of the time sequence. Likewise, the problem of extraneous variables causing a relationship is less strong in experimental research than in any other type of research because the experimenter can control the environment and ensure that as few extraneous factors are involved as possible, as we saw in the section on how to design experiments.
Does this mean that when we do an experiment and find a significant result, we can be certain of cause and effect? This is clearly not the case, for the following reasons:
1. Results from a single experiment may be due to chance. Only if research is replicated, i.e. the findings are repeated in different studies using different participants, preferably in slightly different settings, can we be certain of this.
2. It is always possible that findings are caused by an extraneous factor that we haven’t thought of when setting up our experiment.
3. We are creating an artificial situation. Therefore the question remains: do these effects occur in real-life
situations?
II. Disadvantages
This leads us to some of the weaknesses of the experimental approach. The laboratory set-up is always an artificial one, and the correspondence to real-life situations can be questionable. How applicable are the results of experiments to real-life educational situations? Here, the control that is an advantage of the experimental method becomes a disadvantage.
In everyday settings, any causal effect found in an experimental setting is likely to be influenced by a whole load of contextual factors and influences which will tend to make the relationship far less predictable than in a laboratory setting.
Remember, for example, the study on the effect of violent video games While in an experimental study we may find an effect of watching these videos on children’s behaviour, it is rare that children will be in a situation in which the video will be the only influence on their behaviour. When they are actually playing at school, for example, interactions with peers, school rules, weather, etc. will all influence their behaviour as well.
If we look at the other example about presentation of material in animated form, we would have to question whether this effect really matters in practice, or if it is so small that it makes no real difference to learning in classroom situations compared to other factors (such as teacher interactions).
Transferability is clearly an issue in educational experimental research.Another problem with experimental research is that it can be difficult to put into practice in educational settings. Consider, for example, the issue of evaluating educational programmes and initiatives. We might want to do this using an experimental design because we want to see whether he intervention has caused an improvement in the school. We might want to develop an intervention to improve the reading performance of pupils and would involve randomly assigning pupils to the treatment and control groups in the school in which the experiment is taking place.
This is often problematic in practice. Teachers and parents will be unlikely to be overly keen on this type of design, and there are obvious ethical issues in allowing one group of pupils to receive an intervention that we think/hope is effective while other pupils do not receive this intervention.
In practical terms, realigning timetables etc. to facilitate the experimental design is also difficult. The difficulties are even larger when one is doing an experiment in a number of schools.A further problem occurs when we are implementing an intervention that is specifically designed to take place in a classroom, such as a new teaching method. Obviously, there would
be problems in trying to randomly allocate pupils to teachers who did and did not implement the intervention. As in the example above, this would be disruptive to the school, and lead to possible ethical issues as well as potentially to complaints from parents. Another major problem would be the lack of control over the environment.
In a classroom situation, there is a whole variety of other influences that may affect outcomes, making it difficult to ascribe effects to the intervention. The teachers may be differentially effective, peers may influence each other, and so on. However, taking the intervention out of the classroom and putting it in the laboratory might make the results suspect with regards to transferability. If an intervention is supposed to work in the classroom, testing it in an artificial laboratory environment often would not seem sensible. Because of these problems, educational interventions in schools are typically evaluated using quasi-experimental designs.
