We will now consider several potential threats to the internal validity of a study. The confounds described here are those most encountered in psychological research; depending on the nature of the study,other confounds more specific to the type of research being conducted may arise. The confounds present here will give you an overview of some potential problems and an opportunity to begin developing the critical thinking skills involved in designing a sound study. These confounds are little problematic for nonexperimental designs but may also pose a threat to experimental designs. Taking the precautions described here should indicate whether or not the confound is present in the study.
Nonequivalent control group. One of the most basic concerns in an experiment is that the subjects in a control and experimental groups are equivalent at the beginning of the study. For example, if you wanted to test the effectiveness of a smoking cessation program and you compared a group of smokers who voluntarily signed up for the program to a group of smokers who did not,the groups would not be equivalent . They are not equivalent because one of the group chose to seek help , and this makes them different from the group of smokers who didn’t seek help.They might be different in a number of ways. For example they might be concerned with their health . The point is that they differ, and thus, the groups are not equivalent. Using random sampling and random assignment are not used ,subject selection or assignment problems may result. In this case we would have a quasi-experimental design(discussed in chapter 13), not a true experiment.
History. Changes in the dependent variable may be due to historical events that occur outside of the study,leading to the confound known as history effect.These events are most likely unrelated to the study but nonetheless effects of a certain program on stress reduction in college reduction. The study covers a 2 month period during which students participate in your stress-reduction program. If your posttest measures were taken during midterm or final exams, you might notice an increase in stress even though subjects were involved in a program that was intended to reduce stress. Not taking the historical point in the semester into account might lead you to an erroneous conclusion concerning the stress-reduction program. Notice also that a control group of equivalent subjects would have helped reveal the confound in this study.
Maturation.In the research in which subjects are studied over a period of time, a maturation effect can frequently be a problem. Subjects mature physically,socially and cognitively during the course of study. Any changes in the dependent variable that occur across the course of the study, therefore,may be due to maturation and not to the dependent variable are due to maturation;if they are, the subjects in the control group will change on the dependent variable during the course of the study even though they did not receive the treatment.
Testing.In studies in which are measured number of times , a testing effect may be problem-repeated testing may lead to better or worse performance. Many studies involve pretest and posttest measures. Other studies involve taking measures on an hourly, daily ,weekly or monthly basis. In these cases, subjects are exposed to the same or similar “tests” numerous times. As a result, changes in performance on the test may be due to prior experience with the test and not to the independent variable.If, for example, subjects took the same math test before and after participating in a special math course, the improvement observed in scores might be due to the participants’ familiarity with and practice on the test items.This type of testing confound is sometimes referred as a practice effect.Testing can also result in the opposite of a practice effect, a fatigue effect(sometimes referred to as a negative practice effect).Repeated testing fatigues the subjects, and their performance declines as a result .Once again having a control group of equivalent will help to control for testing confounds because researchers will be able to see practice or fatigue effects in a control group.
Regression to the mean. Statistical Regression occurs when individuals are selected for a study because their scores at some measures were extreme either extreme high or extreme low. If we study students that scored in the top 10% on the SAT and we retested them on SAT, then we would expect them to do well again.Not at all,however,would score as well as they did originally because of Statistical Regression.often referred to as regression to the mean – a threat to internal validity in which extreme scores,upon retesting , tend to be less extreme, moving towards the mean. In other words, some of the students did well the first time due to chance or luck. What is going to happen when they are going to take the test the second time?They will not be as lucky, as their scores will regress toward the mean.
Regression to the mean happens in many situations other than research studies. Many people think that a hex is associated with being on the cover of Sports Illustrated and that an athlete’s performance will decline after appearing on the cover.This can be explained by regression of mean.Athletes most likely appear on the cover of sports illustrated after a very successful season or on the peak of their carrier. What is most likely to happen after athletes perform exceptionally well over a period of time? They are likely to regress toward the mean and perform in amore average manner(Cozby,2001). In a research study having an equivalent control group of subjects with extreme scores will indicate whether changes in the dependent measure are due to regression to the mean or to the effects of the independent variable.
Instrumentation. An instrumentation effect occurs when the measuring device is faulty. Problems of consistency in measuring the dependent variables are most likely to occur when the measuring instrument is an human observer.The observer may become better at taking measures during the course of the study or may become fatigued with taking measures. If the measures taken during the study are not taken consistently, then any change in the dependent variable may be due these measurement changes and not to the independent variable. Once again having a control group of equivalent subjects will help to identify the confound.
Mortality or attrition.Most research studies have a certain amount of Mortality or attrition(dropout).Most of the time, the attrition is across experimental and control groups. It is a concern to the researchers, however, when attrition is not equal across the groups. Assume that we begin a study with two equivalent groups of participants.If more subjects leave one group than the others, then the two groups of subjects are most likely no longer equivalent, meaning the comparisons cannot be between groups. Why might we have differential attrition between the groups?Imagine we are conducting a study to the effects of a program aimed at reducing smokes. We randomly select a group of smokers and then randomly assign half to the control group and half to the experimental group. The experimental group participants in our program reduce smoking, but the heaviest smokers just cannot take the demands of a program and quit the program. When we take a posttest measure on smoking, only those participants who were originally light to moderate smokers are left in the experimental group. Comparing them to the control group would be pointless because the groups are no longer equivalent. Having a control group to determine whether there is differential attrition across the groups.
Diffusion of treatment .When subjects in a study are in close proximity to one another, potential threat to internal validity is diffusion of treatment– observed changes in the behaviors of subjects may be due to the information received from other subjects. For example, college students are frequently used as participants in research studies. Because many students live near one another and share classes, some students discuss an experiment in which they participated . If the other students were planning to participate in the study in the future, the treatment has now been compromised because they know how some of the subjects were treated during the study. They know what is involved in one or more of the conditions in the study, and this knowledge may affect how they respond in the study regardless of the condition to which they are assigned. To control for this confound, researchers might try to test the subjects in a study in large groups or within a short time span so they do not have time to communicate with one another. In addition, researchers should stress to the subjects the importance of not discussing the experiment with anyone until it has ended.
Experimenter and Subject effects.When researchers design experiments, they invest considerable time and effort in endeavor.Often this investment leads the researcher to consciously or unconsciously affect or bias the results of the study. For example,a researcher may unknowingly smile more when subjects are behaving in the predicted manner and frown or grimace when subjects are behaving in a manner undesirable to the researcher. This type of experimenter effect is also referred to as experimenter bias or expectancy effects (see chapter 4) because the results of the study are biased by the experimenter’s expectations.
