******THIS WEBSITE IS CURRENTLY UNDER CONSTRUCTION******


 
STATISTICAL CONSULTING PROGRAM

Observational Studies and Experiments

Department of Science and Mathematics at Montclair State University

    Contents:
Methods of Collecting Information: Observational Studies and Experiments
Details on Observational Studies and Experiments
Control Groups
Principles of Experimental Design
Terminology Table

 


METHODS OF COLLECTING INFORMATION


 

1.OBSERVATIONAL STUDIES

When we conduct observational studies, the individuals are observed the exact way they appear in real life. All the necessary measurements are made to study only those aspects that are of interest to the analyst. All measurements take place without imposing influence on the sample responses. The individuals do not know that they are being observed.

Observational studies do not intend to monitor the effects of treatments on the individuals, and they do not impose any form of intervention. The observational units are fixed, and, therefore, cannot be modified. The sample must be disturbed as little as possible while you gather information.

If we want to see the cause and effect of some factor on our data, we should adhere to experiments instead of studies. You may scroll down for information on experiments. An observational study is a poor source of information on the effects of an active intervention.

* Something we should always remember!

In real life, there are factors like time constraints, cost, and inconvenience, which often forbid us from inspecting the entire population for a particular characteristic. For instance, it is almost impossible to contact all US voters and ask them about their opinion on the presidential candidates for this year's election. Therefore, when you decide to do an observational study, your goal should be to choose a sample from your population, which will represent the entire population. The representatives must be chosen carefully because otherwise your study will be biased.

A survey is a form of an observational study. It is a way of collecting information about a population by first carefully selecting and then analyzing a sample from a population. Surveys can have many different forms. Those most common to us are questionnaires.

Top of Page

2.EXPERIMENTS

A study is an experiment when we do "something" to people or objects in order to monitor their response. The purpose is to see the effects of one variable on another variable. The ideal example of that is clearly a laboratory experiment. In doing an experiment, we impose a treatment on the individuals in our sample and then we observe their response. We control the conditions in which the data is collected, and can eliminate factors that are of no interest to us, but which would be present otherwise. There is an active role played by the experimenter who is not just a passive observer, but rather he controls the setting.

In principle, experiments give good evidence for causation, but control of experimental conditions is often difficult to maintain in practice. There are always ways for things to go wrong. Like observational studies, experiments must be very carefully designed to prevent bias caused by lurking variables and other factors.

Aside from the control issue involved in managing the experimental units, one of the advantages of experiments is our ability to simultaneously study as many variables as we think are necessary for us, in order to understand the interaction between them. In real life, there often exist obstacles that prevent us from studying combined effects of several factors on the individuals. With experiments, the effect of the lurking variables, can be both studied and evaluated.

Aside from laboratories, there are many settings where data is gathered from experiments. One such setting is a hospital testing the effectiveness of a drug on patients. A chemist can experiment with the effects of one chemical compound on another. Educators experiment with innovations in curriculum, and the effectiveness of changes in teaching.
 

Top of Page

 



 ABOUT THE IMPORTANCE OF A CONTROL GROUP...

To better understand the effect of a treatment on the experimental units, we have to have a control group. How will this group help us? The control group will receive a fake treatment. This way we will be able to compare the responses of an actual group with the responses of a control group. Control groups allow us to determine the effects of lurking variables on our sample.

Consider the following example:

A group of concerned parents from town X would like to learn about the effects of television watching on their 6th grade children's level of self-esteem. There are forty teenagers to be studied, and they are in the same class. The parents decide to divide all students into two groups of twenty. One group becomes the control group, and these teenagers maintain unlimited access to television. The other group becomes the treatment group, and those teenagers' access becomes limited. The children do not know about this experiment. After two years, at the end of 8th grade, parents decide to evaluate the results. They prepare the same questionnaire for each group and look at the outcome. As a result, among many other things, parents learn from the control group that there are lurking variables such as puberty, peer pressure, movie theater access, teenage magazines, etc, factors which were not studied in their experiment, but which also affect self-esteem. They realized that their children's levels of self-esteem depended upon more than one factor.

A control group allows us to detect outside variables which may affect our treatment group. Sometimes those variables or factors are not significant for our experiment, but often they are. A control group, if properly observed and analyzed, will help us detect those factors, which would otherwise be omitted. When testing a specific variable, we may omit another variable, perhaps one that is very important. As a result, our conclusions will not be completely correct. In the example above, without considering the outside factors, our results would only be partially correct. Television watching may affect self-esteem, but it certainly is not the only factor which affects it.

Top of Page


 
 

Additional information about OBSERVATIONAL STUDIES

Researchers look at data as it exists in nature, observe its individuals, and measure any variables that could be of interest to them. There is no attempt made to change any of the conditions involved.

Observational studies are very often used to compare different groups.

Example: Doing a political election poll in order to familiarize the public with the views of citizens about the favorite presidential candidates before the presidential election

What are the purposes of Observational Studies?

1. to answer questions associated with the measurements

2. to provide information about the relations among variables in population

3. to document information that can be used for future comparisons

4. to describe changes through a sequence of surveys

5. to compare variables

 Top of Page

 
 

Additional information about the EXPERIMENTS

As was mentioned above, experiments are carefully controlled studies which test and compare variables, treatments, or processes. Researchers deliberately impose treatment or treatments on individuals and observe their responses.

An experiment is a source of convincing data when it comes to understanding the cause and effect relations between variables.

Example: Measuring the effect of a drug on the patients' pulse rate or blood pressure.

What are the purposes of Experiments?

         1. to study the specific factors, and control the effects of other variables

      (factors - explanatory variables of experiments)

      2. to determine and assess consequences of different treatments

      3. to determine causal relations among variables

Top of Page


 

PRINCIPLES OF EXPERIMENTAL DESIGN:

When designing an experiment, we should always adhere to the following basic principles:

 

 

 

 

CONTROL- of the effects of lurking variables (ex. comparing several treatments)
 

 

 

RANDOMIZATION-the use of chance to assign treatments to units

REPLICATION-of the experiment to decrease variation among the results


 

The idea of randomization resembles an act of drawing names from the hat.

 Top of Page

 
 


 

TERM

DEFINITION

BIAS

A study is biased if it favors certain outcomes.

Ex. selecting one gender only to discuss human rights issues

DESIGN

-a method used to select a sample from a population

With experiments, it declares the choice of treatments and the method the treatments are imposed on the units.

DOUBLE-BLIND

This situation occurs when both the experimenter and the experimental units do not know which units belong to a control group and which is not.

EXPERIMENTAL UNITS

-the individuals on which the experiment is done 

Ex. mice in a laboratory 

LURKING VARIABLE

A variable that has a very important effect on the relationship of the variables under study, but is not one of the variables being studied.

NON-RESPONSE

-occurs when one of the individuals selected to appear in the sample refuses to cooperate

POPULATION

-the entire group of individuals we want to study

Ex. population of the US in a pre-election poll

SAMPLE

-a part of the population selected to represent the population

Ex. one thousand voters selected to a pre-election poll

STRATA

-one of the groups of similar individuals into which we divide a population

Ex. humans among the animals 

TREATMENT

-a combination of factors or conditions imposed on the experimental units present in an experiment

UNDER-COVERAGE

-occurs when certain groups from the population are not included in the sample


 Top of Page