Human subjects
There are a few practicalities that complicate matters when data are collected from human subjects, but that do not arise in other kinds of survey or experiment. The problems are different in surveys and experiments so we will treat them separately.
Non-response in surveys
In some official surveys, there is a legal requirement to participate but in most surveys a proportion of the sampled individuals will refuse to answer questions. Different types of survey have different non-response rates — for example, more people respond to questions from an interviewer than from a postal survey.
The response rate can be 20% or lower in some surveys and if people responding have different characteristics from those who refuse, the survey results can be biased. It is therefore important to put considerable effort into achieving a high response rate.
Questionnaire design
In a survey of humans, the questions must be very carefully worded to avoid ambiguities. Questions about age, income and sexual relationships will be considered too personal by some respondents and it is often more acceptable to ask for ages and incomes to be specified in broad groups rather than exactly. Questionnaire design is difficult and a pilot survey with a small number of respondents often shows up problems.
It is best to limit the questionnaire to a small number of well-designed questions that address the aims of the project. Large questionnaires that try to cover too much ground often suffer from a large non-response rate.
Ethical issues in experiments
A different range of issues arise with experiments that involve humans.
For ethical reasons, experiments involving potential danger to the subjects are not possible. Even if there is no known danger, the subjects should be aware of what is involved in the experiment and must give informed consent.
Placebos
The act of administering a treatment to a human subject may itself affect the response, irrespective of the treatment effect. For example, if a drug is being assessed for its ability to reduce headaches, the knowledge that medication has been administered may make the subject feel better, even if the drug has no active ingredient.
To avoid the psychological effect of the treatment on the subject being confounded with the effect of the drug, an indistinguishable 'treatment' with no effect may be given to a control group of subjects; this is called a placebo. For example, two batches of pills of similar size and taste may be prepared, with only one batch containing the active ingredient being assessed. Any difference between the control group and the treatment group can therefore be attributed to the treatment.
Double-blind experiments
A further complication may arise when the measured response from each subject may be affected by knowledge of the treatment applied. If the experimenter knows which treatment has been applied to each experimental unit, there may be a subconscious tendency to systematically over- or under-assess one treatment. To avoid this potential problem, the researcher may be unaware of which experimental units received which treatment until after the experiment.
The experiment is called double-blind if neither the researcher not the subject knows which treatment has been applied. For example, a third party may randomly decide which of two drugs will be given to each subject, and package the appropriate pills for each subject in unlabelled containers. The researcher would administer the treatments and record results without knowing which subjects were receiving which treatments.