A controlled experiment investigates a hypothesis by manipulating
one or more independent variables (e.g. a specific design pattern) to measure their effects
on one or more dependent variables (e.g. complexity or correctness). Each combination of values
for the independent variable(s) is a treatment [Easterbrook, 2007]. In human-based experiments,
participants apply different treatments to objects, while in technology-based experiments, different
technical treatments are applied to different objects. A controlled experiment is considered as the
“gold standard” for research and is usually done in a laboratory environment. When conducting the
experiment, subjects are assigned to different treatments at random. In case the researchers cannot randomly assign treatment to subjects, they may use a quasi-experiment. A quasi-experiment is
similar to a controlled experiment, but the subjects are not randomly assigned to the treatment.