4.1 Limitations of the lab experiments.
What Table 1 does not include is reference to evaluation
studies based on laboratory experiments. All methodologies
have advantages and disadvantages, and one of the most
significant criteria is the cost of experimentation. Laboratory
experiments need instruments and are therefore more costly
than other methodologies. Duh et al. [17] compared
laboratory and field test methodologies, and concluded the
following: “There were many more types and occurrences of
usability problems found in the field than in the laboratory.
Those problems discovered tend to be critical issues
regarding usability. Some of these usability problems are
only related to the device being used in the field, which
could not be found using conventional laboratory usability
tests. With regards to the users’ behaviors, users behave less
positively and more negatively in the field than in the
laboratory. Some behaviors can only be observed in the
field. Users also take longer time to perform certain tasks
and also present more negative feelings, such as
dissatisfaction and difficult of use, to the use of the device in
the field.” This is the other reason why this methodology is
not considered.