In this section, we state our research questions (RQs) and
we describe how we evaluated FOREPOST on two applications:
commercial application Renters that we described
as our motivating example in Section II-B and open-source
application JPetStore that is frequently used as an industry
benchmark.
A. Research Questions
In this paper, we make one meta-claim – FOREPOST
is “better” than exploratory random performance testing
[9], [10], which is a popular industry approach. We define
“better” in two ways: finding inputs that lead to significantly
higher computational workloads and finding performance
bottlenecks. We seek to answer the following research
questions.
RQ1: How effective is FOREPOST in finding input test
data that steer applications towards more computationally
intensive executions?
RQ2: How effective is FOREPOST in identifying bottlenecks
with a high degree of automation?
B. Subject AUTs
We evaluate FOREPOST on two subject applications:
Renters and JPetStore. The former is a commercial mediumsize
application that is built and deployed by a major insurance
company. Renters serves over 50,000 daily customers
in the U.S. and it has been deployed for over seven years.
The latter is an open-source application that is often used as
an industry benchmark, since it is highly representative of
enterprise-level three-tier database-centric applications.