Different researchers have evaluated the results of
different prioritization approaches and then on the basis
of these results they have presented new techniques.
Karlsson et al. have conducted a research in which they
have evaluated different techniques [21] and recorded the
results of different techniques.
Karlsson et al. applied the different prioritization
methods in order ''to prioritize 13 well-defined quality
requirements on a small telephony system" [21]. The
results provided by most of the techniques, except AHP
and Bubblesort, are unreliable and prone to errors while
on the other hand AHP and Bubblesort are more reliable
but time consuming due to the decisions which one must
have to consider during requirements prioritization
methods [21]. Though AHP is a reliable technique for
requirements prioritization but the technique is complex
to implement and more time consuming due to pair
comparisons. The Hierarchy AHP is complex to
implement and also prone to errors so the results given by
Hierarchy AHP are not reliable [21]. The Spanning Tree
technique is also prone to faults and is unreliable. The
prioritization techniques of Binary Search and Priority
Groups are also unreliable because of faulty results. The
role of consistency index is very important because it
helps to reduce the human error so the results provided by
Bubblesort may be faulty and unreliable [18]. All the
techniques are applied to only few requirements so AHP
and Bubblesort though efficient but are considered as
problematic when the requirements are going to scale up
[21]. Laurent et al. have said "AHP does not scale well
because the number of comparisons grows exponentially
with the number of requirements" [22]. All these
tehcniques works well for small or medium size projects
but they are not suitable for projects with large scale of
requirements [22]. It is found that AHP is not suitable
when the requirements exceed the limit of 20 because
when the number of comparisons grows so it becomes
difficult to manage the requirements [24].
Laurent et al. have presented a technique called
"Requirements Triage" which is also prone to errors and
the results don't recall [22]. Peng Shoo also proposed an
algorithm for requirements prioritization and this
algorithm or ''technique allows customers to rank a
relatively small set of items then combines these rankings
over a large number of customers to determine an
ordering for a large requirement set" [31]. The proposed
algorithm is just simulated and not implemented in the
real environment. The simulation is performed on small
number (ten) of customers on ten items [31]. Most of the
researchers have tried to present the extensions of AHP
which are focusing the scalability issue but still there is
no evidence of any experiment that is performed on
projects consisting of hundreds and even thousands of
requirements [11]. Most of the experiments are performed
on only few requirements. The "Requirements
Prioritization Model" for market driven products [26] is
an extension of AHP and is proposed to solve the issue of
scalability. The model is implemented on companies A
and B but there is no data evidence or statistics for the
experimentation which is performed and most of the
results are just opinions. The technique or model is
actually focusing the issue of scalability (large scale
requirements) but the proposed model is not for each and
every organization [26] and also not telling the focused
domain. Further there is no evidence that to which extent
the proposed model is solving the issue of scalability. In a
320 value based Fuzzy Logic prioritization technique [27] the
authors have proposed an Intelligent Fuzzy Logic based
technique for requirements prioritization based on the
perceived value of each requirement. But the results after
sufficient experimentation are documented as: "The
process is prone to error due to its heavy reliance on
experts. The process is very time consuming. The element
of bias was visible in some of the experiments
conducted" [27].
However later on an improved version of the technique
called VIRP is presented comprising of three stages or
levels i.e. "Requirement Elicitation and Stakeholder
Level Prioritization", the "Expert Level Prioritization"
and the last or third stage is "Fuzzy Logic Based
Requirement Prioritization"[45]. The first and the last
steps of the technique are automated and the last step is
als made intelligent andfree of biasness using Fuzzy
Logic but the second step "Expert Level Prioritization" is
not automated and intelligent one which consumes too
much time and reduces the efficiency of the technique so
there is the need to propose improvements in the second
step, in terms of automation and intelligence, which is
heavily based on experts. There is the need to remove the
role of experts or software engineers in order to make the
technique more efficient with respect to time. Different
experiments were performed about the performance of
different prioritization techniques in order to evaluate the
total time consumed by all techniques including VIRP.
The comparison of time consumption of different
prioritization techniques during experimentation
including VIRP is given in the Figure 1 [45] as: