One way to compare system performance is to use a benchmark problem a data-processing task with input, processing, and output jobs typical of those the new AIS will be required to perform. Processing times are calculate and compared, and the AIS with the lowest times is judged the most efficient.
Another approach is point scoring, which is illustrated in Table 19-2. For each system evaluation criterion, a weight is assigned based on its relative importance. Vendors are assigned a score for each criterion based on how well their proposals measure up to the standard. The total of the weighted scores provides a basis for comparing and contrasting the various systems. Based on the point-scoring approach in this example, vendor 3 offers the best system. Its system scored 190 points more than vendor 2, the second- place candidate.
Requirements costing estimates the cost of purchasing or developing features that are not present in a particular AIS. The total cost for each AIS is computed by adding the acquisition cost and the purchasing or developing costs. The resulting totals represent the costs of systems with all required features and provide an equitable basis for comparison.
Neither point scoring nor requirements costing is totally objective. In points scoring, the weights and the points used are assigned subjectively and dollar estimates of costs and benefits are not included. Requirements costing overlooks intangible factors such as reliability and vendor support. In any event, the final choice among vendor proposals is not likely to be clear-cut, because it must rely to some extent on subjective factors and cost considerations.
Once the best AIS has been identified, the software should be thoroughly test-driven, other users contacted to see how satisfied they are, vendor personnel evaluated, and proposal details confirmed. The company wants to verify that the AIS that appears to be the best on paper actually is the best in practice. The lessons that Geophysical Systems Corporation learned from its vendor selection process highlight the importance of a thorough vendor evaluation (see Focus 19-1).
Despite the availability of good software packages, many organizations meet their information needs by writing their own software. The next section discussed in- house software development by IT professionals. The subsequent section then discusses software that users develop themselves.