Results with Data Set #1 2.1 Statistical Method Classification data mining is often accomplished on a statistical basis with a logistic regression model. In order to compare performance of the fuzzy model logistic regression was applied to the same data. The vehicle used was a Microsoft Excel add-in called XLMiner [16].
We report results obtained with several data sets here. Data set number 1 is “Mammographic Mass Data” made public by Schulz-Wendtland [17]. The mammographic Mass database consisted of the following parameters.
1. BI-RADS assessment: 1 to 5 (ordinal) 2. Age: patient's age in years (integer) 3. Shape: mass shape: round=1 oval=2 lobular=3 irregular=4 (nominal) 4. Margin: mass margin: circumscribed=1 microlobulated=2 obscured=3 ill-defined=4 spiculated=5 (nominal) 5. Density: mass density high=1 iso=2 low=3 fatcontaining=4 (ordinal) 6. Severity: benign=0 or malignant=1 (binominal)
Missing Attribute Values: - BI-RADS assessment: 2 - Age: 5 - Shape: 31 - Margin: 48 - Density: 76 - Severity: 0
The predicted classification is either benign or malignant. Cases with missing variables were removed. Summary of results are in the table A. For these calculations the gain of each individual fuzzy set was left constant during the aggregation (averaging) process but individual standard deviations were adjusted so as to narrow or increase the width of the membership function. Percentages shown are error percentages. The ratios, such as 7/270, show error count and total records in that classification.
Validation scores are the important ones from the point of view of data analysis. Also there is not a great deal of variation among results. This seems to match results from some other researchers in this area.
Training and validation scores are shown. As anticipated, training scores are slightly better. Using the genetic algorithm to optimize the membership functions contributed a slight improvement when the decision rule was based on the sum of membership values and made no improvement in overall accuracy when the product of membership values was used. In this latter instance it altered the probabilities for failures in detecting malignancy by increasing them with a consequent decrease in failures to classify as benign.
The genetic algorithm, like other training adjustments, worked to minimize the overall error in the training process. The hope is that this will project into improved performance with the validation partition data. It is true that some adjustment of the sigma values in the individual membership functions can be used to shift errors from one type to another while still maintaining overall error rate near constant.
In lieu of adjusting the standard deviations a weighting factor for each attribute can be adjusted. Of course, this is only possible when the aggregation function is a summation. Thus:
(16) ( ) )( 1 k Ak k n Ak x x b µµ == ∑
The coefficient k b applies to the attribute k. Results in this simple case are shown in Table B. And they are not markedly different from results previously obtained
At least in some instances weighting the membership functions is as useful as adjusting the individual shapes of the membership functions. It is not obvious that this would be the case. It does say that simple dynamic weighting adjustment during the training phase may be as useful as anything else. Results also suggest that the statistical logistic regression approach is about as valuable as the fuzzy logic approach. This is not the case in control systems where fuzzy logic performs exceedingly well. This paper reports on an empirical investigation with a limited amount of data. It offers no proof that statistical methods will always perform as well or less well as fuzzy methods. Not all researchers have reported a comparison so one hesitates to make generalized observations.