Comment: Classifier Technology and the Illusion of Progress — Credit Scoring


These comments support Hand’s argument for the lack of practical progress in classifier technology by pursuing them a little deeper in the specific context of credit scoring. Academic development of modeling techniques tends to ignore the role of the practitioner and the impact of business objectives. In credit scoring it can be seen that the nature of the task forces practitioners to adopt modeling strategies that positively favor simple techniques or, at least, limit the possible advantage of sophisticated techniques. The strategies adopted by credit scorers can be viewed as a heuristic approach to inference of the unobserved (and unobservable) distribution of possible data sets. The technical progress examined by Hand has been aimed toward better goodness of fit. However, technical progress toward a more principled basis for inferring the distribution of future problem data would be more likely to be adopted in practice.

Statistical Science, vol. 21, no. 1, pp. 19–23. doi:10.1214/088342306000000051