Method Evaluation, Parametrization, and Result Validation in Unsupervised Data Mining: A Critical Survey
Résumé
Machine Learning and Data Mining build tools intended to help users solve data-related problems that are infeasible for "unaugmented" humans. Tools need manuals, however, and in the case of ML/DM methods, this means guidance w.r.t. which technique to choose, how to parameterize it, and how to interpret derived results to arrive at knowledge about the phenomena underlying the data. While such information is available in the literature, it has not yet been collected in one place. We survey three types of work for clustering and pattern mining: 1) comparisons of existing techniques, 2) evaluations of different parameterization options and studies giving guidance for setting parameter values, and 3) work comparing mining results with the ground truth in the data. We find that although interesting results exist, as a whole the body of work on these questions is too limited. In addition, we survey recent studies in the field of community detection, as a contrasting example. We argue that an objective obstacle for performing needed studies is a lack of data and survey the state of available data, pointing out certain limitations. As a solution, we propose to augment existing data by artificially generated data, review the state-of-the-art in data generation in unsupervised mining, and identify shortcomings. In more general terms, we call for the development of a true "Data Science" that-based on work in other domains, results in ML, and existing tools-develops needed data generators and builds up the knowledge needed to effectively employ unsupervised mining techniques.
Origine : Fichiers produits par l'(les) auteur(s)
Loading...