Better Rulesets by Removing Redundant Specialisations and Generalisations in Association Rule Mining

Authors

  • Henry Petersen University of Sydney
  • Josiah Poon University of Sydney
  • Simon Poon University of Sydney
  • Clement Loy University of Sydney

DOI:

https://doi.org/10.3127/ajis.v21i0.1542

Keywords:

Association Rule Mining, Redundancy

Abstract

Association rule mining is a fundamental task in many data mining and analysis applications, both for knowledge extraction and as part of other processes (for example, building associative classifiers). It is well known that the number of associations identified by many association rule mining algorithms can be so large as to present a barrier to their interpretability and practical use. A typical solution to this problem involves removing redundant rules. This paper proposes a novel definition of redundancy, which is used to identify only the most interesting associations. Compared to existing redundancy based approaches, our method is both more robust to noise, and produces fewer overall rules for a given data (improving clarity). A rule can be considered redundant if the knowledge it describes is already contained in other rules. Given an association rule, most existing approaches consider rules to be redundant if they add additional variables without increasing quality according to some measure of interestingness. We claim that complex interactions between variables can confound many interestingness measures. This can lead to existing approaches being overly aggressive in removing redundant associations. Most existing approaches also fail to take into account situations where more general rules (those with fewer attributes) can be considered redundant with respect to their specialisations. We examine this problem and provide concrete examples of such errors using artificial data. An alternate definition of redundancy that addresses these issues is proposed. Our approach is shown to identify interesting associations missed by comparable methods on multiple real and synthetic data. When combined with the removal of redundant generalisations, our approach is often able to generate smaller overall rule sets, while leaving average rule quality unaffected or slightly improved.

Downloads

Published

2017-11-08

How to Cite

Petersen, H., Poon, J., Poon, S., & Loy, C. (2017). Better Rulesets by Removing Redundant Specialisations and Generalisations in Association Rule Mining. Australasian Journal of Information Systems, 21. https://doi.org/10.3127/ajis.v21i0.1542

Issue

Section

Selected Papers from the Australasian Conference on Data Mining (AusDM)