A COMPARISON OF MODEL-BASED AND EXPLANATION-BASED FEATURE IMPORTANCE ESTIMATION FOR URL PHISHING DETECTION

This study compares both model-based and explanation-based feature importance estimation methods across four publicly available phishing datasets. The goal of our experiment is to evaluate whether feature importance estimation methods can consistently and coherently identify the most predictive features across different datasets. We are interested in phishing datasets as phishing strategies may change significantly over time, and detecting predictive features might be very relevant. Experimental results show that model-based and explanation-based feature importance values disagree in the feature ranking produced. Moreover, the most predictive features are different across datasets. Our greatest finding is that explanation-based feature ranking is more effective in identifying the most predictive features. Indeed, we show that modelbasmodel-based importance estimation methods are likely to overestimate the predictive power of a few features, while. In contrast,bamodel-baseds prefer to select many featurs, and allow for building more effective models.