Feature Importance Analysis

via “feature importance computation via gain, split, and cover metrics”

LightGBM Python-package

Unique: Three complementary importance metrics (gain, split, cover) computed directly from tree structure during training, enabling lightweight importance computation without additional inference passes

vs others: Faster than SHAP-based importance computation; more interpretable than permutation importance for tree-based models

via “feature-importance-extraction-and-analysis”

XGBoost Python Package

Unique: Supports three orthogonal importance metrics (gain, cover, frequency) extracted directly from compiled tree structure without re-training; enables efficient importance computation in O(n_trees) time with minimal memory overhead

vs others: Faster than SHAP for global feature importance because it doesn't require model re-evaluation; more granular than scikit-learn's feature_importances_ because it separates gain/cover/frequency metrics

via “feature importance ranking via out-of-bag permutation”

* 🏆 2001: [A fast and elitist multiobjective genetic algorithm (NSGA-II)](https://ieeexplore.ieee.org/abstract/document/996017)

Unique: Uses out-of-bag samples (data naturally held out during bootstrap training) to compute importance without requiring a separate validation set, and measures importance via prediction accuracy drop rather than split-based Gini/entropy metrics — this approach captures feature interactions and is more robust to feature scaling

vs others: More computationally efficient than SHAP for tabular data and does not require retraining, while being more interpretable than gradient-based feature importance because it directly measures prediction impact

via “feature importance ranking via impurity reduction”

* 🏆 1989: [A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition (HMM)](https://ieeexplore.ieee.org/abstract/document/18626)

Unique: CART's impurity-reduction-based importance is computationally efficient (O(n_nodes)) and directly tied to the tree's decision logic, making it interpretable. Unlike permutation importance (which requires retraining) or SHAP values (which require complex game-theoretic calculations), it is built into the tree structure itself.

vs others: Faster to compute than permutation importance or SHAP; more directly interpretable than model-agnostic methods because it reflects actual splits; less robust to feature correlations than permutation importance, which accounts for feature interactions

via “feature-importance-analysis”

via “feature-importance-analysis”

via “feature importance and attribution analysis”

via “feature importance and prediction explanation”