ML Feature Importance Analyzer

Understand which features drive your model predictions using scikit-learn importances, SHAP values, permutation importance, and mutual information.

Why Feature Importance Analysis Matters

Feature importance analysis serves two distinct purposes: model understanding and feature selection. Understanding which features drive predictions helps you catch data leakage, validate that the model has learned sensible patterns (does it use clinically meaningful features?), build stakeholder trust, and debug unexpected behavior. Feature selection based on importance scores can reduce dimensionality, improve generalization, and speed up inference — especially important when collecting or computing certain features is expensive.

No single importance metric is universally correct. Each method has assumptions and failure modes. Best practice is to compute multiple metrics and look for agreement — features that rank highly across Gini impurity, permutation importance, and SHAP are almost certainly important. Features that rank highly in only one metric deserve closer investigation.

scikit-learn feature_importances_ (Gini Impurity)

Tree-based models in scikit-learn expose feature_importances_ after fitting. This measures the total reduction in impurity (Gini or MSE) attributable to each feature across all trees and all splits, normalized to sum to 1.0. It is fast, requires no additional computation, and generally provides a reasonable ranking of important features.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

# Load example data
data = load_breast_cancer()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = data.target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train Random Forest
rf = RandomForestClassifier(n_estimators=200, random_state=42, n_jobs=-1)
rf.fit(X_train, y_train)

# Get feature importances
importances = pd.Series(rf.feature_importances_, index=X.columns)
importances_sorted = importances.sort_values(ascending=True)

# Plot
fig, ax = plt.subplots(figsize=(10, 8))
importances_sorted.plot(kind='barh', ax=ax)
ax.set_title('Random Forest Feature Importances (Gini Impurity)')
ax.set_xlabel('Mean Decrease in Impurity')
plt.tight_layout()
plt.savefig('feature_importance_gini.png', dpi=150)
plt.show()

print(importances.nlargest(10))

Known limitations: high-cardinality features (many unique values) get artificially inflated importance because they offer more potential split points. Correlated features split importance between themselves, making each appear less important than they are. For these reasons, treat Gini importance as a quick first-pass filter, not a definitive ranking.

Permutation Importance

Permutation importance directly measures predictive contribution by destroying a feature's relationship with the target via random shuffling. If a feature is truly important, shuffling its values should significantly hurt model performance. If performance barely changes, the feature is either unimportant or redundant with other features.

from sklearn.inspection import permutation_importance
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Compute permutation importance on validation set
# IMPORTANT: always use held-out data, not training data
result = permutation_importance(
    rf,
    X_test,
    y_test,
    n_repeats=30,      # shuffle each feature 30 times
    random_state=42,
    n_jobs=-1,
    scoring='roc_auc'
)

# Build a DataFrame with mean and std
perm_df = pd.DataFrame({
    'feature':    X.columns,
    'importance': result.importances_mean,
    'std':        result.importances_std,
}).sort_values('importance', ascending=False)

# Features with importance <= 0 are likely irrelevant
useful_features = perm_df[perm_df['importance'] > 0]
print(f"Useful features: {len(useful_features)}/{len(perm_df)}")

# Plot with error bars
fig, ax = plt.subplots(figsize=(10, 8))
ax.barh(
    useful_features['feature'],
    useful_features['importance'],
    xerr=useful_features['std'],
    align='center',
    alpha=0.8,
    color='steelblue',
)
ax.set_title('Permutation Importance (ROC-AUC drop)')
ax.set_xlabel('Mean Accuracy Decrease ± Std')
plt.tight_layout()
plt.savefig('permutation_importance.png', dpi=150)

Key advantages: works with any scikit-learn estimator (not just trees), evaluates the exact metric you care about (AUC, F1, RMSE), and running on validation data reveals actual generalization importance rather than training-fit importance. The standard deviations across repeats indicate stability — high std means the importance estimate is noisy.

SHAP Values — Gold Standard for Explainability

SHAP (SHapley Additive exPlanations) is the most theoretically rigorous approach to feature importance. Rooted in cooperative game theory, SHAP computes the marginal contribution of each feature by averaging its effect across all possible feature orderings. This satisfies important mathematical properties: efficiency (contributions sum to the prediction), symmetry (equivalent features get equal attribution), and consistency (removing an important feature never increases its measured importance).

import shap
import pandas as pd
import matplotlib.pyplot as plt

# Install: pip install shap

# TreeExplainer is fast for tree-based models
explainer = shap.TreeExplainer(rf)
shap_values = explainer.shap_values(X_test)

# For binary classification, shap_values is a list [class0, class1]
# Use index 1 for the positive class
sv = shap_values[1] if isinstance(shap_values, list) else shap_values

# Global importance: mean absolute SHAP value
global_importance = pd.DataFrame({
    'feature':    X.columns,
    'mean_abs_shap': np.abs(sv).mean(axis=0)
}).sort_values('mean_abs_shap', ascending=False)

print(global_importance.head(10))

# Beeswarm summary plot — shows direction + magnitude for all samples
shap.summary_plot(sv, X_test, feature_names=X.columns.tolist())

# Waterfall plot for a single prediction
shap.plots.waterfall(
    shap.Explanation(
        values=sv[0],
        base_values=explainer.expected_value[1],
        data=X_test.iloc[0].values,
        feature_names=X.columns.tolist()
    )
)

# Force plot for a single prediction
shap.force_plot(
    explainer.expected_value[1],
    sv[0],
    X_test.iloc[0],
    matplotlib=True
)

SHAP provides both global importance (mean absolute SHAP) and per-sample explanations. The beeswarm plot is the most information-dense visualization: each dot is one sample, x-position shows the SHAP value (impact on prediction), and color shows the feature value (red = high, blue = low). This immediately reveals the direction and magnitude of each feature's effect.

Mutual Information and Correlation Analysis

Mutual information and correlation are model-agnostic metrics that measure the statistical relationship between each feature and the target variable directly, without training a model first. They are useful for fast initial filtering before training expensive models.

from sklearn.feature_selection import (
    mutual_info_classif,
    mutual_info_regression,
    SelectKBest,
    f_classif,
)
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

# Mutual information — captures non-linear relationships
mi_scores = mutual_info_classif(X_train, y_train, random_state=42)
mi_df = pd.DataFrame({
    'feature': X.columns,
    'mutual_info': mi_scores,
}).sort_values('mutual_info', ascending=False)

print("Top features by Mutual Information:")
print(mi_df.head(10))

# Compare with correlation
correlation_df = X_train.corrwith(pd.Series(y_train, name='target')).abs()
correlation_df = correlation_df.reset_index()
correlation_df.columns = ['feature', 'abs_correlation']

# Merge both metrics
comparison = mi_df.merge(correlation_df, on='feature').sort_values('mutual_info', ascending=False)
print("\nMI vs Correlation — top 10 features:")
print(comparison.head(10))

# Features where MI and correlation disagree may have non-linear relationships
disagreement = comparison[
    (comparison['mutual_info'].rank(ascending=False) -
     comparison['abs_correlation'].rank(ascending=False)).abs() > 10
]
print("\nFeatures with strong MI-correlation disagreement (non-linear?):")
print(disagreement)

Mutual information measures any statistical dependency, including non-linear relationships that correlation misses. A feature with high MI but low correlation likely has a non-linear relationship with the target — a strong signal that non-linear models (trees, neural networks) are needed.

Practical Feature Selection Workflow

• Step 1 — Correlation screening: Remove features with near-zero variance (VarianceThreshold) and check pairwise correlations. For groups of highly correlated features (r > 0.9), keep one representative per group.
• Step 2 — Quick mutual information filter: Rank all features by MI score and consider dropping features with MI ≈ 0 (statistically independent of target).
• Step 3 — Train baseline model + Gini importance: Train a Random Forest and review feature_importances_. Identify the top 50–70% of features by cumulative importance.
• Step 4 — Permutation importance validation: Run permutation importance on validation data. Features with importance ≤ 0 (shuffling them doesn't hurt) are candidates for removal.
• Step 5 — SHAP for final interpretation: Compute SHAP values for the final feature set to understand direction and magnitude of effects. Use for model validation and stakeholder communication.
• Step 6 — Ablation study: Train models with progressively smaller feature sets (top-20, top-10, top-5) and compare validation performance. The simplest model within 1–2% of full-feature performance is usually the right choice for production.