ExplainReduce cuts hundreds of local explanations to five, turning proxy models into a readable global explanation
Synopsis
Post-hoc local explanations are numerous and redundant, making them hard to read as a whole. ExplainReduce casts the choice of representative local explanations as a set-cover-like submodular maximization solved by greedy algorithms with worst-case approximation guarantees. Across twelve synthetic and real datasets, a proxy set of as few as five local explanations approaches the fidelity of a full set of 500 on both training and test data, and works for regression and classification with several local explanation methods.
Interpretation
A large set of local explanations can be compressed into a small representative subset that still approximates the closed-box model's predictions about as well as the full set. Local explanation methods typically produce one explanation per data point, leaving hundreds of explanations that are hard to analyse holistically and highly redundant because a single simple model often stays valid over a large region. On twelve synthetic and real datasets, a proxy set of five reaches the training and test fidelity of a full set of 500 local explanations, and this holds across different local explanation generation methods and datasets.
The compression turns the choice of which explanations to keep into a set-selection problem that trades coverage against fidelity, adding explanations greedily by marginal gain. The procedure runs in three stages: generate many local explanations with an existing method, formalize the reduction as an optimization problem, then solve it with a greedy algorithm. Max coverage, min loss and a weighted balance of the two are all monotone submodular maximization, so the greedy solution retains at least about 63.2 percent of the optimal improvement; in experiments the greedy results come close to the true optimum.
The Balanced variant, which weighs coverage against fidelity, matches or leads on fidelity, coverage and stability, and is computationally faster than existing explanation aggregation methods. Existing aggregation methods each carry scope limits: lime's submodular pick applies only to feature-attribution explanations, GLocalX only to rule-based explanations for classification, and the integer-programming spherical-coverage approach only to classification. Across dataset and local-explanation-method combinations, the Balanced variant applies to every combination and reaches the best or within-one-standard-deviation performance on most metrics, with the shortest computation time in all cases.
Perspective
The demonstrated scope is regression and classification on twelve synthetic and real datasets, with local explanations from gradient smoothing, sampling-based methods, embedding-based linear surrogates and rule-based explanations; mapping unseen items to proxies uses nearest neighbours in feature space by Euclidean distance, and the authors suggest choosing a distance measure that fits the data or model in practice.
The current implementation selects proxies only from a global loss or coverage perspective and does not consider the spatial distribution of data with respect to which proxy is associated with each item; metrics such as explanation diversity and usability to humans are not evaluated, and the authors list combining reduction with manifold learning for visualization and studying multi-objective Pareto fronts as future directions.
