Skip to main content
Back to timeline
arXivSource publication:

A summary's sufficiency now comes with its error bar: a classifier estimates the Barber–Agakov gap, making earthquake-catalog comparisons trustworthy

Synopsis

In simulation-based inference, high-dimensional or variable-length data are often compressed into low-dimensional summaries, and how much parameter information a summary retains is measured by mutual information, which is intractable without a likelihood. A trained neural posterior yields a Barber–Agakov lower bound used in place of that mutual information, but a small bound can mean either an insufficient summary or an inaccurate density estimator. This work estimates the gap between bound and mutual information with a classifier and restricts comparisons to a data-space neighborhood around the observation; on temporal ETAS earthquake catalogs, the learned summary has both the larger bound and the larger gap, so the bound difference understates the mutual-information difference.

Source-provided article image: Summary Sufficiency in Simulation-Based Inference via Mutual Information
arXiv

Interpretation

Whether a summary retains enough parameter information can now be reported together with its estimation error, rather than as a single bound that density-estimation error may contaminate. Previously the Barber–Agakov lower bound from a trained neural posterior was treated directly as an estimate of mutual information, assuming the estimator was accurate enough that the gap between bound and mutual information was negligible. Across replications of the temporal ETAS model, at round 20 the learned summary's bound exceeded the hand-crafted one in every replication; yet both summaries' gaps had medians above the zero-gap reference, so the assumption that the gap can be ignored does not hold exactly.

The method rewrites the difference between bound and mutual information as a KL divergence between two joint distributions that can both be sampled, then estimates that difference with a binary classifier. That difference originally involved the unknown summary-conditioned posterior and could not be computed, so it could only be assumed negligible. The classifier is evaluated on draws not used in fitting it; applying the same procedure to two halves of a single sample, where the divergence is zero by construction, gives reference values that show no systematic offset, and their spread fixes the resolution of the diagnostic.

Comparisons are further restricted to a data-space region containing the observation, concentrating simulation on that neighborhood while leaving the target posterior unchanged. Training over the full prior predictive distribution spends simulations on regions unlike the observation, and branching-process models can produce supercritical catalogs with unbounded event counts. The restriction is defined through catalog size, with bounds admitting catalogs between one quarter and four times the observed size, retaining a reasonably broad range while limiting the cost of supercritical draws.

Perspective

The demonstrated setting is two equal-dimension summaries of the temporal ETAS earthquake-catalog model, a hand-crafted map from Stockman et al. and an end-to-end trained embedding network, compared over rounds with increasing simulation budgets and an MCMC posterior as reference. The intended reader is a researcher using simulation-based inference who must compress variable-length or high-dimensional simulator output.

The diagnostic reports an average gap over catalogs within the restricted data space, not a quantity local to a single observation; the results support a relative judgment about which summary retains more parameter information, not how much either has lost relative to the full data. The C2ST results show the learned-summary posterior is closer to the MCMC reference, yet at the simulation budget considered neither approximation is indistinguishable from it. How best to choose the observation-centered region remains problem-specific and is left for future work.