ShrinkEntropyEstimator

class infomeasure.estimators.entropy.ShrinkEntropyEstimator(*args, **kwargs)

Bases: DiscreteHEstimator

Shrinkage (James-Stein) entropy estimator.

This estimator applies James-Stein shrinkage to the probability estimates before computing entropy, which can reduce bias in small sample scenarios. The shrinkage probabilities are calculated as:

\[\hat{p}_x^{\text{SHR}} = \lambda t_x + (1 - \lambda) \hat{p}_x^{\text{ML}}\]

where \(\hat{p}_x^{\text{ML}}\) are the maximum likelihood probability estimates, \(t_x = 1/K\) is the uniform target distribution, and the shrinkage parameter \(\lambda\) is given by:

\[\lambda = \frac{ 1 - \sum_{x=1}^{K} (\hat{p}_x^{\text{SHR}})^2}{(n-1) \sum_{x=1}^K (t_x - \hat{p}_x^{\text{ML}})^2}\]

The entropy is then computed using these shrinkage-corrected probabilities.

Based on the implementation in the R package entropy [HS09].

Attributes:

*dataarray_like

The data used to estimate the entropy.

Attributes Summary

dist_dict

Dictionary of shrinkage probabilities for each unique value.

Attributes Documentation

dist_dict

Dictionary of shrinkage probabilities for each unique value. Used by JSD.