MillerMadowMIEstimator

class infomeasure.estimators.mutual_information.MillerMadowMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseMillerMadowMIEstimator, MutualInformationEstimator

Estimator for the discrete Miller-Madow mutual information.

\[\begin{split}\begin{align}\hat{I}_{\tiny{MM}}(X; Y) &= \hat{H}_{\tiny{MM}}(X) + \hat{H}_{\tiny{MM}}(Y) - \hat{H}_{\tiny{MM}}(X, Y)\\ &= \hat{H}_{\tiny{MLE}}(X) + \hat{H}_{\tiny{MLE}}(Y) - \hat{H}_{\tiny{MLE}}(X, Y) + (K_X + K_Y - K_{XY} - 1)/(2N \cdot \log(\texttt{base}))\\ &= \hat{I}_{\tiny{MLE}}(X; Y) + (K_X + K_Y - K_{XY} - 1)/(2N \cdot \log(\texttt{base}))\\ \end{align}\end{split}\]

For an arbitrary number of random variables this is equivalent to:

\[\begin{align}\hat{I}_{\tiny{MM}}(X_1; \dots; X_n) &= \hat{I}_{\tiny{MLE}}(X_1; \dots; X_n) + \frac{\left(\sum_{i=1}^{n}K_i-1\right) - \left(K_{1,\dots,i}-1\right)}{2N \cdot \log(\texttt{base})} \end{align}\]

\(\hat{I}_{\tiny{MLE}}(X_1; \dots; X_n)\) is the initial DiscreteMIEstimator estimate, \(K_i\) is the number of unique values in the i-th variable, \(K_{1,\dots,i}\) is the number of unique joint values, and \(N\) is the number of samples.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.