infomeasure.composite_measures package

Submodules

infomeasure.composite_measures.jsd module

Jensen-Shannon Divergence (JSD).

infomeasure.composite_measures.jsd.jensen_shannon_divergence(*data, approach: str | None = None, **kwargs)

Calculate the Jensen-Shannon Divergence between two or more distributions.

The Jensen-Shannon Divergence is a symmetrized and smoothed version of the Kullback-Leibler Divergence. It is calculated as the average of the Kullback-Leibler Divergence between each distribution and the average distribution.

\[JSD(P \| Q) = \frac{1}{2} KL(P \| M) + \frac{1}{2} KL(Q \| M)\]

where \(M = \frac{1}{2} (P + Q)\).

Parameters:

parray_like

The first data.

qarray_like

The second data.

…array_like

Further data to compare.

approachstr

The name of the entropy estimator to use.

**kwargsdict

Additional keyword arguments to pass to the entropy estimator.

Returns:

float

The Jensen-Shannon Divergence.

Raises:

ValueError

If the approach is not supported or the entropy estimator is not compatible with the Jensen-Shannon Divergence.

ValueError

If any of the given data is not an array-like object.

infomeasure.composite_measures.kld module

Kullback-Leibler divergence.

infomeasure.composite_measures.kld.kullback_leiber_divergence(data_p, data_q, approach: str = '', **kwargs)

Calculate the Kullback-Leibler Divergence between two distributions.

The Kullback-Leibler Divergence is a measure of the difference between two probability distributions. It is calculated as the expectation of the logarithm of the ratio of the probability of two events. To calculate, we use the identity of combining the joint and marginal entropies:

\[KL(P \| Q) = \sum_{x \in X} P(x) \log \left( \frac{P(x)}{Q(x)} \right) = H_Q(P) - H(P)\]

Parameters:

data_parray_like

The first data.

data_qarray_like

The second data.

approachstr

The name of the entropy estimator to use.

**kwargsdict

Additional keyword arguments to pass to the entropy estimator.

Returns:

float

The Kullback-Leibler Divergence.

Raises:

ValueError

If the approach is not supported or the entropy estimator is not compatible with the Kullback-Leibler Divergence.

Module contents

Composite measures of information.

infomeasure.composite_measures.jensen_shannon_divergence(*data, approach: str | None = None, **kwargs)

Calculate the Jensen-Shannon Divergence between two or more distributions.

The Jensen-Shannon Divergence is a symmetrized and smoothed version of the Kullback-Leibler Divergence. It is calculated as the average of the Kullback-Leibler Divergence between each distribution and the average distribution.

\[JSD(P \| Q) = \frac{1}{2} KL(P \| M) + \frac{1}{2} KL(Q \| M)\]

where \(M = \frac{1}{2} (P + Q)\).

Parameters:

parray_like

The first data.

qarray_like

The second data.

…array_like

Further data to compare.

approachstr

The name of the entropy estimator to use.

**kwargsdict

Additional keyword arguments to pass to the entropy estimator.

Returns:

float

The Jensen-Shannon Divergence.

Raises:

ValueError

If the approach is not supported or the entropy estimator is not compatible with the Jensen-Shannon Divergence.

ValueError

If any of the given data is not an array-like object.

infomeasure.composite_measures.kullback_leiber_divergence(data_p, data_q, approach: str = '', **kwargs)

Calculate the Kullback-Leibler Divergence between two distributions.

The Kullback-Leibler Divergence is a measure of the difference between two probability distributions. It is calculated as the expectation of the logarithm of the ratio of the probability of two events. To calculate, we use the identity of combining the joint and marginal entropies:

\[KL(P \| Q) = \sum_{x \in X} P(x) \log \left( \frac{P(x)}{Q(x)} \right) = H_Q(P) - H(P)\]

Parameters:

data_parray_like

The first data.

data_qarray_like

The second data.

approachstr

The name of the entropy estimator to use.

**kwargsdict

Additional keyword arguments to pass to the entropy estimator.

Returns:

float

The Kullback-Leibler Divergence.

Raises:

ValueError

If the approach is not supported or the entropy estimator is not compatible with the Kullback-Leibler Divergence.