Discrete TE Estimation#

The Transfer Entropy (TE) from the source process \(X(x_n)\) to the target process \(Y(y_n)\) in terms of probabilities is written as:

\[ T_{x \rightarrow y}(k, l) = -\sum_{y_{n+1}, \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)}} p(y_{n+1}, \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)}) \log \left( \frac{p(y_{n+1} \mid \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)})} {p(y_{n+1} \mid \mathbf{y}_n^{(l)})} \right) \]

where

\(y_{n+1}\) is the next state of \(Y\) at time \(n\),
\( \mathbf{y}_n^{(l)} = \{y_n, \dots, y_{n-l+1}\} \) is the embedding vector of \(Y\) considering the \( l \) previous states (history length),
\( \mathbf{x}_n^{(k)} = \{x_n, \dots, x_{n-k+1}\} \) embedding vector of \(X\) considering the \( k \) previous states (history length),
\(p(y_{n+1}, \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)})\) is the joint probability of the next state of \(Y\), its history, and the history of \(X\),
\(p(y_{n+1} \mid \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)})\) is the conditional probability of next state of \(Y\) given the histories of \(X\) and \(Y\),
\(p(y_{n+1} \mid \mathbf{y}_n^{(l)})\) is the conditional probability of next state of \(Y\) given only the history of \(Y\).

Discrete TE estimation estimates the required probability mass function (pmf) by counting the occurrences of matching configuration in the dataset by keeping the record of the frequencies. These pmf estimates are then plugged back into the above expression to estimate the TE. This estimator is simple and computationally efficient.

import infomeasure as im
import numpy as np
rng = np.random.default_rng(5673267189)

data_x = rng.integers(2, size=1000)
data_y = np.roll(data_x, 1)
data_control = rng.integers(2, size=1000)

(im.transfer_entropy(
    data_x,  # source
    data_y,  # target
    approach='discrete',
    step_size = 1, prop_time = 0, src_hist_len = 1, dest_hist_len = 1,
    base = 2,
),
 im.transfer_entropy(
    data_x,  # source
    data_control,  # target
    approach='discrete',
    step_size = 1, prop_time = 0, src_hist_len = 1, dest_hist_len = 1,
    base = 2,
))

(np.float64(0.9980803542617576), np.float64(0.0005593920840170075))

For further methods, create an instance of the estimator.

est = im.estimator(
    data_x,  # source
    data_y,  # target
    measure='te',  # or 'transfer_entropy'
    approach='discrete',
    step_size = 1, prop_time = 0, src_hist_len = 1, dest_hist_len = 1,
    base = 2,
)
est.local_vals()

array([1.006  , 1.10713, 0.99403, ..., 0.90028, 1.10713, 0.99403],
      shape=(999,))

The Effective Transfer Entropy (eTE) method can be accessed like so:

est.effective_val()

np.float64(0.9958051026851572)

Hypothesis testing can also be conducted, with either a permutation test or bootstrapping.

est.p_value(n_tests = 50, method="permutation_test"), est.t_score()

(np.float64(0.0), np.float64(762.5654989671796))

The estimator is implemented in the DiscreteTEEstimator class, which is part of the im.measures.transfer_entropy module.

class infomeasure.estimators.transfer_entropy.discrete.DiscreteTEEstimator(source, dest, *, cond=None, prop_time: int = 0, step_size: int = 1, src_hist_len: int = 1, dest_hist_len: int = 1, cond_hist_len: int = 1, offset: int = None, base: int | float | str = 'e')[source]

Bases: BaseDiscreteTEEstimator, PValueMixin, EffectiveValueMixin, TransferEntropyEstimator

Estimator for discrete transfer entropy.

Attributes:

source, destarray_like: The source (X) and destination (Y) data used to estimate the transfer entropy.
prop_timeint, optional: Number of positions to shift the data arrays relative to each other (multiple of step_size). Delay/lag/shift between the variables, representing propagation time. Assumed time taken by info to transfer from source to destination. Alternatively called offset.
step_sizeint: Step size between elements for the state space reconstruction.
src_hist_len, dest_hist_lenint: Number of past observations to consider for the source and destination data.

import infomeasure as im
source = [0, 1, 0, 1, 0, 1, 0, 1]
target = [0, 0, 1, 1, 0, 0, 1, 1]
im.transfer_entropy(source, target, approach="discrete", base=2)

np.float64(0.9649839288804954)