Ordinal / Symbolic / Permutation TE Estimation

Ordinal / Symbolic / Permutation TE Estimation#

The Transfer Entropy (TE) from the source process \(X(x_n)\) to the target process \(Y(y_n)\) in terms of probabilities is written as:

\[ T_{x \rightarrow y}(k, l) = -\sum_{y_{n+1}, \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)}} p(y_{n+1}, \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)}) \log \left( \frac{p(y_{n+1} \mid \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)})} {p(y_{n+1} \mid \mathbf{y}_n^{(l)})} \right) \]

where

\(y_{n+1}\) is the next state of \(Y\) at time \(n\),
\( \mathbf{y}_n^{(l)} = \{y_n, \dots, y_{n-l+1}\} \) is the embedding vector of \(Y\) considering the \( l \) previous states (history length),
\( \mathbf{x}_n^{(k)} = \{x_n, \dots, x_{n-k+1}\} \) embedding vector of \(X\) considering the \( k \) previous states (history length),
\(p(y_{n+1}, \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)})\) is the joint probability of the next state of \(Y\), its history, and the history of \(X\),
\(p(y_{n+1} \mid \mathbf{y}_n^{(l)}, \mathbf{x}_n^{(k)})\) is the conditional probability of next state of \(Y\) given the histories of \(X\) and \(Y\),
\(p(y_{n+1} \mid \mathbf{y}_n^{(l)})\) is the conditional probability of next state of \(Y\) given only the history of \(Y\).

Ordinal MI estimates the required probability density function (pdf) based on the ordinal structure [SL08]. The details on the pdf estimation based on ordinal structure by Bandt and Pompe [BP02], is provided in Ordinal / Symbolic / Permutation Entropy Estimation.

import infomeasure as im
import numpy as np
rng = np.random.default_rng(5673267189)

data_x = rng.normal(size=1000)
data_y = np.roll(data_x, 1)
data_control = rng.normal(size=1000)

(im.transfer_entropy(
    data_x,  # source
    data_y,  # target
    approach="ordinal", embedding_dim = 3,
    step_size = 1, prop_time = 0, src_hist_len = 1, dest_hist_len = 1,
),
 im.transfer_entropy(
    data_x,  # source
    data_control,  # target
    approach="ordinal", embedding_dim = 3,
    step_size = 1, prop_time = 0, src_hist_len = 1, dest_hist_len = 1,
))

(np.float64(1.0200381035686348), np.float64(0.020355728241622302))

For further methods, create an instance of the estimator.

est = im.estimator(
    data_x,  # source
    data_y,  # target
    measure='te',  # or 'transfer_entropy'
    approach="ordinal", embedding_dim = 3,
    step_size = 1, prop_time = 0, src_hist_len = 1, dest_hist_len = 1,
)
est.local_vals()

array([0.6597 , 0.71696, 1.38002, ..., 1.52382, 1.52382, 0.55266],
      shape=(997,))

The Effective Transfer Entropy (eTE) method can be accessed like so:

est.effective_val()

np.float64(0.9950759624975345)

Hypothesis testing can also be conducted, with either a permutation test or bootstrapping.

stat_test = est.statistical_test(n_tests=50, method="permutation_test")
stat_test.p_value, stat_test.t_score, stat_test.confidence_interval(90), stat_test.percentile(50)

(np.float64(0.0),
 np.float64(184.23483083269147),
 array([0.02144, 0.03841]),
 np.float64(0.0295775111732639))

The estimator is implemented in the OrdinalTEEstimator class, which is part of the im.measures.mutual_information module.

class infomeasure.estimators.transfer_entropy.ordinal.OrdinalTEEstimator(source, dest, *, cond=None, embedding_dim: int, stable: bool = False, prop_time: int = 0, step_size: int = 1, src_hist_len: int = 1, dest_hist_len: int = 1, cond_hist_len: int = 1, offset: int = None, base: int | float | str = 'e', **kwargs)[source]

Bases: BaseOrdinalTEEstimator, TransferEntropyEstimator

Estimator for the Ordinal / Permutation transfer entropy.

Attributes:

source, destarray_like: The source (X) and dest (Y) data used to estimate the transfer entropy.
embedding_dimint: The size of the permutation patterns.
prop_timeint, optional: Number of positions to shift the data arrays relative to each other (multiple of step_size). Delay/lag/shift between the variables, representing propagation time. Assumed time taken by info to transfer from source to destination. Alternatively called offset.
step_sizeint: Step size between elements for the state space reconstruction.
src_hist_len, dest_hist_lenint: Number of past observations to consider for the source and destination data.

Raises:

ValueError: If the embedding_dim is negative or not an integer.
ValueError: If the embedding_dim is too large for the given data.
ValueError: If step_size, prop_time, and embedding_dim are such that the data is too small.

Notes

If embedding_dim is set to 1, the transfer entropy is always 0.