mutualInfo

Syntax

mutualInfo(X, Y)

Arguments

X is a scalar/vector/ matrix.

Y is a scalar/vector/ matrix.

X and Y can be integral or symbol types.

Details

Calculate the mutual information of X and Y.

The calculation uses the following formula:

\(MI(U,V) = \sum\limits_{i = 1}^{|U|}\sum\limits_{j = 1}^{|V|}\dfrac{|{U_i} \cap {V_j}|}{N}log \dfrac{N\lvert{U_i} \cap {V_i}\rvert}{|U_i||V_j|}\)

If X or Y is a matrix, calculate the mutual information of each column and return a vector.

Please note that the natural logarithm is used in this formula. If base is set to 2 or 10, please divide the result by log 2 or log 10.

Examples

$ a = [NULL,4,NULL,NULL,-82,97,NULL,56,5,-92]
$ b = [NULL,53,NULL,18,97,-4,-73,NULL,NULL,24]
$ mutualInfo(a, b)
2.302585

$ t=table(take(1..10,10000000) as id, rand(10,10000000) as x, rand(10,10000000) as y);
$ mutualInfo(t.x, t.y)
0.000004

$ m1 = 1..12$3:4
$ m2 = 1..3
$ mutualInfo(m1, m2)
[1.0986, 1.0986, 1.0986, 1.0986]

If X is a matrix, Y can be a vector/matrix with the same row number as X.

$ m1 = [27,29,NULL,56,57,-2,68,38,100,55,94,87,2,29,-5,34,32,86,-4,13,66,28,33,87,20,88,13,51,13,79]$6:5
$ m2 = [44,29,44,NULL,36,57,48,71,39,6,30,NULL,42,NULL,95,55,22,93,70,27,51,24,63,45,-10,87,44,92,69,100]$6:5
$ mutualInfo(m1, m2)