each (:E)

Syntax

each(func, args…)
(apply a function to each element of the specified parameters)

or

F :E X
(apply a function to each element of X)

or

X <operator>:E Y
(apply an operator to each element of X and Y)

or

func:E(args…)

Arguments

func is a function.

args … are the required parameters of func.

operator is a binary operator.

X / Y can be pair/vector/matrix/table/dictionary. X and Y must have the same dimensions.

Details

Apply a function (specified by func or operator) to each element of args/ X/ Y.

  • For matrices, calculate in each column;

  • For tables, calculate in each row;

  • For dictionaries, calculate each value.

The data type and form of the return value are determined by each calculation result. It returns a vector or matrix if all calculation results have the same data type and form, otherwise it returns a tuple.

The difference between func(X) and func :E X is that the former treats X as the one input variable while the later takes each element in X as an input variable. If func is a vector function, avoid using “:E” since element-wise operations are very slow with a large number of elements.

Examples

Suppose we need to calculate the daily compensation for 3 workers. Their working hours are stored in vector x=[9,6,8]. Their hourly rate is \(10 under 8 hours and \)20 beyond 8 hours. Consider the following function wage:

$ x=[9,6,8]
$ def wage(x){if(x<=8) return 10*x; else return 20*x-80}
$ wage x;
The vector can't be converted to bool scalar.

wage(x) does not return a result, as x<=8, i.e., [9,6,8]<=8 returns a vector of conditions [0,1,1], not a scalar condition that is required by if.

In contrast, consider the following solutions:

$ each(wage, x);
[100,60,80]

$ wage :E x;
[100,60,80]

$ def wage2(x){return iif(x<=8, 10*x, 20*x-80)};
// the iif function is an element-wise conditional operation

$ wage2(x);
[100,60,80]

Similarly, each can also be applied to a function with more than one parameter.

$ def addeven(x,y){if (x%2==0) return x+y; else return 0}
$ x1=1 2 3
$ x2=4 5 6;
$ each(addeven, x1, x2);
[0,7,0]

each with a tuple:

$ t = table(1 2 3 as id, 4 5 6 as value, `IBM`MSFT`GOOG as name);
$ t;

id

value

name

1

4

IBM

2

5

MSFT

3

6

GOOG

 
$ each(max, t[`id`value]);
[3,6]

each with matrices:

$ m=1..12$4:3;
$ m;

col1

col2

col3

1

5

9

2

6

10

3

7

11

4

8

12

 
$ each(add{1 2 3 4}, m);
// note add{1 2 3 4} is a partial application, which adds [1, 2, 3, 4] to each of the 3 columns

col1

col2

col3

2

6

10

4

8

12

6

10

14

8

12

16

 
$ x=1..6$2:3;
$ y=6..1$2:3;
$ x;

col1

col2

col3

1

3

5

2

4

6

 
$ y;

col1

col2

col3

6

4

2

5

3

1

 
$ each(**, x, y);
[16,24,16]
// e.g., 24=3*4+4*3

When there are multiple objects passed in as args/X/Y, the function takes the element at the same position from each object as arguments for each calculation.

$ m1 = matrix(1 3 6, 4 6 8, 5 -1 3)
$ m2 = matrix(3 -6 0, 2 NULL 3, 6 7 9)
$ each(corr, m1, m2)
// equal to corr(m1[0], m2[0]) join corr(m1[1], m2[1]) join corr(m1[2], m2[2])
[-0.216777, 1, -0.142857]

Starting from version 1.30.21, each supports dictionary:

$ d=dict(`a`b`c, [[1, 2, 3],[4, 5, 6], [7, 8, 9]])
$ each(sum, d)
b->15
c->24
a->6

When func is a user-defined function that operates on dictionaries whose keys are STRINGs, the each template combines each dictionary and outputs a table following these rules: - The table schema is only determined by the first dictionary whose values are appended to the first row, and keys are treated as column names. The number of keys is the number of columns. - Iterate through the remaining dictionary and append each dictionary value as a new row in the table. Specifically:

  • When a dictionary key matches a column name, append the corresponding value to that column.

  • For any extra keys in the dictionary that don’t match columns, discard those values.

  • For any extra column names without matching keys, fill in missing values as NULL.

days = 2023.01.01..2023.01.10
def mf(day) {
    out = dict(STRING, ANY)
    if(day==2023.01.05){
        out["v"] = 3
    }
    else{
        out["day"] = day
        out["v"] = 1
    }
    return out
}
each(mf, days)

v   day
1   2023.01.01
1   2023.01.02
1   2023.01.03
1   2023.01.04
3
1   2023.01.06
1   2023.01.07
1   2023.01.08
1   2023.01.09
1   2023.01.10

In the example below, we use the function call in a partial application that applies each of [sin, log] to vector 1..3

// when "functionName" is empty, it will be filled with function names dynamically.
$ each(call{, 1..3},(sin,log));

sin

log

0.841471

0

0.909297

0.693147

0.14112

1.098612

Performance Note

  1. Template peach is recommended for tasks that take a long time to execute.

$ m=rand(1,20000:5000)
$ timer f=peach(mskew{,8},m)
Time elapsed: 3134.71 ms
$ timer f=mskew(m,8)
Time elapsed: 8810.485 ms

  1. Template :E (each) is not recommended when there is a large number of elements. In those scenarios we should look for more efficient vector solutions.

$ x=rand(16, 1000000);
$ timer(10){each(wage, x)};
Time elapsed: 38164.9 ms

$ timer(10){iif(x<8,10*x,20*x-80)};
Time elapsed: 81.516 ms