# Higher-Order Functions

Introduction

A template is a built-in higher order function, which can extend or enhance a function or an operator. A template takes a function and some objects as input. It works like a pipe between the function and its input data. In general, the input data is dissembled into multiple pieces (which may or may not overlap) in a preset way at first; then the individual pieces of data are applied to the given function to produce results one at a time; finally all individual results are assembled into one object to return. The input data for a template can be vectors, matrices or tables, with the occasional use of scalars and dictionaries. With a template, complicated analytical tasks can be accomplished very efficiently with only a few lines of statements.

Syntax

Templates are always used together with operators, user-defined functions, or system functions. All template symbols start with the symbol colon “:” followed by an upper-case single character.

templateFunctionName (<functionName>, functionParameter1, …functionParameterN)

or

<functionName> :<template symbol> functionParameter

or

functionParameter1 <functionName> :<template symbol> functionParameter2

In DolphinDB, the first parameter of all templates must be a function name. It cannot be a function name array. If we would like to run a template over multiple functions, we need to use the template call inside the template. Please refer to the section about the template call.

Template Summary

The summary table below shows template names and applications.

Symbol

Name

Applications

Examples

:E

each

unary operators, binary operators, function calls

peach

parallel version of `each`

:R

eachRight

binary operators

:L

eachLeft

binary operators

:P

eachPre

binary operators, function calls

:O

eachPost

binary operators, function calls

pivot

function calls

Transpose rows and columns of raw data or grouped aggregation results

:A

accumulate

binary operators, function calls

:T

reduce

binary operators, function calls

:G

groupby

binary operators, function calls

:C

cross

binary operators, function calls

pcross

parallel version of `cross`

:M

moving

binary operators, function calls (aggregate function)

loop

unary operators, binary operators, function calls, mixed return types

ploop

parallel version of `loop`

all

binary operators, function calls

any

binary operators, function calls

call

function calls

Usually used with function `each` to simultaneously call a batch of functions

pcall

parallel version of `call`

:X

contextby

binary operators, function calls

Perform specified calculations in groups

segmentby

binary operators, function calls

rolling

binary operators, function calls

Calculate the beta value of APPL on the market (SPY)

withNullFill

binary operators

Replace the Null value with a specific value in the calculation

byRow

function calls

Calculate data in each row of a matrix

The adverbs listed above can be used iteratively.

The operations of each adverb are conducted in left-to-right order. Take X <operator> :E:L Y for example, the leftmost adverb :E is first applied to each element in X and Y (X(i) and Y(i)), and then the next adverb :L is used to apply the operator for each X(i). The result is returned after all adverbs are applied.

```\$ a=1 2 3
\$ b=4 5 6
\$ c=(a,b)
\$ re=c +:E:L c
\$ re
(1 2 3
- - -
2 3 4
3 4 5
4 5 6
,4  5  6
-- -- --
8  9  10
9  10 11
10 11 12
)
```

Rules to Dissemble and Assemble

In general, a vector is dissembled into scalars, a matrix into columns (vectors), a tuple into multiple elements (of different data forms), and a table into rows (represented by dictionaries).

In the assembly phase, the results of scalar type are merged to form a vector, vectors to a matrix, dictionaries to a table, and other data forms (which are inconsistent) into a tuple.

A template function iterates over a vector/tuple by elements, a matrix by columns, and a table by rows.