Moving Functions (m-functions)

DolphinDB provides m-functions for sliding window aggregation. A window of a specified size slides over each element in the given data set to perform a calculation on the windowed values. The result has the same size as the given data.

Introduction

  • Higher-order function moving:

$ moving(func, funcArgs, window, [minPeriods])

Note: The tm-functions are optimized for their specialized use cases and therefore have a better performance than the higher-order function moving().

  • A template of the m-functions:

$ mfunc(X, window, [minPeriods])
$ mfunc(X, Y, window, [minPeriods])

Parameters

X (Y) is a vector/matrix/table.

window is an integer no smaller than 2 or a scalar of DURATION type indicating the size of the sliding window.

Note: The window size is capped at 102400 when m-functions are used in the streaming engines.

minPeriods is a positive integer indicating the minimum number of observations in a window required to be not NULL (otherwise the result is NULL).

List of Functions

  • Based on the value passed by window, m-functions can be divided into two categories:

  1. window is a positive integer (or of integral type) or a DURATION value:

See also

mfunc(X, window, [minPeriods])

msum, msum2, mavg, mprod, mmax, mmin, mmed,

mfirst, mlast, mrank, mcount, mpercentile, mstd,

mstdp, mvar, mvarp, mkurtosis, mskew, mifirstNot, milastNot.

mfunc(X, Y, window, [minPeriods])

mwavg, mwsum, mcorr, mcovar, mbeta.

  1. window is a positve interger:

See also

mimax, mimin, mmad, mmaxPositiveStreak, mmse, mslr.

Windowing Logic

The m-functions operate over backward windows. That is, each element in the given data corresponds to a window (with the size of window) where it is included as the last value.

(1) When X is a regular (non-indexed) vector, matrix or table, the size of the sliding window is based on element counts.

Note: For tables, only columns of Boolean and numeric types participate in the calculation.

The m-functions (excluding mrank and mcount) provide the minPeriods parameter for specifying the minimum number of observations in a window.

Specifically,

  • If minPeriods is not specified, return NULL for the first (window-1) windows.

  • If minPeriods is specified, return NULL for the first (minPeriods-1) windows.

The following example illustrates how the window slides:

$ X = 2 1 3 7 6 5 4 9 8 10
$ msum(X, 3);
[ , , 6, 11, 16, 18, 15, 18, 21, 27]
../../_images/mfunc_1.png

(2) When X is an indexed series or an indexed matrix, the size of the sliding window is based on time. If window is a positive integer, it takes the same time unit as the index of X.

The following example illustrates how the window slides:

$ T = [2022.01.01, 2022.01.02, 2022.01.03, 2022.01.06, 2022.01.07, 2022.01.08, 2022.01.10, 2022.01.11]
$ X = 1..8
$ X1 = indexedSeries(T, X)
$ msum(X1, window=3d);

label

col0

2022.01.01

1

2022.01.02

3

2022.01.03

6

2022.01.06

4

2022.01.07

9

2022.01.08

15

2022.01.10

13

2022.01.11

15
../../_images/mfunc_2.png