Moving TopN Functions (mTopN functions)
Sort the data based on the specified index, and obtain the top N elements in each sliding window for calculation.
First Release 1.30.16/2.00.4
Introduction
Higher-order function aggrTopN:
$ aggrTopN(func, funcArgs, sortingCol, top, [ascending=true])
For calculations on the top N elements in the sliding window, moving TopN (mTopN) functions are introduced to DolphinDB.
A general template for mTopN functions is:
$ mTopN(X, S, window, top, [ascending=true], [tiesMethod])
$ mTopN(X, Y, S, window, top, [ascending=true], [tiesMethod])
Parameters
X (Y) is a numeric vector or matrix.
S is a numeric/temporal vector or matrix, based on which X are sorted.
window is an integer greater than 1, indicating the sliding window size.
top is an integer in (1, window], indicating the first top elements of X after sorted based on S.
ascending is a Boolean value indicating whether to sort S in ascending order. It is an optional parameter and the default value is true.
tiesMethod is a string that specifies how to select elements if there are more elements with the same value than spots available in the top N after sorting X within a sliding window. It can be:
‘oldest’: select elements starting from the earliest entry into the window;
‘latest’: select elements starting from the latest entry into the window;
‘all’: select all elements.
Note, for backward compatibility, the default value of tiesMethod is ‘oldest’ for the following functions: mstdTopN, mstdpTopN, mvarTopN, mvarpTopN, msumTopN, mavgTopN, mwsumTopN, mbetaTopN, mcorrTopN, mcovarTopN.
For the remaining mTopN functions, the default value of tiesMethod is ‘latest’.
List of Functions
See also
mTopN(X, S, window, top, [ascending=true], [tiesMethod])
mstdTopN, mstdpTopN, mvarTopN, mvarpTopN, msumTopN, mavgTopN, mskewTopN, mkurtosisTopN
mTopN(X, Y, S, window, top, [ascending=true], [tiesMethod])
Windowing Logic
The window size in mTopN functions is measured based on the number of elements.
After sorting X (or X, Y) based on S in a sliding window, the function obtains the first top elements for calculation. It adopts stable sorting algorithms and the order is specified by ascending.
Note: NULL values in S are ignored in data sorting.
In the following figure, the elements of X are sorted based on S in ascending order in a sliding window of length 6, and the first 3 elements are selected for calculation.
For the first top windows, all the elements are taken for calculation. Therefore, the figure illustrates the rules starting from the top + 1 window.
The following example uses function msumTopN:
$ X = [2, 1, 5, 3, 4, 3, 1, 9, 0, 5, 2, 3]
$ S = [5, 8, 1, 9, 7, 3, 1, NULL, 0, 8, 7, 7]
$ msumTopN(X, S, window=6, top=3)
[2,3,8,8,11,10,9,9,4,4,4,3]
The following examples show the usage of parameter tiesMethod:
$ X = [2, 1, 4, 3, 4, 3, 1]
$ S = [5, 8, 1, 1, 1, 3, 1]
// For the last window, there are four elements of value 1
// As tiesMethod is not specified, the default 'oldest' is used, meaning the first 3 occurrences of 1 (corresponding to 4, 3, 4 of X) are selected
$ msumTopN(X, S, window=6, top=3)
[2,3,7,9,11,11,11]
// As tiesMethod is set to 'latest', the latest 3 occurrences of 1 (corresponding to 3, 4, 1 of X) are selected
$ msumTopN(X, S, window=6, top=3, tiesMethod=`latest)
[2,3,7,9,11,11,8]
// As tiesMethod is set to 'all', all the occurrences of 1 (corresponding to 4, 3, 4, 1 of X) are selected
$ msumTopN(X, S, window=6, top=3, tiesMethod=`all)
[2,3,7,9,11,11,12]