replayDS

Syntax

replayDS(sqlObj, [dateColumn], [timeColumn], [timeRepartitionSchema])

Arguments

sqlObj is metacode with SQL statements. The table object in the SQL statement is a DFS table and must use a DATE type column as one of the partitioning columns.

dateColumn must be a time column of the table queried by the SQL statement, based on which the data is sorted. It can be of DATE (most commonly used), MONTH or other temporal types. If dateColumn is specified, it must be one of the partitioning columns of the DFS table. Data sources are generated based on the time precision of the dateColumn, e.g., if the dateColumn is partitioned by day, the data source is also divided by day.

timeColumn must be a time column of the table queried by the SQL statement, based on which the data is sorted. If dateColumn is of DATE type, you can further deliminate data sources by specifying timeColumn as SECOND, TIME or NANOTIME type.

Note:

  • Currently, parameters dateColumn and timeColumn do not support DATEHOUR type.

  • If dateColumn is not specified, the first column of the table object is treated as the date column.

timeRepartitionSchema is a vector of temporal type. If timeColumn is specified, timeRepartitionSchema deliminates multiple data sources based on timeColumn. For example, if timeRepartitionSchema =[t1, t2, t3], then there are 4 data sources within each day: [00:00:00.000,t1), [t1,t2), [t2,t3), and [t3,23:59:59.999).

Details

Generates a tuple of data sources from a DFS table (queried by a SQL statement) based on its time columns. It can be further divided by the parameters timeColumn and timeRepartitionSchema.

It is used as the inputs of function replay. To replay a DFS table, the replay` function must be conjuncted with the replayDS` function.

Examples

$ n=int(60*60*6.5)
$ sym = take(take(`IBM,n).join(take(`GS,n)), n*2*3)
$ date=take(2021.01.04..2021.01.06, n*2*3).sort!()
$ time=take(09:30:00..15:59:59,n*2*3)
$ volume = rand(100, n*2*3)
$ t=table(sym,date,time,volume)
$ if(existsDatabase("dfs://test_stock")){
$ dropDatabase("dfs://test_stock")
$ }
$ db1=database("",RANGE, 2021.01.04..2021.01.07)
$ db2=database("",VALUE,`IBM`GS)
$ db=database("dfs://test_stock",COMPO,[db1, db2])
$ trades=db.createPartitionedTable(t,`trades,`date`sym)
$ trades.append!(t);
$ ds = replayDS(sqlObj=<select * from loadTable(db, `trades)>, dateColumn=`date, timeColumn=`time)
$ ds.size();
3

$ ds = replayDS(sqlObj=<select * from loadTable(db, `trades)>, dateColumn=`date, timeColumn=`time, timeRepartitionSchema=[11:30:00, 14:00:00])
$ ds.size();
9