sqlDS

Syntax

sqlDS(sqlObj, [forcePartition=false])

Arguments

sqlObj is SQL metacode. For more details about metacode please refer to the section of Metaprogramming.

forcePartition is a Boolean value. The default value is false. If it is set to false, the system checks if the query can be splitted into multiple child queries. If not then it won’t split the query over partitions and will throw an exception. However, when it is set to true, the system splits the query over partitions and runs the query on selected partitions. Cases when a query cannot be splitted into child queries include (1) the group by columns are not partitioning columns (2) the order by clause is specified, among others.

Details

Create a list of data sources based on the input SQL metacode. If the table in the SQL metacode has n partitions, sqlDS generates n data sources. If the SQL metacode doesn’t contain any partitioned table, sqlDS returns a tuple containing one data source.

Examples

$ n=1000000
$ date=take(2019.01.01..2019.01.03,n)
$ sym = take(`C`MS`MS`MS`IBM`IBM`IBM`C`C$SYMBOL,n)
$ price= take(49.6 29.46 29.52 30.02 174.97 175.23 50.76 50.32 51.29,n)
$ qty = take(2200 1900 2100 3200 6800 5400 1300 2500 8800,n)
$ t=table(date, sym, price, qty)

$ db1=database("",VALUE,2019.01.01..2019.01.03)
$ db2=database("",VALUE,`C`MS`IBM)
$ db=database("dfs://stock",COMPO,[db1,db2])
$ trades=db.createPartitionedTable(t,`trades,`date`sym).append!(t)

$ ds=sqlDS(<select * from trades where date=2019.01.02>);

$ typestr ds;
ANY VECTOR

$ size ds;
3

$ ds[0];
DataSource< select [7] * from trades [partition = /stock/20190102/C] >

$ ds[1];
DataSource< select [7] * from trades [partition = /stock/20190102/IBM] >

$ ds[2];
DataSource< select [7] * from trades [partition = /stock/20190102/MS] >