-
Why is DolphinDB so fast?
DolphinDB runs extremely fast due to the following reasons:
-
DolphinDB seamlessly integrates a high performance time series database, a lightweight distributed computing framework and a fully-featured programming language. The system minimizes data transfer across multiple components of the system.
-
DolphinDB uses its proprietary memory management system for optimal performance. In big data processing, allocating and releasing memory account for a significant part of the computational cost.
-
DolphinDB is written in C++. In DolphinDB, all parallel or iterative computing uses a lightweight "thread" model, which outperforms systems written in Java that adopt a heavyweight "process" model.
-
DolphinDB's scripting language is vector based and supports functional programming. The language is extremely expressive and effective for data manipulation and analysis. The interpretation overhead is very small.
-
DolphinDB optimizes all built-in functions/commands from basic functions (e.g. sort) to high level functions (e.g. table joins).
-
DolphinDB has an in-memory database. Does it mean it can only work with data smaller than available memory?
No. DolphinDB is a hybrid (in-memory and disk-based) database and can be used to store and analyze huge volumes of data on disk. The system only loads selected columns of necessary partitions into memory for each operation. If there is insufficient memory, the system will remove unnecessary partitions from memory. The size of databases can be significantly larger than memory size.
-
What is the total cost of ownership (TCO) of DolphinDB compared with other databases?
Compared with other database systems, DolphinDB can provide significant TCO savings on hardware cost and personnel cost. Based on our clients’ experience, DolphinDB requires about 10 times less computing infrastructure and maintenance cost. Thanks to the single-stack nature of DolphinDB and the user-friendly programming language, developers are at least 5 times more productive working with DolphinDB than with traditional databases.
-
Does DolphinDB have a steep learning curve?
No. The programming language of DolphinDB is very similar to SQL and Python. Most users can start writing DolphinDB script right away.
-
What's the maximum amount of data that DolphinDB can handle?
DolphinDB has a built-in distributed file system to handle metadata of partitions. It enables DolphinDB to support at least 10 million partitions for a database. In comparison, the popular MPP system can only support tens of thousands of partitions. If the partitions have an average size of 1GB, DolphinDB can handle at least 10PB. This satisfies virtually all financial institutions’ needs for storage and analysis of structured data.
-
What programming paradigms does your programming language offer?
DolphinDB supports imperative programming, vector programming, functional programming, SQL programming, RPC programming and metaprogramming.
-
How many concurrent users can DolphinDB support?
An ordinary PC server can support several thousand concurrent users. The high throughput and concurrency of DolphinDB benefit from the following features:
-
DolphinDB’s optimized data structure
-
DolphinDB adopts the thread pool model. It has better concurrency than the single thread model. It also uses less computing resources than a thread-per-client model or process model.
-
How to deploy DolphinDB?
The entire DolphinDB system is no more than 40MB. DolphinDB can be deployed in the following 4 ways:
-
On PC for personal use. There is no need to configure. Just click on the executable to start DolphinDB.
-
On a single server with partitions. Users can keep the default configuration parameter values or set a few configuration parameters.
-
On a cluster of multiple servers.
-
On cloud.
-
What operating systems are supported?
DolphinDB support Linux and Windows.
-
What data interfaces are supported?
DolphinDB provides CSV, ODBC, JSON and HDF5 data interfaces.
-
Is DolphinDB callable from other programming languages?
Yes. DolphinDB offers APIs for popular languages including Python, Java, C++, C#, R, Javascript and Excel.
-
How can I migrate data between DolphinDB and traditional databases?
DolphinDB supports ODBC and JDBC.
-
Do you support data compression?
Yes. DolphinDB offers both lossless and lossy compression.
-
Do you offer high availability?
Users can specify the number of data replicas in the system. If a data chuck is corrupted, the system will automatically use another copy.
An advantage of multiple data copies in DolphinDB is that all the copies can be utilized in a query execution to improve performance. In comparison, in many other systems only one copy is utilized in a query execution. All other copies only serve as backups.
-
Do you support MapReduce?
Yes. DolphinDB offers built-in functions for MapReduce and iterative MapReduce.
-
How to monitor cluster performance?
DolphinDB provides the following 3 ways of performance monitoring:
-
With built-in functions.
-
On the web interface.
-
Through API with third-party systems, such as Prometheus.
-
Do you support user access control?
DolphinDB offers access control on tables and views. A view can be thought of as the result of a stored query.
-
Do you support NULL values?
Yes. Aggregate functions ignore NULL values. A binary operation involving a NULL value generally returns NULL. Operations involving NULL values are much faster than other systems.
-
What happens if a subtask fails?
If a subtask fails due to a node crash, the system will automatically send the subtask to another node with the necessary data. How the system handles node crash is transparent to users.