GPU databases;talk of the town

Note: Some products are still in enhancement as Technology is new but contact me for further incorporation as I am in discussion with few vendors too. Any feedback please feel free to ask excepting Test cases evolving on UEFI, BIOS, SQL scripts...It is a quick one.

Big data, HPC solutions are the talk of the town these days. Companies are on the rat race to get solutions around Big data or resource intensive applications. NoSQL, HTAP databases in various forms like graph, columnar, kv, document, object,in-memory or multimodel are harnessing various techniques to solve variety of problems using hardware,software,networking etc. One interesting area is GPU-based solutions.Kinectica DB,Blazegraph,Mapd,Alenka,Brytlyt,PG-Strom, SQream DB, BlazingDB and many others have ventured out to harness the power of GPU side by side with high-end CPU. I am in touch with few folks and I am updating info as and when I get time. CoGaDB, GPUQP, GPUTx, Ocelot,OmniDB, Virginian are vying for new architectures to solve assorted problems. Enhancements are still going on and hiccups are still there in GPU-based DB solutions but it works pretty well with many. Without the help of GPU co-processors DB engines will be almost helpless to meet the needs of modern solutions. PCIe NV solution is becoming bottleneck with the advent of non-volatile persistent storage class memory or the CPU-GPU or FPGA or ASIC accelerators for compute offload for MPP.

Data transfer between CPU and GPU has to be minimal by way of sophisticated data placement algorithms or caching techniques.Most databases developed so far are from the standpoint of CPU and not GPU or processing unit agnostic. The advent of HBM2 also looks to solve many issues. Memory has to catch up with ASIC...speed. Security is still in its fancy stage.GPU connection via PCIe bus to the host, with SIMD features , has high-bandwidth SoC shared memory. Data movement between host and device should be veered for performance issue as it gives rise to IO competition.... Complex control structures, leading to threads workgroup diverge like data-driven conditions can slow down performance . Currently two prominent frameworks viz CUDA, OpenCL are leveraged to develop GPU DBs. UVA provides a single virtual memory address space for all memory in the system, and enables pointers to be accessed from GPU code no matter where in the system they reside, whether its device memory (on the same or a different GPU), host memory, or on-chip shared memory. It also allows cudaMemcpy to be used without specifying where exactly the input and output parameters reside. UVA enables “Zero-Copy” memory, which is pinned host memory accessible by device code directly, over PCI-Express, without a memcpy. Zero-Copy provides some of the convenience of Unified Memory, but none of the performance, because it is always accessed with PCI-Express’s low bandwidth and high latency.

Few sample queries to test GPUs in terms of lingua franca SQL are highlighted below( I have SQL specific test cases with different workloads)

Select Max_ID + Row_Number() Over (Order by ID1,ID2,ID3....)

, ID1, ID2, ID3.....

From

      (Select SRC_Col1 as ID1, SRC_Col2 as ID2, SRC_Col3 as ID3

......

      From

            Source_Table_Name Left Outer join

            Lookup_Table_Name LKP

      On

            ID1 = LKP.Col1 , ID2 = LKP.Col2 AND ID3 = LKP.Col3

.....

      ) STG

      ,

      (

      Select Coalesce(Max(Surrogatekey_Col),0) MaX_ID

      FROM

      Lookup_Table_Name

      ) MX1

SELECT {DISTINCT} <list of columns>  

FROM <list of tables>  

{WHERE <list of "Boolean Factors" (predicates in CNF)>}  

{GROUP BY <list of columns>  

{HAVING <list of Boolean Factors>}}  

{ORDER BY <list of columns>};  

Merge into....

select a.* from a inner join b on(a.id=b.id)...

select a.* from a left outer join b on(a.id=b.id)...

select a.* from a right outer join b on(a.id=b.id)...

select a.* from a right outer join b on(a.id=b.id) where a.id> 10...

select avg(a.sal),a.accountid... from a inner join b on(a.id=b.id) group by a.accountid....

select a.id, sum(a.sal) over(partition by a.deptid order by a.orig desc) from a qualify count(*) > 10

and many other test cases.....

GPUs are special processor and hence faster than CPUs on specific tasks like computations, but when it comes to complex logics, CPUs are far ahead.Hash joins are faster in GPUs rather than CPUs but it is opposite when it comes to selection. There should be an AI heursitic mechanism for data and hardware aware to route queries for CPU-bound or GPU-bound based on certain conditions that suit best for performance. Most database compilers are CPU-vendor agnostic whereas for GPU devices it may not be so plus versions and releases and code can be different.

Disk-based operations are not reaping much benefit from GPU-accelerated DBMS solution as memory is nearer.Column store is favorable for GPU as it features coalesced memory access , compress data, narrow down data transfer rather than heterogeneous row store. The processing of row-at-a-time model or operator-at-a-time model is subjected the environment of the project and is case-to-case basis. GPUs are widespread across clusters of compute nodes due to their attractive performance for data parallel codes. However, communicating between GPUs across the cluster is cumbersome when compared to CPU networking implementations. A number of recent works have enabled GPUs to more naturally access the network, but suffer from performance problems, require hidden CPU helper threads, or restrict communications to kernel boundaries.Lack of sophisticated virtual function calls plus inability to handle complex logic forces many to resort to using operator-at-a-time model.Multiple logical operators can also be merged using dynamic code compilation and tuples are available in registers or shared memory.

GPU RAM resident database seems to be more advantageous as there is less traffic from host to device and GPU RAM's bandwidth is ~10 times of PCIe bus plus consistency check is not necessary between CPU and GPU.Unfortunately GPU RAM around 16 GB is very small compared to CPU's, close to 2TB. Will confirm this when I speak to one client that utilizes this model. For analytics, small amount of data or partitioning does not make sense. Hybrid system is faster than a pure CPU- or GPU-resident system.

Processing and inter-transferring data in smaller GPU RAM and CPU RAM with each system transcoding can affect performance.Page size adjustments with PCIe bus to get large data sets transfer is cumbersome.

GPU DBMS requires robust AI-powered algorithms to develop query routing and optimization techniques to leverage the potential of GPU, CPU and both based on different criteria. The design should be such that workloads are properly utilized. GPU DBs do not face any inconsistency issues as lock-free protocols are enforced. Any locking-based solutions will always breed performance issues.Keeping one copy of data is always safe.The system can perform updates always on one processing device (e.g., the CPU) and periodically synchronize these changes to the other devices.

Kitectica DB: GPUDB keeps the database in the CPU’s main memory in columnar format to avoid the hard-disk bottleneck for GPU's exploitation. Data is compressed using RLE, bit encoding and dictionary encoding to ease PCIe bottleneck and speed up data processing. It uses block-oriented processing or vector processing thus reducing PCIe limitation.Queries are sent to driver programs which are executed calling pre-implemented GPU operators. CPUs manage only the dispatching and post processing jobs. There is no query placement and optimization techniques. It is not recommended for transaction processing.

A typical cluster might consist of multiple identical nodes, each with a couple GPUs and 1TB RAM per node. It is meant mostly for structured data. Speed layer of Lambda can mesmerize with it.

CoGaDB: A column-oriented coprocessor-accelerated DBMS is worth seeing. It attempts on abstracting a layer on top of heterogeneous co-processors to provide agnostic processing--Hawk–A Hardware Adaptive Query Compiler. The downside is that it is disk-driven and when required load data from disk to memory and recourse to using system virtual memory to swap LRU pages to disk. Its data structure fits well for in-memory processing hand in glove with dictionary encoding for varchars and there is no cpu-gpu transcoding. The operator-at-a-time feature and Hybrid Query processing Engine to monitor load for parallel processing shows the class of product.

GPUQP : It develops Mars-a MapReduce framework on graphic processors (GPUs). It supports both disk and in-memory processing. The relational query engine employs both the CPU and GPU accelerated features, exploiting GPU cache availability. Besides the operator-at-a-time technique it also uses partitioning scheme and execute operator on both CPU and GPU. It borrows the idea from Selinger-style optimizer with an analytical cost model to select the cheapest query plan for IO,CPU, GPU. For each operator, GPUQP allocates either the CPU, the GPU, or both processors (partitioned execution). The query optimizer splits a query plan to multiple sub-plans containing at most ten operators. For each sub-query, all possible plans are created and the cheapest sub-plan is selected. Finally, GPUQP combines the sub-plans to a final physical query plan. Works on heavy-duty database constructs, such as tree indexes and joins, and how well a full-fledged GPU query co-processor performs in comparison with their CPU counterparts are not highlighted much or done. Hope someone knows.

CUDB: a GPU boosted relational database engine. CuDB is based on SQLite and preserves its user interface. The heavy-lifting is done by GPUs and the CPU does the interfacing and parsing the queries and generates query plans.

GPUTx : GPUTx keeps all OLTP data inside the GPU’s memory to minimize the impact of the PCIe bottleneck. It also applies a columnar data layout to fit the characteristics of modern GPUs. The main drawback of GPUTx is that it executes only pre-compiled procedures. GPUTx executes pre-compiled stored procedures, which are grouped into one GPU kernel. There is no Query Placement & Optimization technique as all are in GPUs. 3 transaction protocols in use are 2PL, partition-based execution and k-set-based execution. Locking-based protocols do not work well on GPUs. Instead, lock-free protocols such as partition-based execution or k-set should be used. OLAP is not recommended.

Blazegraph : An native gremlin-compliant RDF-SPARQL graph database .DASL is an acronym for a Domain specific language for graphs with Accelerated Scala using Linear algebra (quite a mouthful). With DASL, NVIDIA provides a binding to the Scala programming language for graphs and machine learning algorithms. Blazegraph DASL is a system that executes DASL programs by automatic translation into program code that is optimized for GPUs. DASL enables implementing complex applications that efficiently run on GPUs, without the need for parallel programming expertise or low-level device optimization. As to the nitty gritty of reference architecture, I have not got the details.

MapD : Another distributed scale out columnar Analytics platform.The architecture that MapD has adopted is a familiar one for distributed databases and datastores, and is based on ideas that have been deployed in MemSQL or Vertica databases. Data can be sharded across chunks of GPU frame buffer memory (or any other storage media) and SQL queries can be broken up and run in parallel across those shards, and then the results can be combined together at the end of each individual query. MapD stores data in a columnar layout, and further partitions columns into chunks. A chunk is the basic unit of MapD’s memory manager.The basic processing model of MapD is processing one operator-at-a-time. Due to the partitioning of data into chunks, it is also possible to process on a per chunk basis. Hence, MapD is capable of applying block-oriented processing. MapD processes queries by compiling a query to executable code for the CPU and GPU. The optimizer tries to split a query plan in parts, and processes each part on the most suitable processing device (e.g., text search using an index on the CPU and table scans on the GPU). MapD does not assume that an input data set fits in GPU RAM, and it applies a streaming mechanism for data processing. Not recommended for OLTP.

BlazingDB: A parquet columnar GPU DB.

Ocelot: Ocelot’s storage system is built on top of the in-memory model of MonetDB. Input data is automatically transferred from columnar MonetDB to the GPU when needed by an operator. In order to avoid expensive transfers, operator results are typically kept on the GPU. They are only returned at the end of a query, or if the device memory is too filled to fulfill requests. Additionally, Ocelot implements a device cache to keep relevant input data available on the GPU. Ocelot/MonetDB stores data column-wise in Binary Association Tables (BATs). Each BAT consists of two columns: One (optional) head storing object identifiers, and one (mandatory) tail storing the actual values.

Ocelot inherits the operator-at-a-time bulk processing model of MonetDB, but extends it by introducing lazy evaluation and making heavy use of the OpenCL event model to forward operator dependency information to the GPU. This allows the OpenCL driver to automatically interleave and reorder operations, e.g., to hide transfer latencies by overlapping the transfer with the execution of a previous operator.

In MonetDB, each query plan is represented in the MonetDB Assembly Language (MAL). Ocelot reuses this infrastructures and adds a new query optimizer, which rewrites MAL plans by replacing data processing MAL instructions of vanilla MonetDB with the highly parallel OpenCL MAL instructions of Ocelot. Ocelot does not support CDP, meaning it executes the complete workload either on the CPU or on the GPU.Good for OLAP.

PG-Strom : Extension module of PostgreSQL designed for version 9.6 or later. By utilization of GPU device, it accelerates SQL workloads for data analytics or batch processing to big data set. Its core features are GPU code generator that automatically generates GPU program according to the SQL commands and asynchronous parallel execution engine to run SQL workloads on GPU device. The latest version supports SCAN (evaluation of WHERE-clause), JOIN and GROUP BY workloads. In the case when GPU-processing has advantage, PG-Strom replaces the vanilla implementation of PostgreSQL and transparently works from users and applications. SSD-to-GPU direct SQL execution is the roadmap of PG-Strom. It automatically generates CUDA GPU code, loads data blocks of PostgreSQL to GPU using P2P DMA, then drops unnecessary data with parallel SQL execution by GPU. CPU runs pre-processed data set that is much smaller , fitting the capacity of CPU.

Unlike some DWH systems, PG-Strom shares the storage system of PostgreSQL which saves data in row-format. It is not always best choice for summary or analytics workloads, however, it is also an advantage as well. Users don't need to export and transform the data from transactional database for processing.

Alenka: Is not yet production-ready but it is interesting.GPU based database engine written to use vector based processing and high bandwidth of modern GPUs.

Brytlyt:GPU accelerated database that is based on PostgreSQL 9.4 and uses a Massively Parallel Processing (MPP) architecture to provide horizontal scale out for handling large amounts of data.

Virginian : GPU-accelerated DBMS keeping data in main memory and supporting filter and aggregation operations on all processing devices. It uses uniform virtual addressing (UVA). This technique allows a GPU kernel to directly access data stored in pinned host memory. The accessed data is transferred over the bus transparently to the device and efficiently overlaps computation and data transfers.

Virginian implements a data structure called tablet, which stores fixed size values column oriented. Additionally, tables can handle variable sized data types such as strings, which are stored in a dedicated section inside the tablet. Thus, Virginian supports strings on the GPU. This is a major difference to other GDBMSs, which apply dictionary compression on strings first and work only on compressed values in the GPU RAM.

Virginian uses operator-at-a-time processing as basic query processing model. It implements an alternative processing scheme. While most systems call a sequence of highly parallel primitives requiring one new kernel invocation per primitive, Virginian uses the opcode model, which combines all primitives in a single kernel. This avoids writing data back to global memory and reading it again in the next kernel ultimately resulting in block-wise processing on the GPU.

Virginian can either process queries on the CPU or on the GPU. Thus, there is no mechanism splitting up the workload between CPU and GPU processing devices and hence, no hybrid query optimizer is available.

Note: I referred many papers and could not download and test all these products. But for a Techie, it is understandable and we can verify claims with assorted test cases from all angles.