Optimizing Performance in Rust for Low-Latency Database Drivers

Optimizing Performance
in Rust for Low-Latency
Database Drivers
Piotr Grabowski, Software Team Leader, ScyllaDB

Poll
Where are you in your NoSQL adoption?

Presenter
Piotr Grabowski, Software Team Leader, ScyllaDB
+ Software Team Leader at ScyllaDB
+ Responsible for all ScyllaDB drivers, ScyllaDB Kafka
Connectors (ScyllaDB Sink Connector and ScyllaDB CDC
Source Connector)
+ Joined ScyllaDB 2.5 years ago

+ For data-intensive applications that require high
throughput and predictable low latencies
+ Close-to-the-metal design takes full advantage of
modern infrastructure
+ >5x higher throughput
+ >20x lower latency
+ >75% TCO savings
+ Compatible with Apache Cassandra and Amazon
DynamoDB
+ DBaaS/Cloud, Enterprise and Open Source
solutions
The Database for Gamechangers
4
“ScyllaDB stands apart...It’s the rare product
that exceeds my expectations.”
– Martin Heller, InfoWorld contributing editor and reviewer
“For 99.9% of applications, ScyllaDB delivers all the
power a customer will ever need, on workloads that other
databases can’t touch – and at a fraction of the cost of
an in-memory solution.”
– Adrian Bridgewater, Forbes senior contributor

+ ScyllaDB runs only on Linux
+ We take advantage of many Linux-only APIs:
+ io_uring
+ (previously) epoll/aio
+ Avi Kivity, CTO and cofounder of ScyllaDB, began
the development of KVM in Linux kernel
+ Great performance and low latencies are our
focus, frequently looking into how ScyllaDB can
work more efficiently with Linux kernel
The Linux-native Database
5
“ScyllaDB stands apart...It’s the rare product
that exceeds my expectations.”
– Martin Heller, InfoWorld contributing editor and reviewer
“For 99.9% of applications, ScyllaDB delivers all the
power a customer will ever need, on workloads that other
databases can’t touch – and at a fraction of the cost of
an in-memory solution.”
– Adrian Bridgewater, Forbes senior contributor

6
+400 Gamechangers Leverage ScyllaDB
Seamless experiences
across content + devices
Digital experiences at
massive scale
Corporate fleet
management
Real-time analytics 2,000,000 SKU -commerce
management
Video recommendation
management
Threat intelligence service
using JanusGraph
Real time fraud detection
across 6M
transactions/day
Uber scale, mission critical
chat & messaging app
Network security threat
detection
Power ~50M X1 DVRs with
billions of reqs/day
Precision healthcare via
Edison AI
Inventory hub for retail
operations
Property listings and
updates
Unified ML feature store
across the business
Cryptocurrency exchange
app
Geography-based
recommendations
Global operations- Avon,
Body Shop + more
Predictable performance
for on sale surges
GPS-based exercise
tracking
Serving dynamic live
streams at scale
Powering India's top
social media platform
Personalized
advertising to players
Distribution of game
assets in Unreal Engine

Optimizing Performance in Rust for Low-Latency Database Drivers

Agenda
+ Introduction
+ ScyllaDB Rust Driver
+ Bindings to ScyllaDB Rust Driver

Drivers 101
+ Drivers (in this presentation) - libraries that allow sending queries to
ScyllaDB
+ Primary protocol: CQL (Cassandra Query Language) protocol
+ TCP
+ ScyllaDB supports CQL v4
+ Frame-based protocol, supporting multiple streams
+ Supports LZ4 and Snappy compression
+ ScyllaDB drivers support shard awareness:
+ Driver can connect to a specific shard of ScyllaDB

Drivers 101 - Role of Drivers
+ The role of drivers:
+ Serialization/deserialization of CQL frames
+ Serialization/deserialization of ScyllaDB types
+ Querying and maintaining metadata about tables/nodes
+ Routing requests to correct nodes (and shards)
+ Sending request across network
+ Conveniently constructing and executing queries in your language of choice:
+ gocqlx
+ Java Driver’s Mapper interface

Drivers 101 - Performance
+ How can the driver improve performance?
+ Shard awareness: sending the query to a correct shard
+ Partitioners: ScyllaDB’s CDC (Change Data Capture) implements a custom
partitioner which determines a node to send the query to
+ LWT Optimization: consistently prefer a single replica when executing a
LWT query to avoid Paxos conflicts
+ Optimizing hot paths in the driver:
+ Serialization/deserialization
+ Routing code
+ Avoiding copies, allocations and locks

ScyllaDB Rust Driver
+ The idea was born during a hackathon in 2020
+ Over the last 3 years we continued the development

+ The idea was born during a hackathon in 2020
+ Over the last 3 years we continued the development
+ Uses Tokio framework
+ The driver is now feature complete, supporting many advanced features:
+ Shard awareness
+ Asynchronous interface with support for large concurrency
+ Compression
+ All CQL types
+ Speculative execution
+ TLS support

ScyllaDB Rust Driver - Runtime
+ Async Rust is based on a quite unique future/promise model:
+ Running a function which returns a future does not automatically spawn an
asynchronous task, as in many other languages
+ Instead, async functions need a runtime to execute them
+ Which runtime to choose?
+ Tokio (http://tokio.rs) is a de-facto standard runtime for async Rust
projects.
We chose it due to its rich set of APIs, popularity and very active
community of developers and contributors.

ScyllaDB Rust Driver - API Design
+ A central component of our driver is a session, established once and then
used to communicate with Scylla. It has many customizable parameters,
but most of them have sensible defaults.
let uri = "127.0.0.1:9042";
let session: Session = SessionBuilder::new().known_node(uri).build().await?;
if let Some(rows) = session.query("SELECT a, b, c FROM ks.t", &[]).await?.rows {
for row in rows.into_typed::<(i32, i32, String)>() {
let (a, b, c) = row?;
println!("a, b, c: {}, {}, {}", a, b, c);
}
}

ScyllaDB Rust Driver - O(N²) in Tokio?
+ Issue raised by the author of latte - a benchmark tool for ScyllaDB and Cassandra
+ The driver had problems scaling with high concurrency of requests
+ We managed to identify a root cause in the implementation of FuturesUnordered, a
utility to gather many futures and wait for them
+ Due to cooperative scheduling in Tokio, it was possible for
FuturesUnordered to iterate over all futures each time
it was polled
+ A fix was merged to Tokio to limit the number of
Futures iterated over in each poll

ScyllaDB Rust Driver - Connection Management
Ability to customize the number of connections is critical for performance. Our driver
uses a default of 1 connection per shard, but can be customized to instead establish a
fixed number of connections, be it per node or per shard.

ScyllaDB Rust Driver - Shard Awareness
Scylla takes it even further - drivers can try to connect directly to a core which
owns a particular partition, which implies better latency. Shard awareness is built in
into Scylla Rust Driver from the start.

ScyllaDB Rust Driver - Load Balancing

SELECT * FROM table
WHERE partition_key = “R1250GS”
hash(“R1250GS”) = replica nodes

+ Main goal: reduce number of allocations and atomic operations while
building the query plan, especially on the happy path:
+ Plan function was split to pick() and fallback() methods. This allowed to
better optimize the most common case, where only one node from the load
balancing plan is needed
+ Precomputation of replica sets:
+ A struct introduced that precomputes replica lists of a given strategies, and
provides O(1) access to desired replica slices
Refactor

Refactor
Inserts:
----------
allocs/req: 15.00
reallocs/req: 8.00
frees/req: 15.00
bytes allocated/req: 2458.05
bytes reallocated/req: 269.06
bytes freed/req: 2456.80
(allocated - freed)/req: 1.25
Inserts:
----------
allocs/req: 6.01
reallocs/req: 6.00
frees/req: 6.00
Before After

Refactor
Inserts:
----------
allocs/req: 15.00
reallocs/req: 8.00
frees/req: 15.00
Inserts:
----------
allocs/req: 6.01
reallocs/req: 6.00
frees/req: 6.00
Before After
9 fewer allocations (-
60%)

Refactor
Inserts:
----------
allocs/req: 15.00
reallocs/req: 8.00
frees/req: 15.00
Inserts:
----------
allocs/req: 6.01
reallocs/req: 6.00
frees/req: 6.00
Before After
84% fewer
bytes
allocated

Refactor
Selects:
----------
allocs/req: 48.00
reallocs/req: 8.00
frees/req: 48.00
Selects:
----------
allocs/req: 39.00
reallocs/req: 6.00
frees/req: 39.00
Before After

Refactor
Selects:
----------
allocs/req: 48.00
reallocs/req: 8.00
frees/req: 48.00
Selects:
----------
allocs/req: 39.00
reallocs/req: 6.00
frees/req: 39.00
Before After
Less
difference
compared to
inserts

ScyllaDB Rust Driver - Other Efforts
+ Rack-aware load balancing
+ Reduce the cost of querying ScyllaDB nodes in other racks (corresponding
for example to AWS Availability Zones)
+ Reduce the latency by querying the nearest rack
+ Iterator-based deserialization
+ The current implementation deserializes row data into equivalent of
Vec<Vec<Option<CqlValue>>
+ Skip materializing all rows into vector, deserialize on-the-fly
+ Make great use of Rust lifetimes to guarantee memory safety

ScyllaDB Rust Driver - Iterator-based
Deserialization
+ Reworked Deserialization API
+ Solves performance issues and improves type safety
+ Old API marked as "Legacy" for backward compatibility
+ Problems with Current API
+ Inefficient representation with rows and vecs
+ Incomplete information for FromCqlVal and FromRow
+ New API with DeserializeCql and DeserializeRow
+ Allows on-demand deserialization, reducing allocations
+ More comprehensive type checking and improved deserialization
+ Migration from Legacy API
+ Mechanical changes for most use cases
+ Legacy and new API can be used simultaneously

ScyllaDB Rust Driver - Removing All
Allocations?
+ A community-started project, led by Joseph Perez (@wyfo) written from
scratch to have zero-copy deserialization, zero (or one) allocations per
request
+ Core ideas:
+ Query plan caching
+ Zero/one allocation per request
+ We are looking into incorporating the ideas shown in this project into

ScyllaDB Rust Driver - Profiling tools
Rust ecosystem makes it easy to look for performance issues in your
project. One of such tools is cargo flamegraph, a utility for creating
flamegraphs, which can be examined to see if any function calls take up too
much CPU time.

For projects based on Tokio, tokio-console can be used to inspect
running asynchronous tasks in real time, browse the used resources, and so
on.
Ref: https://tokio.rs/blog/2021-12-announcing-tokio-console

Bindings to ScyllaDB
Rust Driver

Bindings to ScyllaDB Rust Driver
+ When benchmarking ScyllaDB Rust Driver against other drivers, we
measured it was the most performant driver, beating the C++ driver
+ Why not develop a way to use ScyllaDB Rust Driver from C++ code?
+ Benefits of a unified core:
+ Higher performance
+ Easier maintenance
+ Fewer bugs

Bindings to ScyllaDB Rust Driver - C/C++
+ We started development for the C/C++ language
+ C++ bindings to the Rust driver; the same API as the original
C++ driver
+ Drop-in replacement (just replacing .so file)
+ The resulting project has an order-of-magnitude fewer LoC
+ Better stability, fewer problems compared to the original C++
driver

Bindings to ScyllaDB Rust Driver - C/C++
#[no_mangle]
pub unsafe extern "C" fn cass_future_ready(future_raw: *const
CassFuture) -> cass_bool_t {
let state_guard = ptr_to_ref(future_raw).state.lock().unwrap();
match state_guard.value {
None => cass_false,
Some(_) => cass_true,
}
}
Rust
__attribute__ ((visibility("default")))
cass_bool_t
cass_future_ready(CassFuture* future);
C

Q&A
ScyllaDB University
Free online learning
scylladb.com/university
scylladb.com/events
Build Low-Latency
Rust Applications
on ScyllaDB
June 21 2023
October 18 + 19, 2023
p99conf.io

Thank you
for joining us today.
@scylladb scylladb/
slack.scylladb.com
@scylladb company/scylladb/
scylladb/

Optimizing Performance in Rust for Low-Latency Database Drivers

Recommended

More Related Content

What's hot (20)

Similar to Optimizing Performance in Rust for Low-Latency Database Drivers (20)

More from ScyllaDB (20)

Recently uploaded (20)

Optimizing Performance in Rust for Low-Latency Database Drivers

Editor's Notes