TechTalk #14 Grokking: Couchbase - NoSQL + Memcached + Real-time + Offline!

Scalability of NoSQL. Flexibility of JSON. Power of SQL.
Vu Nguyen

What is Couchbase?
Couchbase Server Couchbase Mobile
• Document Database
• Key/Value Store
• Distributed Cache
• Embedded Database
• Offline & Real-time sync

Data in Couchbase
• Key: “DocumentID”
• Any string up to 250 bytes
• Unique inside a bucket
• Value
• Any value up to 20 MB
• Any thing: JSON, XML, serialized object, string, etc.
• Meta data
• CAS (check-and-set)
• TTL (time-to-live)
• Optional flags

Architecture
• Node
A Couchbase instance.

Architecture
• Node
• Cluster
A scalable, networked set of nodes.

Architecture
• Node
• Cluster
A scalable, networked set of nodes.
• Bucket
A logical collection of documents with unique keys.
Buckets are evenly distributed across a cluster.
[key1: value]
[key2: value]
[key3: value]
[key4: value]
[key5: value]
[key6: value]

Architecture
• Node
A Couchbase instance
• Cluster
A scalable, networked set of nodes
• Bucket
A logical collection of documents with unique keys.
Buckets are evenly distributed across a cluster.
• Document
A key/value pair within a bucket. Keys are unique
within a bucket.
[key1: value]
[key2: value]
[key1: value]
[key2: value]
[key3: value]
[key4: value]
[key3: value]
[key4: value]
[key5: value]
[key6: value]
[key5: value]
[key6: value]

How to access data?
1. Read/write documents by their keys
• Extremely fast due to cache
2. Map Reduce views
• Distributed secondary indexes, built using map-reduce
3. N1QL (“nickel”) query
• SQL superset for indexing and querying JSON structure
4. Full text search
• Couchbase FTS

Single-Node Architecture
Cluster Manager
Data Service
Index Service
Query Service
Managed Cache
Storage
Couchbase Server nodes are identical.
Two core components
- Cluster Manager
- Node Managers
Three independent services
- Data Service
- Index Service
- Query Service

Cluster Architecture
Cluster Manager
Data Service
Index Service
Query Service
Managed Cache
Storage
Cluster Manager
Data Service
Index Service
Query Service
Managed Cache
Storage
Cluster Manager
Data Service
Index Service
Query Service
Managed Cache
Storage
Cluster Manager
Data Service
Index Service
Query Service
Managed Cache
Storage
Cluster

How does a get operate?
Data Service
Managed Cache
Storage
doc 2 doc 3
doc 1 doc 2 doc 3 doc 4
doc 2 doc 4
Replication
queue
Persistence
queue
Get is extremely fast:
- Common docs
are cached
- Binary TCP connection
Two configs:
- Value-only ejection
- Full meta-data ejection

How does a set operate?
Data Service
Managed Cache
Storage
doc 4
doc 4
Replication
queue
Persistence
queue
Acknowledge type:
- When in RAM (default)
- When in storage
- When replicated
Replication
- Very fast, RAM to RAM

Why Couchbase server fast?
• Get and set are served primarily from cache
• Disk writes are append only
doc 1 doc 2 doc 3

doc 1 doc 2 doc 3 doc 1 doc 3

• Data files are periodically compressed
doc 2 doc 1 doc 3doc 1 doc 3

• Data files are periodically compressed
doc 2 doc 1 doc 3

Map Reduce
Views are secondary indexes
• Written in JavaScript
• Executed by V8 engine
function (doc, meta) {
if (doc.age > 30) {
emit(doc.department, null)
}
}

N1QL Queries
SQL for JSON data
• SELECT, INSERT, UPDATE, DELETE
• JOIN, WHERE, GROUP BY
• CREATE INDEX, DROP INDEX
• MIN, MAX, COUNT
• UNION, INTERSECT, EXCEPT, ...
SELECT count(*), city FROM customer
WHERE customer.type = “vip”
GROUP BY customer.city ORDER BY customer.city
LIMIT 5 OFFSET 5

Bucket & vBucket
• Each bucket is divided into 1024 vBucket.
• vBuckets are distributed evenly across all nodes
in the cluster.
• As nodes join/leave cluster, vBuckets adjust
automatically.
Active
vBuckets
Replicated
vBuckets
Active
vBuckets
Replicated
vBuckets
Active
vBuckets
Replicated
vBuckets

Couchbase Mobile
• Embedded database for iOS & Android
• Unofficial JavaScript database (browser & NodeJs)
• Real-time synchronization
• CouchDB replication protocol

CouchDB protocol
GET /source/_changes?feed=normal&style=all_docs&heartbeat=10000
{
"results":[
{"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true},
{"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}],
],
"last_req": 78,
}
GET /source/_changes?feed=continuous&style=all_docs&heartbeat=10000
{"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true}
{"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}]

Sync Function
function (doc, oldDoc) {
if (doc._deleted) {
requireRole("role:editor");
requireUser(oldDoc.writers);
return;
}
if (!doc.title || !doc.creator || !doc.channels || !doc.writers) {
throw({forbidden: "Missing required properties"});
} else if (doc.writers.length == 0) {
throw({forbidden: "No writers"});
}
if (oldDoc == null) {
requireRole("role:editor");
requireUser(doc.creator)
} else {
requireUser(oldDoc.writers);
if (doc.creator != oldDoc.creator) {
throw({forbidden: "Can't change creator"});
}
}
channel(doc.channels);
}

TechTalk #14 Grokking: Couchbase - NoSQL + Memcached + Real-time + Offline!

Recommended

More Related Content

What's hot (20)

Similar to TechTalk #14 Grokking: Couchbase - NoSQL + Memcached + Real-time + Offline! (20)

More from Grokking VN (20)

Recently uploaded (20)

TechTalk #14 Grokking: Couchbase - NoSQL + Memcached + Real-time + Offline!