NoSQL Data Modeling using Couchbase

Editor's Notes

• #6: Scalability – Multi node, auto-sharded architecture makes it easy to scale out horizontally Availability – Multi node architecture makes high availability easy COUCH = Cluster of Unreliable Commodity Hardware Agility – JSON documents without schema enforcement makes it easy for teams to iterate quickly
• #8: What are some of the questions you have that you’d like to see answered in this presentation? Who is a SQL Server user? Who has used NoSQL? Couchbase? MongoDB?
• #11: Max is 250 bytes 1 Char = 1 Byte, as long as you don’t get into large international chars
• #12: Max is 250 bytes 1 Char = 1 Byte, as long as you don’t get into large international chars
• #13: “cust-evt” is microservice “cust” and type “evt” May want to suppress dashes in GUIDs (.ToString(“N”))
• #17: Let’s see how to represent customer data in JSON. The primary (CustomerID) becomes the DocumentKey Column name-Column value becomes KEY-VALUE pair.
• #18: Date/times are sometimes also stored as Unix timestamps instead of ISO8601
• #19: Also note IS VALUED and IS NOT VALUED
• #21: Couchbase probably isn’t the most cost effective option in the cloud
• #22: Use a separate document to atomically increment the identity
• #25: We aren’t normal form anymore
• #26: Because Purchases is an array, it can have many purchases for a single customer
• #27: You don’t have to stop at a single relationship, lots of relationships can be implicitly created by embedding in a single document
• #30: These are not hard and fast rules, just patterns to use, always analyze your use case
• #31: Sub doc API shifts derserialization load to Couchbase, but reduces network traffic
• #33: Note: My numbers here are somewhat arbitrary
• #34: Under the technical def, any time you use a nested doc you’ve denormalized Here I mean having data duplicated rather than referencing another table Note: This is a counterpoint to the earlier statement about keeping doc small. It’s about striking a balance.
• #35: Explain an online order display where we need to include the product description in the list of products Much more optimized if we don’t need to load all of the product data on each display, the name is already here Good example of a case where mutating the product may not require updating the denormalized copies
• #36: This is a key difference compared to SQL, where ALL data must be retrieved via a query
• #37: Example case where you are very frequently lookup up users by email, but the main user document is keyed by ID Getting this document, then getting the main user document, is usually still faster than a query by email
• #38: Sub document access pattern moves reduces client-side deserialization load (CPU and memory) Server side uses a streaming parser Reduces network traffic
• #39: Patterns for concurrency often conflict with patterns for perf, balance what’s important
• #42: Opposite of the earlier Big Documents For Atomicity slide
• #43: As an example, after the online order is posted we probably need to reduce quantity on hand in some other document This also has load balancing advantages at scale
• #44: Patterns for concurrency often conflict with patterns for perf, balance what’s important
• #45: Don’t over optimize for low frequency access patterns Meet the SLA for low frequency Beet the SLA for high frequency to reduce server load
• #46: Atomic updates: An update to a single document is guaranteed to be atomic High performance: Avoids costly joins, reading/writing multiple documents Clearer data organization: Everything is right there!
• #47: Deduplication – Don’t store the airport details in every single route document Reduced contention – Lots of users updating the same document simultaneously can cause issues, since a document update is atomic of the entire document. Smaller documents – Document size maximum is 20MB, but that size is rarely performant for JSON. N:N – No way to model N:N clearly within a single JSON document
• #53: Patterns for concurrency often conflict with patterns for perf, balance what’s important
• #55: In particular, we’ll avoid bulk operations wherever possible – BIG DATA! Note: Edson Yanaga spoke at TriNUG last year
• #56: Single most common schema change, and it couldn’t be easier
• #57: Simple example: Converting an attribute with one value to an array of multiple values
• #58: 3. Dropping old attr is important for fields that are nullable, since C# can’t tell the difference between NULL and MISSING
• #60: Here we’re dealing with cases where conversion is a MUST, we can’t just maintain a fallback because new features of the new schema are required. Example: changing from basic documents to a CQRS Event Sourcing system
• #63: Diagram is simple UML You can actually model things using polymorphism constructs that never worked well in RDBMS
• #64: Where did Entity Framework get it’s name?
• #67: If modeling payments normalized for RDBMS, you would have needed two tables, one for CC and one for GC. Note a single array with “type” can do it here.