SlideShare a Scribd company logo

Painless Data Storage with MongoDB & Go

Steven Francia
Steven Francia

The document discusses using MongoDB and Go together for painless data storage. Some key points: - Go is a good language for working with MongoDB as it is fast, friendly to use, and supports concurrency well out of the box. - MongoDB is also fast and scales well. It uses a flexible document model that maps well to Go's data types. - The mgo driver makes it easy to connect to MongoDB from Go, perform CRUD operations, and work with data concurrently. Data can be serialized and deserialized to BSON format efficiently. - Features like GridFS allow storing files within MongoDB, providing replicated file storage.

1 of 70
Downloaded 304 times
Painless Data Storage with MongoDB & Go
• Author of Hugo, Cobra, Viper & More • Chief Developer Advocate for MongoDB • Gopher @spf13
Why Go?
Why Another Language? • Software is slow • Sofware is hard to write • Software doesn’t scale well
Go is Fast • Go execution speed is close to C • Go compile time rivals dynamic interpretation
Go is Friendly • Feels like a dynamic language in many ways • Very small core language, easy to remember all of it • Single binary installation, no dependencies • Extensive Tooling & StdLib
Go is Concurrent • Concurrency is part of the language • Any function can become a goroutine • Goroutines run concurrently, communicate through channels • Select waits for communication on any of a set of channels
MongoDB
Why Another Database? • Databases are slow • Relational structures don’t fit well with modern programming (ORMs) • Databases don’t scale well
MongoDB is Fast • Written in C++ • Extensive use of memory-mapped files 
 i.e. read-through write-through memory caching. • Runs nearly everywhere • Data serialized as BSON (fast parsing) • Full support for primary & secondary indexes • Document model = less work
MongoDB is Friendly • Ad Hoc queries • Real time aggregation • Rich query capabilities • Traditionally consistent • Geospatial features • Support for most programming languages • Flexible schema
MongoDB is “Web Scale” • Built in sharding support distributes data across many nodes • MongoS intelligently routes to the correct nodes • Aggregation done in parallel across nodes
Document Database • Not for .PDF & .DOC files • A document is essentially an associative array • Document == JSON object • Document == PHP Array • Document == Python Dict • Document == Ruby Hash • etc
Data Serialization • Applications need persistant data • The process of translating data structures into a format that can be stored • Ideal format accessible from many languages
BSON • Inspired by JSON • Cross language binary serialization format • Optimized for scanning • Support for richer types
MongoDB & Go
Go’s Data Types • Go uses strict & static typing • 2 Types are similar to a BSON document • Struct • Map
bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON Data: "%x00x00x00x02name x00x04x00x00x00Bob x00tbirthdayx00x80rx97| ^x00x00x00x00"
 ! Person: {Bob 2014-07-21 18:00:00 -0500 EST}
! type Project struct { Name string `bson:"name"` ImportPath string `bson:"importPath"` } project := Project{name, path} ! ! ! project := map[string]string{"name": name, "importPath": path} ! ! ! project := bson.D{{"name", name}, {"importPath", path}} Equal After Marshaling Struct Custom Map Document Slice
mgo (mango) • Pure Go • Created in late 2010 
 ("Where do I put my Go data?") • Adopted by Canonical and MongoDB Inc. itself • Sponsored by MongoDB Inc. from late 2011
Connecting
• Same interface for server, replica set, or shard • Driver discovers and maintains topology • Server added/removed, failovers, response times, etc Connecting session, err := mgo.Dial("localhost") if err != nil { return err }
• Sessions are lightweight • Sessions are copied (settings preserved) • Single management goroutine for all copied sessions Sessions func (s *Server) handle(w http.ResponseWriter, r *http.Request) { session := s.session.Copy() defer session.Close() // ... handle request ... }
• Saves typing • Uses the same session over and over Convenient Access projects := session.DB("OSCON").C("projects")
Writing
type Project struct { Name string `bson:"name,omitempty"` ImportPath string `bson:"importPath,omitempty"` } Defining Our Own Type
var projectList = []Project{ {"gocheck", "gopkg.in/check.v1"}, {"qml", "gopkg.in/qml.v0"}, {"pipe", "gopkg.in/pipe.v2"}, {"yaml", "gopkg.in/yaml.v1"}, } ! for _, project := range projectList { err := projects.Insert(project) if err != nil { return err } } fmt.Println("Okay!") Insert Okay!
type M map[string]interface{} ! change := M{"$set": Project{ImportPath: "gopkg.in/ qml.v1"}} ! err = projects.Update(Project{Name: "qml"}, change) if err != nil { return err } ! fmt.Println("Done!") Update Done!
Querying
var project Project ! err = projects.Find(Project{Name: "qml"}).One(&project) if err != nil { return err } ! fmt.Printf("Project: %vn", project) Find Project: 
 {qml gopkg.in/qml.v0}
iter := projects.Find(nil).Iter() ! var project Project for iter.Next(&project) { fmt.Printf("Project: %vn", project) } ! return iter.Err() Iterate Project: {gocheck gopkg.in/check.v1} Project: {qml gopkg.in/qml.v0} Project: {pipe gopkg.in/pipe.v2} Project: {yaml gopkg.in/yaml.v1}
m := map[string]interface{}{ "name": "godep", "tags": []string{"tool", "dependency"}, "contact": bson.M{ "name": "Keith Rarick", "email": "kr@nospam.com", }, } ! err = projects.Insert(m) if err != nil { return err } fmt.Println("Okay!") Nesting Okay!
type Contact struct { Name string Email string } ! type Project struct { Name string Tags []string `bson:",omitempty"` Contact Contact `bson:",omitempty"` } ! err = projects.Find(Project{Name: "godep"}).One(&project) if err != nil { return err } ! pretty.Println("Project:", project) Nesting II Project: main.Project{ Name: "godep", Tags: {"tool", "dependency"}, Contact: {Name:"Keith Rarick", Email:"kr@XZY.com"}, }
• Compound • List indexing (think tag lists) • Geospatial • Dense or sparse • Full-text searching Indexing // Root field err = projects.EnsureIndexKey("name") ... ! // Nested field err = projects.EnsureIndexKey("author.email") ...
Concurrency
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent Project: {qml gopkg.in/qml.v1} Project: {gocheck gopkg.in/ check.v1}
• Find 1 issued
 • Doc 1 returned • Find 2 issued
 • Doc 2 returned A Common Approach Find 1 Find 2 DB } }
• Find 1 issued • Find 2 issued
 • Doc 1 returned • Doc 2 returned Concurrent Queries Find 1 Find 2 DB } }
• Loads 200 results at a time • Loads next batch with (0.25 * 200) results left to process Concurrent Loading session.SetBatch(200) session.SetPrefetch(0.25) ! for iter.Next(&result) { ... }
• Each Copy uses a different connection • Closing session returns socket to the pool • defer runs at end of function Handler With Session Copy func (s *Server) handle(w http.ResponseWriter, r *http.Request) { session := s.session.Copy() defer session.Close() ! // ... handle request ... }
• Shares a single connection • Still quite efficient thanks to concurrent capabilities of go + mgo Handler With Single Session func (s *Server) handle(w http.ResponseWriter, r *http.Request) { session := s.session ! // ... handle request ... }
GridFS
GridFS • Not quite a file system • Really useful for local file storage • A convention, not a feature • Supported by all drivers • Fully replicated, sharded file storage
gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS ! Wrote 470386961 bytes in 7.0s
Full Featured
Features • Transactions (mgo/txn experiment) • Aggregation pipelines • Full-text search • Geospatial support • Hadoop
In Conclusion
Getting Started • 1. Install MongoDB • 2. go get gopkg.in/mgo.v2 • 3. Start small • 4. Build something great
Learning More • MongoDB Manual • Effective Go • labix.org/mgo
Workshop Using mgo on spf13.com
on spf13.com
• @spf13 • Author of Hugo, Cobra, Viper & More • Chief Developer Advocate for MongoDB • Gopher Thank You
Ad

Recommended

7 Common mistakes in Go and when to avoid them
7 Common mistakes in Go and when to avoid them7 Common mistakes in Go and when to avoid them
7 Common mistakes in Go and when to avoid them
Steven Francia
 
7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)
Steven Francia
 
Getting Started with Go
Getting Started with GoGetting Started with Go
Getting Started with Go
Steven Francia
 
Go for Object Oriented Programmers or Object Oriented Programming without Obj...
Go for Object Oriented Programmers or Object Oriented Programming without Obj...Go for Object Oriented Programmers or Object Oriented Programming without Obj...
Go for Object Oriented Programmers or Object Oriented Programming without Obj...
Steven Francia
 
Building Awesome CLI apps in Go
Building Awesome CLI apps in GoBuilding Awesome CLI apps in Go
Building Awesome CLI apps in Go
Steven Francia
 
Build your first MongoDB App in Ruby @ StrangeLoop 2013
Build your first MongoDB App in Ruby @ StrangeLoop 2013Build your first MongoDB App in Ruby @ StrangeLoop 2013
Build your first MongoDB App in Ruby @ StrangeLoop 2013
Steven Francia
 
Happy Go Programming Part 1
Happy Go Programming Part 1Happy Go Programming Part 1
Happy Go Programming Part 1
Lin Yo-An
 
Mongo db - How we use Go and MongoDB by Sam Helman
Mongo db - How we use Go and MongoDB by Sam HelmanMongo db - How we use Go and MongoDB by Sam Helman
Mongo db - How we use Go and MongoDB by Sam Helman
Hakka Labs
 
Happy Go Programming
Happy Go ProgrammingHappy Go Programming
Happy Go Programming
Lin Yo-An
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in Go
Amr Hassan
 
JRuby with Java Code in Data Processing World
JRuby with Java Code in Data Processing WorldJRuby with Java Code in Data Processing World
JRuby with Java Code in Data Processing World
SATOSHI TAGOMORI
 
Golang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyGolang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war story
Aerospike
 
Elegant concurrency
Elegant concurrencyElegant concurrency
Elegant concurrency
Mosky Liu
 
Vim Script Programming
Vim Script ProgrammingVim Script Programming
Vim Script Programming
Lin Yo-An
 
Php’s guts
Php’s gutsPhp’s guts
Php’s guts
Elizabeth Smith
 
Fluentd v0.14 Plugin API Details
Fluentd v0.14 Plugin API DetailsFluentd v0.14 Plugin API Details
Fluentd v0.14 Plugin API Details
SATOSHI TAGOMORI
 
Fluentd meetup in japan
Fluentd meetup in japanFluentd meetup in japan
Fluentd meetup in japan
Treasure Data, Inc.
 
Using Logstash, elasticsearch & kibana
Using Logstash, elasticsearch & kibanaUsing Logstash, elasticsearch & kibana
Using Logstash, elasticsearch & kibana
Alejandro E Brito Monedero
 
Fluentd unified logging layer
Fluentd unified logging layerFluentd unified logging layer
Fluentd unified logging layer
Kiyoto Tamura
 
JRuby: Pushing the Java Platform Further
JRuby: Pushing the Java Platform FurtherJRuby: Pushing the Java Platform Further
JRuby: Pushing the Java Platform Further
Charles Nutter
 
Fluentd at HKOScon
Fluentd at HKOSconFluentd at HKOScon
Fluentd at HKOScon
N Masahiro
 
Learning Python from Data
Learning Python from DataLearning Python from Data
Learning Python from Data
Mosky Liu
 
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Charles Nutter
 
Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12
N Masahiro
 
Fluentd meetup #2
Fluentd meetup #2Fluentd meetup #2
Fluentd meetup #2
Treasure Data, Inc.
 
Modern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real WorldModern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real World
SATOSHI TAGOMORI
 
Logstash-Elasticsearch-Kibana
Logstash-Elasticsearch-KibanaLogstash-Elasticsearch-Kibana
Logstash-Elasticsearch-Kibana
dknx01
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web Development
Cheng-Yi Yu
 
The Future of the Operating System - Keynote LinuxCon 2015
The Future of the Operating System - Keynote LinuxCon 2015The Future of the Operating System - Keynote LinuxCon 2015
The Future of the Operating System - Keynote LinuxCon 2015
Steven Francia
 
What every successful open source project needs
What every successful open source project needsWhat every successful open source project needs
What every successful open source project needs
Steven Francia
 
Ad

More Related Content

What's hot (20)

Happy Go Programming
Happy Go ProgrammingHappy Go Programming
Happy Go Programming
Lin Yo-An
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in Go
Amr Hassan
 
JRuby with Java Code in Data Processing World
JRuby with Java Code in Data Processing WorldJRuby with Java Code in Data Processing World
JRuby with Java Code in Data Processing World
SATOSHI TAGOMORI
 
Golang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyGolang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war story
Aerospike
 
Elegant concurrency
Elegant concurrencyElegant concurrency
Elegant concurrency
Mosky Liu
 
Vim Script Programming
Vim Script ProgrammingVim Script Programming
Vim Script Programming
Lin Yo-An
 
Php’s guts
Php’s gutsPhp’s guts
Php’s guts
Elizabeth Smith
 
Fluentd v0.14 Plugin API Details
Fluentd v0.14 Plugin API DetailsFluentd v0.14 Plugin API Details
Fluentd v0.14 Plugin API Details
SATOSHI TAGOMORI
 
Fluentd meetup in japan
Fluentd meetup in japanFluentd meetup in japan
Fluentd meetup in japan
Treasure Data, Inc.
 
Using Logstash, elasticsearch & kibana
Using Logstash, elasticsearch & kibanaUsing Logstash, elasticsearch & kibana
Using Logstash, elasticsearch & kibana
Alejandro E Brito Monedero
 
Fluentd unified logging layer
Fluentd unified logging layerFluentd unified logging layer
Fluentd unified logging layer
Kiyoto Tamura
 
JRuby: Pushing the Java Platform Further
JRuby: Pushing the Java Platform FurtherJRuby: Pushing the Java Platform Further
JRuby: Pushing the Java Platform Further
Charles Nutter
 
Fluentd at HKOScon
Fluentd at HKOSconFluentd at HKOScon
Fluentd at HKOScon
N Masahiro
 
Learning Python from Data
Learning Python from DataLearning Python from Data
Learning Python from Data
Mosky Liu
 
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Charles Nutter
 
Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12
N Masahiro
 
Fluentd meetup #2
Fluentd meetup #2Fluentd meetup #2
Fluentd meetup #2
Treasure Data, Inc.
 
Modern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real WorldModern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real World
SATOSHI TAGOMORI
 
Logstash-Elasticsearch-Kibana
Logstash-Elasticsearch-KibanaLogstash-Elasticsearch-Kibana
Logstash-Elasticsearch-Kibana
dknx01
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web Development
Cheng-Yi Yu
 
Happy Go Programming
Happy Go ProgrammingHappy Go Programming
Happy Go Programming
Lin Yo-An
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in Go
Amr Hassan
 
JRuby with Java Code in Data Processing World
JRuby with Java Code in Data Processing WorldJRuby with Java Code in Data Processing World
JRuby with Java Code in Data Processing World
SATOSHI TAGOMORI
 
Golang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyGolang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war story
Aerospike
 
Elegant concurrency
Elegant concurrencyElegant concurrency
Elegant concurrency
Mosky Liu
 
Vim Script Programming
Vim Script ProgrammingVim Script Programming
Vim Script Programming
Lin Yo-An
 
Php’s guts
Php’s gutsPhp’s guts
Php’s guts
Elizabeth Smith
 
Fluentd v0.14 Plugin API Details
Fluentd v0.14 Plugin API DetailsFluentd v0.14 Plugin API Details
Fluentd v0.14 Plugin API Details
SATOSHI TAGOMORI
 
Fluentd meetup in japan
Fluentd meetup in japanFluentd meetup in japan
Fluentd meetup in japan
Treasure Data, Inc.
 
Using Logstash, elasticsearch & kibana
Using Logstash, elasticsearch & kibanaUsing Logstash, elasticsearch & kibana
Using Logstash, elasticsearch & kibana
Alejandro E Brito Monedero
 
Fluentd unified logging layer
Fluentd unified logging layerFluentd unified logging layer
Fluentd unified logging layer
Kiyoto Tamura
 
JRuby: Pushing the Java Platform Further
JRuby: Pushing the Java Platform FurtherJRuby: Pushing the Java Platform Further
JRuby: Pushing the Java Platform Further
Charles Nutter
 
Fluentd at HKOScon
Fluentd at HKOSconFluentd at HKOScon
Fluentd at HKOScon
N Masahiro
 
Learning Python from Data
Learning Python from DataLearning Python from Data
Learning Python from Data
Mosky Liu
 
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Charles Nutter
 
Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12
N Masahiro
 
Fluentd meetup #2
Fluentd meetup #2Fluentd meetup #2
Fluentd meetup #2
Treasure Data, Inc.
 
Modern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real WorldModern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real World
SATOSHI TAGOMORI
 
Logstash-Elasticsearch-Kibana
Logstash-Elasticsearch-KibanaLogstash-Elasticsearch-Kibana
Logstash-Elasticsearch-Kibana
dknx01
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web Development
Cheng-Yi Yu
 

Viewers also liked (7)

The Future of the Operating System - Keynote LinuxCon 2015
The Future of the Operating System - Keynote LinuxCon 2015The Future of the Operating System - Keynote LinuxCon 2015
The Future of the Operating System - Keynote LinuxCon 2015
Steven Francia
 
What every successful open source project needs
What every successful open source project needsWhat every successful open source project needs
What every successful open source project needs
Steven Francia
 
Big data for the rest of us
Big data for the rest of usBig data for the rest of us
Big data for the rest of us
Steven Francia
 
MongoDB, Hadoop and humongous data - MongoSV 2012
MongoDB, Hadoop and humongous data - MongoSV 2012MongoDB, Hadoop and humongous data - MongoSV 2012
MongoDB, Hadoop and humongous data - MongoSV 2012
Steven Francia
 
OSCON 2012 MongoDB Tutorial
OSCON 2012 MongoDB TutorialOSCON 2012 MongoDB Tutorial
OSCON 2012 MongoDB Tutorial
Steven Francia
 
A Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to GoA Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to Go
Matt Stine
 
From SIP to WebRTC and vice versa
From SIP to WebRTC and vice versaFrom SIP to WebRTC and vice versa
From SIP to WebRTC and vice versa
Saúl Ibarra Corretgé
 
The Future of the Operating System - Keynote LinuxCon 2015
The Future of the Operating System - Keynote LinuxCon 2015The Future of the Operating System - Keynote LinuxCon 2015
The Future of the Operating System - Keynote LinuxCon 2015
Steven Francia
 
What every successful open source project needs
What every successful open source project needsWhat every successful open source project needs
What every successful open source project needs
Steven Francia
 
Big data for the rest of us
Big data for the rest of usBig data for the rest of us
Big data for the rest of us
Steven Francia
 
MongoDB, Hadoop and humongous data - MongoSV 2012
MongoDB, Hadoop and humongous data - MongoSV 2012MongoDB, Hadoop and humongous data - MongoSV 2012
MongoDB, Hadoop and humongous data - MongoSV 2012
Steven Francia
 
OSCON 2012 MongoDB Tutorial
OSCON 2012 MongoDB TutorialOSCON 2012 MongoDB Tutorial
OSCON 2012 MongoDB Tutorial
Steven Francia
 
A Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to GoA Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to Go
Matt Stine
 
From SIP to WebRTC and vice versa
From SIP to WebRTC and vice versaFrom SIP to WebRTC and vice versa
From SIP to WebRTC and vice versa
Saúl Ibarra Corretgé
 
Ad

Similar to Painless Data Storage with MongoDB & Go (20)

An Introduction to Go
An Introduction to GoAn Introduction to Go
An Introduction to Go
Cloudflare
 
Go from a PHP Perspective
Go from a PHP PerspectiveGo from a PHP Perspective
Go from a PHP Perspective
Barry Jones
 
MongoDB at ZPUGDC
MongoDB at ZPUGDCMongoDB at ZPUGDC
MongoDB at ZPUGDC
Mike Dirolf
 
#Pharo Days 2016 Data Formats and Protocols
#Pharo Days 2016 Data Formats and Protocols#Pharo Days 2016 Data Formats and Protocols
#Pharo Days 2016 Data Formats and Protocols
Philippe Back
 
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeAllura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Rick Copeland
 
Barcelona MUG MongoDB + Hadoop Presentation
Barcelona MUG MongoDB + Hadoop PresentationBarcelona MUG MongoDB + Hadoop Presentation
Barcelona MUG MongoDB + Hadoop Presentation
Norberto Leite
 
Go for Rubyists
Go for RubyistsGo for Rubyists
Go for Rubyists
tchandy
 
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rick Copeland
 
Node js
Node jsNode js
Node js
hazzaz
 
Monitoring and Debugging your Live Applications
Monitoring and Debugging your Live ApplicationsMonitoring and Debugging your Live Applications
Monitoring and Debugging your Live Applications
Robert Coup
 
AMD - Why, What and How
AMD - Why, What and HowAMD - Why, What and How
AMD - Why, What and How
Mike Wilcox
 
Conexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDays
Conexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDaysConexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDays
Conexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDays
CAPSiDE
 
Unleash your inner console cowboy
Unleash your inner console cowboyUnleash your inner console cowboy
Unleash your inner console cowboy
Kenneth Geisshirt
 
Inroduction to golang
Inroduction to golangInroduction to golang
Inroduction to golang
Yoni Davidson
 
Code for Startup MVP (Ruby on Rails) Session 2
Code for Startup MVP (Ruby on Rails) Session 2Code for Startup MVP (Ruby on Rails) Session 2
Code for Startup MVP (Ruby on Rails) Session 2
Henry S
 
MongoDB Munich 2012: MongoDB for official documents in Bavaria
MongoDB Munich 2012: MongoDB for official documents in BavariaMongoDB Munich 2012: MongoDB for official documents in Bavaria
MongoDB Munich 2012: MongoDB for official documents in Bavaria
MongoDB
 
Practical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.jsPractical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.js
async_io
 
BreizhCamp 2013 - Pimp my backend
BreizhCamp 2013 - Pimp my backendBreizhCamp 2013 - Pimp my backend
BreizhCamp 2013 - Pimp my backend
Horacio Gonzalez
 
Asynchronous I/O in NodeJS - new standard or challenges?
Asynchronous I/O in NodeJS - new standard or challenges?Asynchronous I/O in NodeJS - new standard or challenges?
Asynchronous I/O in NodeJS - new standard or challenges?
Dinh Pham
 
MongoDB and Ruby on Rails
MongoDB and Ruby on RailsMongoDB and Ruby on Rails
MongoDB and Ruby on Rails
rfischer20
 
An Introduction to Go
An Introduction to GoAn Introduction to Go
An Introduction to Go
Cloudflare
 
Go from a PHP Perspective
Go from a PHP PerspectiveGo from a PHP Perspective
Go from a PHP Perspective
Barry Jones
 
MongoDB at ZPUGDC
MongoDB at ZPUGDCMongoDB at ZPUGDC
MongoDB at ZPUGDC
Mike Dirolf
 
#Pharo Days 2016 Data Formats and Protocols
#Pharo Days 2016 Data Formats and Protocols#Pharo Days 2016 Data Formats and Protocols
#Pharo Days 2016 Data Formats and Protocols
Philippe Back
 
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeAllura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Rick Copeland
 
Barcelona MUG MongoDB + Hadoop Presentation
Barcelona MUG MongoDB + Hadoop PresentationBarcelona MUG MongoDB + Hadoop Presentation
Barcelona MUG MongoDB + Hadoop Presentation
Norberto Leite
 
Go for Rubyists
Go for RubyistsGo for Rubyists
Go for Rubyists
tchandy
 
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rick Copeland
 
Node js
Node jsNode js
Node js
hazzaz
 
Monitoring and Debugging your Live Applications
Monitoring and Debugging your Live ApplicationsMonitoring and Debugging your Live Applications
Monitoring and Debugging your Live Applications
Robert Coup
 
AMD - Why, What and How
AMD - Why, What and HowAMD - Why, What and How
AMD - Why, What and How
Mike Wilcox
 
Conexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDays
Conexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDaysConexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDays
Conexión de MongoDB con Hadoop - Luis Alberto Giménez - CAPSiDE #DevOSSAzureDays
CAPSiDE
 
Unleash your inner console cowboy
Unleash your inner console cowboyUnleash your inner console cowboy
Unleash your inner console cowboy
Kenneth Geisshirt
 
Inroduction to golang
Inroduction to golangInroduction to golang
Inroduction to golang
Yoni Davidson
 
Code for Startup MVP (Ruby on Rails) Session 2
Code for Startup MVP (Ruby on Rails) Session 2Code for Startup MVP (Ruby on Rails) Session 2
Code for Startup MVP (Ruby on Rails) Session 2
Henry S
 
MongoDB Munich 2012: MongoDB for official documents in Bavaria
MongoDB Munich 2012: MongoDB for official documents in BavariaMongoDB Munich 2012: MongoDB for official documents in Bavaria
MongoDB Munich 2012: MongoDB for official documents in Bavaria
MongoDB
 
Practical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.jsPractical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.js
async_io
 
BreizhCamp 2013 - Pimp my backend
BreizhCamp 2013 - Pimp my backendBreizhCamp 2013 - Pimp my backend
BreizhCamp 2013 - Pimp my backend
Horacio Gonzalez
 
Asynchronous I/O in NodeJS - new standard or challenges?
Asynchronous I/O in NodeJS - new standard or challenges?Asynchronous I/O in NodeJS - new standard or challenges?
Asynchronous I/O in NodeJS - new standard or challenges?
Dinh Pham
 
MongoDB and Ruby on Rails
MongoDB and Ruby on RailsMongoDB and Ruby on Rails
MongoDB and Ruby on Rails
rfischer20
 
Ad

More from Steven Francia (19)

State of the Gopher Nation - Golang - August 2017
State of the Gopher Nation - Golang - August 2017State of the Gopher Nation - Golang - August 2017
State of the Gopher Nation - Golang - August 2017
Steven Francia
 
Modern Database Systems (for Genealogy)
Modern Database Systems (for Genealogy)Modern Database Systems (for Genealogy)
Modern Database Systems (for Genealogy)
Steven Francia
 
Introduction to MongoDB and Hadoop
Introduction to MongoDB and HadoopIntroduction to MongoDB and Hadoop
Introduction to MongoDB and Hadoop
Steven Francia
 
Future of data
Future of dataFuture of data
Future of data
Steven Francia
 
Replication, Durability, and Disaster Recovery
Replication, Durability, and Disaster RecoveryReplication, Durability, and Disaster Recovery
Replication, Durability, and Disaster Recovery
Steven Francia
 
Multi Data Center Strategies
Multi Data Center StrategiesMulti Data Center Strategies
Multi Data Center Strategies
Steven Francia
 
NoSQL databases and managing big data
NoSQL databases and managing big dataNoSQL databases and managing big data
NoSQL databases and managing big data
Steven Francia
 
MongoDB, Hadoop and Humongous Data
MongoDB, Hadoop and Humongous DataMongoDB, Hadoop and Humongous Data
MongoDB, Hadoop and Humongous Data
Steven Francia
 
MongoDB and hadoop
MongoDB and hadoopMongoDB and hadoop
MongoDB and hadoop
Steven Francia
 
MongoDB for Genealogy
MongoDB for GenealogyMongoDB for Genealogy
MongoDB for Genealogy
Steven Francia
 
Hybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS ApplicationsHybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS Applications
Steven Francia
 
Building your first application w/mongoDB MongoSV2011
Building your first application w/mongoDB MongoSV2011Building your first application w/mongoDB MongoSV2011
Building your first application w/mongoDB MongoSV2011
Steven Francia
 
MongoDB, E-commerce and Transactions
MongoDB, E-commerce and TransactionsMongoDB, E-commerce and Transactions
MongoDB, E-commerce and Transactions
Steven Francia
 
MongoDB, PHP and the cloud - php cloud summit 2011
MongoDB, PHP and the cloud - php cloud summit 2011MongoDB, PHP and the cloud - php cloud summit 2011
MongoDB, PHP and the cloud - php cloud summit 2011
Steven Francia
 
MongoDB and PHP ZendCon 2011
MongoDB and PHP ZendCon 2011MongoDB and PHP ZendCon 2011
MongoDB and PHP ZendCon 2011
Steven Francia
 
MongoDB
MongoDBMongoDB
MongoDB
Steven Francia
 
Blending MongoDB and RDBMS for ecommerce
Blending MongoDB and RDBMS for ecommerceBlending MongoDB and RDBMS for ecommerce
Blending MongoDB and RDBMS for ecommerce
Steven Francia
 
Augmenting RDBMS with MongoDB for ecommerce
Augmenting RDBMS with MongoDB for ecommerceAugmenting RDBMS with MongoDB for ecommerce
Augmenting RDBMS with MongoDB for ecommerce
Steven Francia
 
MongoDB and Ecommerce : A perfect combination
MongoDB and Ecommerce : A perfect combinationMongoDB and Ecommerce : A perfect combination
MongoDB and Ecommerce : A perfect combination
Steven Francia
 
State of the Gopher Nation - Golang - August 2017
State of the Gopher Nation - Golang - August 2017State of the Gopher Nation - Golang - August 2017
State of the Gopher Nation - Golang - August 2017
Steven Francia
 
Modern Database Systems (for Genealogy)
Modern Database Systems (for Genealogy)Modern Database Systems (for Genealogy)
Modern Database Systems (for Genealogy)
Steven Francia
 
Introduction to MongoDB and Hadoop
Introduction to MongoDB and HadoopIntroduction to MongoDB and Hadoop
Introduction to MongoDB and Hadoop
Steven Francia
 
Future of data
Future of dataFuture of data
Future of data
Steven Francia
 
Replication, Durability, and Disaster Recovery
Replication, Durability, and Disaster RecoveryReplication, Durability, and Disaster Recovery
Replication, Durability, and Disaster Recovery
Steven Francia
 
Multi Data Center Strategies
Multi Data Center StrategiesMulti Data Center Strategies
Multi Data Center Strategies
Steven Francia
 
NoSQL databases and managing big data
NoSQL databases and managing big dataNoSQL databases and managing big data
NoSQL databases and managing big data
Steven Francia
 
MongoDB, Hadoop and Humongous Data
MongoDB, Hadoop and Humongous DataMongoDB, Hadoop and Humongous Data
MongoDB, Hadoop and Humongous Data
Steven Francia
 
MongoDB and hadoop
MongoDB and hadoopMongoDB and hadoop
MongoDB and hadoop
Steven Francia
 
MongoDB for Genealogy
MongoDB for GenealogyMongoDB for Genealogy
MongoDB for Genealogy
Steven Francia
 
Hybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS ApplicationsHybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS Applications
Steven Francia
 
Building your first application w/mongoDB MongoSV2011
Building your first application w/mongoDB MongoSV2011Building your first application w/mongoDB MongoSV2011
Building your first application w/mongoDB MongoSV2011
Steven Francia
 
MongoDB, E-commerce and Transactions
MongoDB, E-commerce and TransactionsMongoDB, E-commerce and Transactions
MongoDB, E-commerce and Transactions
Steven Francia
 
MongoDB, PHP and the cloud - php cloud summit 2011
MongoDB, PHP and the cloud - php cloud summit 2011MongoDB, PHP and the cloud - php cloud summit 2011
MongoDB, PHP and the cloud - php cloud summit 2011
Steven Francia
 
MongoDB and PHP ZendCon 2011
MongoDB and PHP ZendCon 2011MongoDB and PHP ZendCon 2011
MongoDB and PHP ZendCon 2011
Steven Francia
 
MongoDB
MongoDBMongoDB
MongoDB
Steven Francia
 
Blending MongoDB and RDBMS for ecommerce
Blending MongoDB and RDBMS for ecommerceBlending MongoDB and RDBMS for ecommerce
Blending MongoDB and RDBMS for ecommerce
Steven Francia
 
Augmenting RDBMS with MongoDB for ecommerce
Augmenting RDBMS with MongoDB for ecommerceAugmenting RDBMS with MongoDB for ecommerce
Augmenting RDBMS with MongoDB for ecommerce
Steven Francia
 
MongoDB and Ecommerce : A perfect combination
MongoDB and Ecommerce : A perfect combinationMongoDB and Ecommerce : A perfect combination
MongoDB and Ecommerce : A perfect combination
Steven Francia
 

Recently uploaded (20)

Shoehorning dependency injection into a FP language, what does it take?
Shoehorning dependency injection into a FP language, what does it take?Shoehorning dependency injection into a FP language, what does it take?
Shoehorning dependency injection into a FP language, what does it take?
Eric Torreborre
 
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Maarten Verwaest
 
GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...
GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...
GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...
James Anderson
 
IT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information TechnologyIT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information Technology
SHEHABALYAMANI
 
AsyncAPI v3 : Streamlining Event-Driven API Design
AsyncAPI v3 : Streamlining Event-Driven API DesignAsyncAPI v3 : Streamlining Event-Driven API Design
AsyncAPI v3 : Streamlining Event-Driven API Design
leonid54
 
AI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of DocumentsAI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of Documents
UiPathCommunity
 
fennec fox optimization algorithm for optimal solution
fennec fox optimization algorithm for optimal solutionfennec fox optimization algorithm for optimal solution
fennec fox optimization algorithm for optimal solution
shallal2
 
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Wonjun Hwang
 
Reimagine How You and Your Team Work with Microsoft 365 Copilot.pptx
Reimagine How You and Your Team Work with Microsoft 365 Copilot.pptxReimagine How You and Your Team Work with Microsoft 365 Copilot.pptx
Reimagine How You and Your Team Work with Microsoft 365 Copilot.pptx
John Moore
 
Slack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teamsSlack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teams
Nacho Cougil
 
Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025
Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025
Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025
João Esperancinha
 
Artificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptxArtificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptx
03ANMOLCHAURASIYA
 
Mastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B LandscapeMastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B Landscape
marketing943205
 
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier VroomAI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
UXPA Boston
 
Developing System Infrastructure Design Plan.pptx
Developing System Infrastructure Design Plan.pptxDeveloping System Infrastructure Design Plan.pptx
Developing System Infrastructure Design Plan.pptx
wondimagegndesta
 
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Safe Software
 
May Patch Tuesday
May Patch TuesdayMay Patch Tuesday
May Patch Tuesday
Ivanti
 
Dark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanizationDark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanization
Jakub Šimek
 
The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...
The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...
The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...
SOFTTECHHUB
 
Smart Investments Leveraging Agentic AI for Real Estate Success.pptx
Smart Investments Leveraging Agentic AI for Real Estate Success.pptxSmart Investments Leveraging Agentic AI for Real Estate Success.pptx
Smart Investments Leveraging Agentic AI for Real Estate Success.pptx
Seasia Infotech
 
Shoehorning dependency injection into a FP language, what does it take?
Shoehorning dependency injection into a FP language, what does it take?Shoehorning dependency injection into a FP language, what does it take?
Shoehorning dependency injection into a FP language, what does it take?
Eric Torreborre
 
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Maarten Verwaest
 
GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...
GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...
GDG Cloud Southlake #42: Suresh Mathew: Autonomous Resource Optimization: How...
James Anderson
 
IT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information TechnologyIT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information Technology
SHEHABALYAMANI
 
AsyncAPI v3 : Streamlining Event-Driven API Design
AsyncAPI v3 : Streamlining Event-Driven API DesignAsyncAPI v3 : Streamlining Event-Driven API Design
AsyncAPI v3 : Streamlining Event-Driven API Design
leonid54
 
AI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of DocumentsAI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of Documents
UiPathCommunity
 
fennec fox optimization algorithm for optimal solution
fennec fox optimization algorithm for optimal solutionfennec fox optimization algorithm for optimal solution
fennec fox optimization algorithm for optimal solution
shallal2
 
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Wonjun Hwang
 
Reimagine How You and Your Team Work with Microsoft 365 Copilot.pptx
Reimagine How You and Your Team Work with Microsoft 365 Copilot.pptxReimagine How You and Your Team Work with Microsoft 365 Copilot.pptx
Reimagine How You and Your Team Work with Microsoft 365 Copilot.pptx
John Moore
 
Slack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teamsSlack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teams
Nacho Cougil
 
Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025
Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025
Could Virtual Threads cast away the usage of Kotlin Coroutines - DevoxxUK2025
João Esperancinha
 
Artificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptxArtificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptx
03ANMOLCHAURASIYA
 
Mastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B LandscapeMastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B Landscape
marketing943205
 
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier VroomAI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
UXPA Boston
 
Developing System Infrastructure Design Plan.pptx
Developing System Infrastructure Design Plan.pptxDeveloping System Infrastructure Design Plan.pptx
Developing System Infrastructure Design Plan.pptx
wondimagegndesta
 
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Safe Software
 
May Patch Tuesday
May Patch TuesdayMay Patch Tuesday
May Patch Tuesday
Ivanti
 
Dark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanizationDark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanization
Jakub Šimek
 
The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...
The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...
The No-Code Way to Build a Marketing Team with One AI Agent (Download the n8n...
SOFTTECHHUB
 
Smart Investments Leveraging Agentic AI for Real Estate Success.pptx
Smart Investments Leveraging Agentic AI for Real Estate Success.pptxSmart Investments Leveraging Agentic AI for Real Estate Success.pptx
Smart Investments Leveraging Agentic AI for Real Estate Success.pptx
Seasia Infotech
 

Painless Data Storage with MongoDB & Go

  • 2. • Author of Hugo, Cobra, Viper & More • Chief Developer Advocate for MongoDB • Gopher @spf13
  • 4. Why Another Language? • Software is slow • Sofware is hard to write • Software doesn’t scale well
  • 5. Go is Fast • Go execution speed is close to C • Go compile time rivals dynamic interpretation
  • 6. Go is Friendly • Feels like a dynamic language in many ways • Very small core language, easy to remember all of it • Single binary installation, no dependencies • Extensive Tooling & StdLib
  • 7. Go is Concurrent • Concurrency is part of the language • Any function can become a goroutine • Goroutines run concurrently, communicate through channels • Select waits for communication on any of a set of channels
  • 9. Why Another Database? • Databases are slow • Relational structures don’t fit well with modern programming (ORMs) • Databases don’t scale well
  • 10. MongoDB is Fast • Written in C++ • Extensive use of memory-mapped files 
 i.e. read-through write-through memory caching. • Runs nearly everywhere • Data serialized as BSON (fast parsing) • Full support for primary & secondary indexes • Document model = less work
  • 11. MongoDB is Friendly • Ad Hoc queries • Real time aggregation • Rich query capabilities • Traditionally consistent • Geospatial features • Support for most programming languages • Flexible schema
  • 12. MongoDB is “Web Scale” • Built in sharding support distributes data across many nodes • MongoS intelligently routes to the correct nodes • Aggregation done in parallel across nodes
  • 13. Document Database • Not for .PDF & .DOC files • A document is essentially an associative array • Document == JSON object • Document == PHP Array • Document == Python Dict • Document == Ruby Hash • etc
  • 14. Data Serialization • Applications need persistant data • The process of translating data structures into a format that can be stored • Ideal format accessible from many languages
  • 15. BSON • Inspired by JSON • Cross language binary serialization format • Optimized for scanning • Support for richer types
  • 17. Go’s Data Types • Go uses strict & static typing • 2 Types are similar to a BSON document • Struct • Map
  • 18. bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
  • 19. bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
  • 20. bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
  • 21. bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
  • 22. bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON
  • 23. bob := &Person{ Name: "Bob", Birthday: time.Now(), } ! data, err := bson.Marshal(bob) if err != nil { return err } fmt.Printf("Data: %qn", data)
 ! var person Person err = bson.Unmarshal(data, &person) if err != nil { return err } fmt.Printf("Person: %vn", person) Serializing with BSON Data: "%x00x00x00x02name x00x04x00x00x00Bob x00tbirthdayx00x80rx97| ^x00x00x00x00"
 ! Person: {Bob 2014-07-21 18:00:00 -0500 EST}
  • 24. ! type Project struct { Name string `bson:"name"` ImportPath string `bson:"importPath"` } project := Project{name, path} ! ! ! project := map[string]string{"name": name, "importPath": path} ! ! ! project := bson.D{{"name", name}, {"importPath", path}} Equal After Marshaling Struct Custom Map Document Slice
  • 25. mgo (mango) • Pure Go • Created in late 2010 
 ("Where do I put my Go data?") • Adopted by Canonical and MongoDB Inc. itself • Sponsored by MongoDB Inc. from late 2011
  • 27. • Same interface for server, replica set, or shard • Driver discovers and maintains topology • Server added/removed, failovers, response times, etc Connecting session, err := mgo.Dial("localhost") if err != nil { return err }
  • 28. • Sessions are lightweight • Sessions are copied (settings preserved) • Single management goroutine for all copied sessions Sessions func (s *Server) handle(w http.ResponseWriter, r *http.Request) { session := s.session.Copy() defer session.Close() // ... handle request ... }
  • 29. • Saves typing • Uses the same session over and over Convenient Access projects := session.DB("OSCON").C("projects")
  • 31. type Project struct { Name string `bson:"name,omitempty"` ImportPath string `bson:"importPath,omitempty"` } Defining Our Own Type
  • 32. var projectList = []Project{ {"gocheck", "gopkg.in/check.v1"}, {"qml", "gopkg.in/qml.v0"}, {"pipe", "gopkg.in/pipe.v2"}, {"yaml", "gopkg.in/yaml.v1"}, } ! for _, project := range projectList { err := projects.Insert(project) if err != nil { return err } } fmt.Println("Okay!") Insert Okay!
  • 33. type M map[string]interface{} ! change := M{"$set": Project{ImportPath: "gopkg.in/ qml.v1"}} ! err = projects.Update(Project{Name: "qml"}, change) if err != nil { return err } ! fmt.Println("Done!") Update Done!
  • 35. var project Project ! err = projects.Find(Project{Name: "qml"}).One(&project) if err != nil { return err } ! fmt.Printf("Project: %vn", project) Find Project: 
 {qml gopkg.in/qml.v0}
  • 36. iter := projects.Find(nil).Iter() ! var project Project for iter.Next(&project) { fmt.Printf("Project: %vn", project) } ! return iter.Err() Iterate Project: {gocheck gopkg.in/check.v1} Project: {qml gopkg.in/qml.v0} Project: {pipe gopkg.in/pipe.v2} Project: {yaml gopkg.in/yaml.v1}
  • 37. m := map[string]interface{}{ "name": "godep", "tags": []string{"tool", "dependency"}, "contact": bson.M{ "name": "Keith Rarick", "email": "kr@nospam.com", }, } ! err = projects.Insert(m) if err != nil { return err } fmt.Println("Okay!") Nesting Okay!
  • 38. type Contact struct { Name string Email string } ! type Project struct { Name string Tags []string `bson:",omitempty"` Contact Contact `bson:",omitempty"` } ! err = projects.Find(Project{Name: "godep"}).One(&project) if err != nil { return err } ! pretty.Println("Project:", project) Nesting II Project: main.Project{ Name: "godep", Tags: {"tool", "dependency"}, Contact: {Name:"Keith Rarick", Email:"kr@XZY.com"}, }
  • 39. • Compound • List indexing (think tag lists) • Geospatial • Dense or sparse • Full-text searching Indexing // Root field err = projects.EnsureIndexKey("name") ... ! // Nested field err = projects.EnsureIndexKey("author.email") ...
  • 41. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
  • 42. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
  • 43. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
  • 44. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
  • 45. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
  • 46. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
  • 47. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent
  • 48. func f(projects *mgo.Collection, name string, done chan error) { var project Project err := projects.Find(Project{Name: name}).One(&project) if err == nil { fmt.Printf("Project: %vn", project) } done <- err } ! done := make(chan error) ! go f(projects, "qml", done) go f(projects, "gocheck", done) ! if err = firstError(2, done); err != nil { return err } Concurrent Project: {qml gopkg.in/qml.v1} Project: {gocheck gopkg.in/ check.v1}
  • 49. • Find 1 issued
 • Doc 1 returned • Find 2 issued
 • Doc 2 returned A Common Approach Find 1 Find 2 DB } }
  • 50. • Find 1 issued • Find 2 issued
 • Doc 1 returned • Doc 2 returned Concurrent Queries Find 1 Find 2 DB } }
  • 51. • Loads 200 results at a time • Loads next batch with (0.25 * 200) results left to process Concurrent Loading session.SetBatch(200) session.SetPrefetch(0.25) ! for iter.Next(&result) { ... }
  • 52. • Each Copy uses a different connection • Closing session returns socket to the pool • defer runs at end of function Handler With Session Copy func (s *Server) handle(w http.ResponseWriter, r *http.Request) { session := s.session.Copy() defer session.Close() ! // ... handle request ... }
  • 53. • Shares a single connection • Still quite efficient thanks to concurrent capabilities of go + mgo Handler With Single Session func (s *Server) handle(w http.ResponseWriter, r *http.Request) { session := s.session ! // ... handle request ... }
  • 55. GridFS • Not quite a file system • Really useful for local file storage • A convention, not a feature • Supported by all drivers • Fully replicated, sharded file storage
  • 56. gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
  • 57. gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
  • 58. gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
  • 59. gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
  • 60. gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
  • 61. gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS
  • 62. gridfs := session.DB("OSCON").GridFS("fs") ! file, err := gridfs.Create("cd.iso") if err != nil { return err } defer file.Close() ! started := time.Now() ! _, err = io.Copy(file, iso) if err != nil { return err } ! fmt.Printf("Wrote %d bytes in %sn", file.Size(), time.Since(started)) GridFS ! Wrote 470386961 bytes in 7.0s
  • 64. Features • Transactions (mgo/txn experiment) • Aggregation pipelines • Full-text search • Geospatial support • Hadoop
  • 66. Getting Started • 1. Install MongoDB • 2. go get gopkg.in/mgo.v2 • 3. Start small • 4. Build something great
  • 67. Learning More • MongoDB Manual • Effective Go • labix.org/mgo
  • 68. Workshop Using mgo on spf13.com
  • 70. • @spf13 • Author of Hugo, Cobra, Viper & More • Chief Developer Advocate for MongoDB • Gopher Thank You
  翻译：