SlideShare a Scribd company logo
Group Replication In MYSQL 8.0
A walk through
Presented by
VIGNESH PRABHU S
www.mydbops.com info@mydbops.com
About Mydbops
● Founded in 2015, HQ in Bangalore India with 450+ customer base across the globe.
● Mydbops is on Database Consulting with core specialization on MySQL ,MongoDB & PostgreSQL
Administration and Support.
● We have expert team with 30+ certified DBA’s providing full time support and currently managing 300+
servers on cloud.
● Mydbops was created with a motto of developing a Devops model for Database administration offering
24*7 expert remote DBA support.
● We help organisations to architect and scale systems in MySQL/Mongo by implementing the advanced
technologies in industry which are completely open source.
● We are a leading solution provider in the market for all sort of cloud based deployments and management.
Agenda
● Group Replication - Introduction
● Group Replication - Architecture
● Group Replication - Monitoring
● Group Replication - Modes
● Primary Member Change - Cases
● Primary Election Algorithm
● Conditions for using online UDF
● Communication Protocol
● Transaction Consistency
Group Replication - Introduction
● High Availability Solution
● To create a fault tolerant System.
● Supports Server side failover.
● Client side failover is not supported by GR.
● Distributed functionality.
● Based on the GCS protocol and paxos algorithm.
● Split brain issue is avoided by Quorum.
Group Replication - Introduction
Services of Group Replication:-
● Group Membership (VIEW)
● Failure Detection (Detect the failure & update the VIEW)
● Fault-tolerance ( QUORUM condition n = 2 x f + 1. )
● Monitoring ( performance schema tables )
All these Services are built into the GCS Protocol.
Group Replication - Architecture
Group Replication - Architecture
Functionality of Each Component:-
● Capture :Keep track of all the local transactions.
● Applier :Remote transactions.
● Recovery :Manage the distributed recovery.
● Replication :Define the replication protocol & handle the
conflict detection in the group.
● GCS & Xcom :GR Services & communication protocol.
Group Replication - ( Commit )
Group Replication - Monitoring
Tables in Perf schema:-
● replication_group_member_stats ( info about certification process)
● replication_group_members ( info about the group members )
● replication_connection_status ( info about the queue )
● replication_applier_status ( shows state of replication worker
thread )
Replication Channel:-
● group_replication_recovery - ( Channel for the recovery process)
● group_replication_applier - ( transactions from the other members
or from the group )
Group Replication - Monitoring
Group Replication Server States:-
● Online :will accept the connection from the client.
● Recovering :Applying the events from the donor.
● Offline :Member doesn’t belong to any group.
● ERROR :Member is in the group but not working properly.
“group_replication_exit_state_action”
● UNREACHABLE :It is mostly because of the network failure.
Group Replication - Modes
● Based on the variable - (group_replication_single_primary_mode)
● Single Primary Mode
○ One member accept the writes.
○ group_replication_enforce_update_everywhere_checks (off)
● Multi Primary Mode
○ All the members can accept the writes.
○ group_replication_enforce_update_everywhere_checks (ON)
Group Replication - Modes
● From MySQL version 8.0.13 we can change the modes in
online Using the UDF’s
○ group_replication_switch_to_single_primary_mode()
○ group_replication_switch_to_multi_primary_mode()
● From Version 8.0.13 to 8.0.16, no check for the safeguard.
● From Version 8.0.17, GR has safeguard check.
○ (i.e) Lower MySQL version member will become the
Primary.
Primary Member Change - Cases
● Primary leaves the group .
● While Changing the primary member manually using UDF.
● While Changing the multi primary to single primary.
Primary Election Algorithm
First Election Check:-
● Ordering based on MySQL major Version & patch Version.
Second Election Check:-
● Based on lowest member weight. ( group_replication_member_weight ).
Third Election Check:-
● Based on the lowest server UUID.
Conditions for using online UDF
1. All members must be in ONLINE state
2. No members can join in the group during a configuration change.
3. Only one configuration at once.
4. All members must be running MySQL 8.0.13 or higher
Communication Protocol
● From MySQL 8.0.16, Configure different versions of MySQL members.
● Group must use same communication protocol Version.
Reason:-
● From MySQL 5.7.14, allow compression of messages
● From MySQL 8.0.16, allow fragmentation of messages
Functions:-
● group_replication_get_communication_protocol();
● group_replication_set_communication_protocol(“version”);
Transaction Consistency.
● Controlled by group_replication_consistency variable.
○ EVENTUAL
○ BEFORE_ON_PRIMARY_FAILOVER
○ BEFORE ( RO trx wait for preceding trx to apply )
○ AFTER ( RW trx wait for preceding trx to apply in other members )
○ BEFORE_AND_AFTER
Email Us : Info@mydbops.com
Thank you
Ad

More Related Content

What's hot (20)

MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바
NeoClova
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOps
Sveta Smirnova
 
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQLMySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
Olivier DASINI
 
Upgrade from MySQL 5.7 to MySQL 8.0
Upgrade from MySQL 5.7 to MySQL 8.0Upgrade from MySQL 5.7 to MySQL 8.0
Upgrade from MySQL 5.7 to MySQL 8.0
Olivier DASINI
 
MySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software TestMySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software Test
I Goo Lee
 
Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기
NeoClova
 
MySQL High Availability with Group Replication
MySQL High Availability with Group ReplicationMySQL High Availability with Group Replication
MySQL High Availability with Group Replication
Nuno Carvalho
 
MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바
NeoClova
 
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
Severalnines
 
The Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialThe Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication Tutorial
Jean-François Gagné
 
MySQL Group Replication - Ready For Production? (2018-04)
MySQL Group Replication - Ready For Production? (2018-04)MySQL Group Replication - Ready For Production? (2018-04)
MySQL Group Replication - Ready For Production? (2018-04)
Kenny Gryp
 
MySQL User Group NL - MySQL 8
MySQL User Group NL - MySQL 8MySQL User Group NL - MySQL 8
MySQL User Group NL - MySQL 8
Frederic Descamps
 
InnoDb Vs NDB Cluster
InnoDb Vs NDB ClusterInnoDb Vs NDB Cluster
InnoDb Vs NDB Cluster
Mark Swarbrick
 
MySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & OperationsMySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & Operations
Frederic Descamps
 
MySQL Shell for DBAs
MySQL Shell for DBAsMySQL Shell for DBAs
MySQL Shell for DBAs
Frederic Descamps
 
Galera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slidesGalera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slides
Severalnines
 
Almost Perfect Service Discovery and Failover with ProxySQL and Orchestrator
Almost Perfect Service Discovery and Failover with ProxySQL and OrchestratorAlmost Perfect Service Discovery and Failover with ProxySQL and Orchestrator
Almost Perfect Service Discovery and Failover with ProxySQL and Orchestrator
Jean-François Gagné
 
Maxscale_메뉴얼
Maxscale_메뉴얼Maxscale_메뉴얼
Maxscale_메뉴얼
NeoClova
 
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Mydbops
 
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
Jean-François Gagné
 
MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바
NeoClova
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOps
Sveta Smirnova
 
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQLMySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
Olivier DASINI
 
Upgrade from MySQL 5.7 to MySQL 8.0
Upgrade from MySQL 5.7 to MySQL 8.0Upgrade from MySQL 5.7 to MySQL 8.0
Upgrade from MySQL 5.7 to MySQL 8.0
Olivier DASINI
 
MySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software TestMySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software Test
I Goo Lee
 
Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기
NeoClova
 
MySQL High Availability with Group Replication
MySQL High Availability with Group ReplicationMySQL High Availability with Group Replication
MySQL High Availability with Group Replication
Nuno Carvalho
 
MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바
NeoClova
 
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
Severalnines
 
The Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialThe Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication Tutorial
Jean-François Gagné
 
MySQL Group Replication - Ready For Production? (2018-04)
MySQL Group Replication - Ready For Production? (2018-04)MySQL Group Replication - Ready For Production? (2018-04)
MySQL Group Replication - Ready For Production? (2018-04)
Kenny Gryp
 
MySQL User Group NL - MySQL 8
MySQL User Group NL - MySQL 8MySQL User Group NL - MySQL 8
MySQL User Group NL - MySQL 8
Frederic Descamps
 
MySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & OperationsMySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & Operations
Frederic Descamps
 
Galera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slidesGalera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slides
Severalnines
 
Almost Perfect Service Discovery and Failover with ProxySQL and Orchestrator
Almost Perfect Service Discovery and Failover with ProxySQL and OrchestratorAlmost Perfect Service Discovery and Failover with ProxySQL and Orchestrator
Almost Perfect Service Discovery and Failover with ProxySQL and Orchestrator
Jean-François Gagné
 
Maxscale_메뉴얼
Maxscale_메뉴얼Maxscale_메뉴얼
Maxscale_메뉴얼
NeoClova
 
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Mydbops
 
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
Jean-François Gagné
 

Similar to Group Replication in MySQL 8.0 ( A Walk Through ) (20)

Sprint 71
Sprint 71Sprint 71
Sprint 71
ManageIQ
 
AutoDOPandRest
AutoDOPandRestAutoDOPandRest
AutoDOPandRest
Rick van Ek
 
What is new in Galera 4 ?
What is new in Galera 4 ?What is new in Galera 4 ?
What is new in Galera 4 ?
Mydbops
 
InnoDB Cluster Experience (MySQL User Camp)
InnoDB Cluster Experience (MySQL User Camp)InnoDB Cluster Experience (MySQL User Camp)
InnoDB Cluster Experience (MySQL User Camp)
Mydbops
 
Sprint 44 review
Sprint 44 reviewSprint 44 review
Sprint 44 review
ManageIQ
 
Customer Experience: InnoDB Cluster Implementation by PR Karthik
Customer Experience: InnoDB Cluster Implementation by PR KarthikCustomer Experience: InnoDB Cluster Implementation by PR Karthik
Customer Experience: InnoDB Cluster Implementation by PR Karthik
Mysql User Camp
 
Insight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops TeamInsight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Mydbops
 
Introduction to mongo db
Introduction to mongo dbIntroduction to mongo db
Introduction to mongo db
Lawrence Mwai
 
Cloudify 4.2 Webinar - Agility & Control
Cloudify 4.2 Webinar - Agility & ControlCloudify 4.2 Webinar - Agility & Control
Cloudify 4.2 Webinar - Agility & Control
Cloudify Community
 
MySQL Live Migration - Common Scenarios
MySQL Live Migration - Common ScenariosMySQL Live Migration - Common Scenarios
MySQL Live Migration - Common Scenarios
Mydbops
 
Managing 600 instances
Managing 600 instancesManaging 600 instances
Managing 600 instances
Geoffrey Beausire
 
Heart of the SwarmKit: Store, Topology & Object Model
Heart of the SwarmKit: Store, Topology & Object ModelHeart of the SwarmKit: Store, Topology & Object Model
Heart of the SwarmKit: Store, Topology & Object Model
Docker, Inc.
 
Sprint 68
Sprint 68Sprint 68
Sprint 68
ManageIQ
 
MySQL Workbench for DFW Unix Users Group
MySQL Workbench for DFW Unix Users GroupMySQL Workbench for DFW Unix Users Group
MySQL Workbench for DFW Unix Users Group
Dave Stokes
 
SVCC-2014
SVCC-2014SVCC-2014
SVCC-2014
John Brinnand
 
Slack in the Age of Prometheus
Slack in the Age of PrometheusSlack in the Age of Prometheus
Slack in the Age of Prometheus
George Luong
 
Red Hat Advanced Cluster Manager Details
Red Hat Advanced Cluster Manager DetailsRed Hat Advanced Cluster Manager Details
Red Hat Advanced Cluster Manager Details
Sourav Jyoti Das
 
Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3
LibbySchulze
 
Sprint 50 review
Sprint 50 reviewSprint 50 review
Sprint 50 review
ManageIQ
 
PostgreSQL 9.5 - Major Features
PostgreSQL 9.5 - Major FeaturesPostgreSQL 9.5 - Major Features
PostgreSQL 9.5 - Major Features
InMobi Technology
 
What is new in Galera 4 ?
What is new in Galera 4 ?What is new in Galera 4 ?
What is new in Galera 4 ?
Mydbops
 
InnoDB Cluster Experience (MySQL User Camp)
InnoDB Cluster Experience (MySQL User Camp)InnoDB Cluster Experience (MySQL User Camp)
InnoDB Cluster Experience (MySQL User Camp)
Mydbops
 
Sprint 44 review
Sprint 44 reviewSprint 44 review
Sprint 44 review
ManageIQ
 
Customer Experience: InnoDB Cluster Implementation by PR Karthik
Customer Experience: InnoDB Cluster Implementation by PR KarthikCustomer Experience: InnoDB Cluster Implementation by PR Karthik
Customer Experience: InnoDB Cluster Implementation by PR Karthik
Mysql User Camp
 
Insight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops TeamInsight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Insight on MongoDB Change Stream - Abhishek.D, Mydbops Team
Mydbops
 
Introduction to mongo db
Introduction to mongo dbIntroduction to mongo db
Introduction to mongo db
Lawrence Mwai
 
Cloudify 4.2 Webinar - Agility & Control
Cloudify 4.2 Webinar - Agility & ControlCloudify 4.2 Webinar - Agility & Control
Cloudify 4.2 Webinar - Agility & Control
Cloudify Community
 
MySQL Live Migration - Common Scenarios
MySQL Live Migration - Common ScenariosMySQL Live Migration - Common Scenarios
MySQL Live Migration - Common Scenarios
Mydbops
 
Heart of the SwarmKit: Store, Topology & Object Model
Heart of the SwarmKit: Store, Topology & Object ModelHeart of the SwarmKit: Store, Topology & Object Model
Heart of the SwarmKit: Store, Topology & Object Model
Docker, Inc.
 
MySQL Workbench for DFW Unix Users Group
MySQL Workbench for DFW Unix Users GroupMySQL Workbench for DFW Unix Users Group
MySQL Workbench for DFW Unix Users Group
Dave Stokes
 
Slack in the Age of Prometheus
Slack in the Age of PrometheusSlack in the Age of Prometheus
Slack in the Age of Prometheus
George Luong
 
Red Hat Advanced Cluster Manager Details
Red Hat Advanced Cluster Manager DetailsRed Hat Advanced Cluster Manager Details
Red Hat Advanced Cluster Manager Details
Sourav Jyoti Das
 
Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3
LibbySchulze
 
Sprint 50 review
Sprint 50 reviewSprint 50 review
Sprint 50 review
ManageIQ
 
PostgreSQL 9.5 - Major Features
PostgreSQL 9.5 - Major FeaturesPostgreSQL 9.5 - Major Features
PostgreSQL 9.5 - Major Features
InMobi Technology
 
Ad

More from Mydbops (20)

Scaling TiDB for Large-Scale Application
Scaling TiDB for Large-Scale ApplicationScaling TiDB for Large-Scale Application
Scaling TiDB for Large-Scale Application
Mydbops
 
AWS MySQL Showdown - RDS vs RDS Multi AZ vs Aurora vs Serverless - Mydbops...
AWS MySQL Showdown - RDS vs  RDS Multi AZ vs  Aurora vs  Serverless - Mydbops...AWS MySQL Showdown - RDS vs  RDS Multi AZ vs  Aurora vs  Serverless - Mydbops...
AWS MySQL Showdown - RDS vs RDS Multi AZ vs Aurora vs Serverless - Mydbops...
Mydbops
 
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
Mydbops
 
Migration Journey To TiDB - Kabilesh PR - Mydbops MyWebinar 38
Migration Journey To  TiDB - Kabilesh PR - Mydbops MyWebinar 38Migration Journey To  TiDB - Kabilesh PR - Mydbops MyWebinar 38
Migration Journey To TiDB - Kabilesh PR - Mydbops MyWebinar 38
Mydbops
 
AWS Blue Green Deployment for Databases - Mydbops
AWS Blue Green Deployment for Databases - MydbopsAWS Blue Green Deployment for Databases - Mydbops
AWS Blue Green Deployment for Databases - Mydbops
Mydbops
 
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36
Mydbops
 
What's New in PostgreSQL 17? - Mydbops MyWebinar Edition 35
What's New in PostgreSQL 17? -  Mydbops MyWebinar Edition 35What's New in PostgreSQL 17? -  Mydbops MyWebinar Edition 35
What's New in PostgreSQL 17? - Mydbops MyWebinar Edition 35
Mydbops
 
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
Mydbops
 
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Mydbops
 
Read/Write Splitting using MySQL Router - Mydbops Meetup16
Read/Write Splitting using MySQL Router - Mydbops Meetup16Read/Write Splitting using MySQL Router - Mydbops Meetup16
Read/Write Splitting using MySQL Router - Mydbops Meetup16
Mydbops
 
TiDB - From Data to Discovery: Exploring the Intersection of Distributed Dat...
TiDB  - From Data to Discovery: Exploring the Intersection of Distributed Dat...TiDB  - From Data to Discovery: Exploring the Intersection of Distributed Dat...
TiDB - From Data to Discovery: Exploring the Intersection of Distributed Dat...
Mydbops
 
MySQL InnoDB Storage Engine: Deep Dive - Mydbops
MySQL InnoDB Storage Engine: Deep Dive - MydbopsMySQL InnoDB Storage Engine: Deep Dive - Mydbops
MySQL InnoDB Storage Engine: Deep Dive - Mydbops
Mydbops
 
Demystifying Real time Analytics with TiDB
Demystifying Real time Analytics with TiDBDemystifying Real time Analytics with TiDB
Demystifying Real time Analytics with TiDB
Mydbops
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
Efficient MySQL Indexing and what's new in MySQL Explain
Efficient MySQL Indexing and what's new in MySQL ExplainEfficient MySQL Indexing and what's new in MySQL Explain
Efficient MySQL Indexing and what's new in MySQL Explain
Mydbops
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
Mydbops
 
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024
Mydbops
 
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...
Mydbops
 
Mastering Aurora PostgreSQL Clusters for Disaster Recovery
Mastering Aurora PostgreSQL Clusters for Disaster RecoveryMastering Aurora PostgreSQL Clusters for Disaster Recovery
Mastering Aurora PostgreSQL Clusters for Disaster Recovery
Mydbops
 
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...
Mydbops
 
Scaling TiDB for Large-Scale Application
Scaling TiDB for Large-Scale ApplicationScaling TiDB for Large-Scale Application
Scaling TiDB for Large-Scale Application
Mydbops
 
AWS MySQL Showdown - RDS vs RDS Multi AZ vs Aurora vs Serverless - Mydbops...
AWS MySQL Showdown - RDS vs  RDS Multi AZ vs  Aurora vs  Serverless - Mydbops...AWS MySQL Showdown - RDS vs  RDS Multi AZ vs  Aurora vs  Serverless - Mydbops...
AWS MySQL Showdown - RDS vs RDS Multi AZ vs Aurora vs Serverless - Mydbops...
Mydbops
 
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
Mydbops
 
Migration Journey To TiDB - Kabilesh PR - Mydbops MyWebinar 38
Migration Journey To  TiDB - Kabilesh PR - Mydbops MyWebinar 38Migration Journey To  TiDB - Kabilesh PR - Mydbops MyWebinar 38
Migration Journey To TiDB - Kabilesh PR - Mydbops MyWebinar 38
Mydbops
 
AWS Blue Green Deployment for Databases - Mydbops
AWS Blue Green Deployment for Databases - MydbopsAWS Blue Green Deployment for Databases - Mydbops
AWS Blue Green Deployment for Databases - Mydbops
Mydbops
 
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36
Mydbops
 
What's New in PostgreSQL 17? - Mydbops MyWebinar Edition 35
What's New in PostgreSQL 17? -  Mydbops MyWebinar Edition 35What's New in PostgreSQL 17? -  Mydbops MyWebinar Edition 35
What's New in PostgreSQL 17? - Mydbops MyWebinar Edition 35
Mydbops
 
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
Mydbops
 
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Mydbops
 
Read/Write Splitting using MySQL Router - Mydbops Meetup16
Read/Write Splitting using MySQL Router - Mydbops Meetup16Read/Write Splitting using MySQL Router - Mydbops Meetup16
Read/Write Splitting using MySQL Router - Mydbops Meetup16
Mydbops
 
TiDB - From Data to Discovery: Exploring the Intersection of Distributed Dat...
TiDB  - From Data to Discovery: Exploring the Intersection of Distributed Dat...TiDB  - From Data to Discovery: Exploring the Intersection of Distributed Dat...
TiDB - From Data to Discovery: Exploring the Intersection of Distributed Dat...
Mydbops
 
MySQL InnoDB Storage Engine: Deep Dive - Mydbops
MySQL InnoDB Storage Engine: Deep Dive - MydbopsMySQL InnoDB Storage Engine: Deep Dive - Mydbops
MySQL InnoDB Storage Engine: Deep Dive - Mydbops
Mydbops
 
Demystifying Real time Analytics with TiDB
Demystifying Real time Analytics with TiDBDemystifying Real time Analytics with TiDB
Demystifying Real time Analytics with TiDB
Mydbops
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
Efficient MySQL Indexing and what's new in MySQL Explain
Efficient MySQL Indexing and what's new in MySQL ExplainEfficient MySQL Indexing and what's new in MySQL Explain
Efficient MySQL Indexing and what's new in MySQL Explain
Mydbops
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
Mydbops
 
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024
Mydbops
 
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...
Mydbops
 
Mastering Aurora PostgreSQL Clusters for Disaster Recovery
Mastering Aurora PostgreSQL Clusters for Disaster RecoveryMastering Aurora PostgreSQL Clusters for Disaster Recovery
Mastering Aurora PostgreSQL Clusters for Disaster Recovery
Mydbops
 
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...
Mydbops
 
Ad

Recently uploaded (20)

May Patch Tuesday
May Patch TuesdayMay Patch Tuesday
May Patch Tuesday
Ivanti
 
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Safe Software
 
AI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of DocumentsAI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of Documents
UiPathCommunity
 
AI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamsonAI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamson
UXPA Boston
 
Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...
Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...
Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...
Mike Mingos
 
Kit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdf
Kit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdfKit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdf
Kit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdf
Wonjun Hwang
 
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier VroomAI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
UXPA Boston
 
UiPath Automation Suite – Cas d'usage d'une NGO internationale basée à Genève
UiPath Automation Suite – Cas d'usage d'une NGO internationale basée à GenèveUiPath Automation Suite – Cas d'usage d'une NGO internationale basée à Genève
UiPath Automation Suite – Cas d'usage d'une NGO internationale basée à Genève
UiPathCommunity
 
An Overview of Salesforce Health Cloud & How is it Transforming Patient Care
An Overview of Salesforce Health Cloud & How is it Transforming Patient CareAn Overview of Salesforce Health Cloud & How is it Transforming Patient Care
An Overview of Salesforce Health Cloud & How is it Transforming Patient Care
Cyntexa
 
DevOpsDays SLC - Platform Engineers are Product Managers.pptx
DevOpsDays SLC - Platform Engineers are Product Managers.pptxDevOpsDays SLC - Platform Engineers are Product Managers.pptx
DevOpsDays SLC - Platform Engineers are Product Managers.pptx
Justin Reock
 
Unlocking Generative AI in your Web Apps
Unlocking Generative AI in your Web AppsUnlocking Generative AI in your Web Apps
Unlocking Generative AI in your Web Apps
Maximiliano Firtman
 
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Wonjun Hwang
 
On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...
On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...
On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...
Ivano Malavolta
 
Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...
Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...
Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...
Markus Eisele
 
How to Install & Activate ListGrabber - eGrabber
How to Install & Activate ListGrabber - eGrabberHow to Install & Activate ListGrabber - eGrabber
How to Install & Activate ListGrabber - eGrabber
eGrabber
 
Bepents tech services - a premier cybersecurity consulting firm
Bepents tech services - a premier cybersecurity consulting firmBepents tech services - a premier cybersecurity consulting firm
Bepents tech services - a premier cybersecurity consulting firm
Benard76
 
Slack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teamsSlack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teams
Nacho Cougil
 
Build With AI - In Person Session Slides.pdf
Build With AI - In Person Session Slides.pdfBuild With AI - In Person Session Slides.pdf
Build With AI - In Person Session Slides.pdf
Google Developer Group - Harare
 
IT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information TechnologyIT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information Technology
SHEHABALYAMANI
 
Config 2025 presentation recap covering both days
Config 2025 presentation recap covering both daysConfig 2025 presentation recap covering both days
Config 2025 presentation recap covering both days
TrishAntoni1
 
May Patch Tuesday
May Patch TuesdayMay Patch Tuesday
May Patch Tuesday
Ivanti
 
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Integrating FME with Python: Tips, Demos, and Best Practices for Powerful Aut...
Safe Software
 
AI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of DocumentsAI Agents at Work: UiPath, Maestro & the Future of Documents
AI Agents at Work: UiPath, Maestro & the Future of Documents
UiPathCommunity
 
AI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamsonAI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamson
UXPA Boston
 
Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...
Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...
Optima Cyber - Maritime Cyber Security - MSSP Services - Manolis Sfakianakis ...
Mike Mingos
 
Kit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdf
Kit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdfKit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdf
Kit-Works Team Study_팀스터디_김한솔_nuqs_20250509.pdf
Wonjun Hwang
 
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier VroomAI x Accessibility UXPA by Stew Smith and Olivier Vroom
AI x Accessibility UXPA by Stew Smith and Olivier Vroom
UXPA Boston
 
UiPath Automation Suite – Cas d'usage d'une NGO internationale basée à Genève
UiPath Automation Suite – Cas d'usage d'une NGO internationale basée à GenèveUiPath Automation Suite – Cas d'usage d'une NGO internationale basée à Genève
UiPath Automation Suite – Cas d'usage d'une NGO internationale basée à Genève
UiPathCommunity
 
An Overview of Salesforce Health Cloud & How is it Transforming Patient Care
An Overview of Salesforce Health Cloud & How is it Transforming Patient CareAn Overview of Salesforce Health Cloud & How is it Transforming Patient Care
An Overview of Salesforce Health Cloud & How is it Transforming Patient Care
Cyntexa
 
DevOpsDays SLC - Platform Engineers are Product Managers.pptx
DevOpsDays SLC - Platform Engineers are Product Managers.pptxDevOpsDays SLC - Platform Engineers are Product Managers.pptx
DevOpsDays SLC - Platform Engineers are Product Managers.pptx
Justin Reock
 
Unlocking Generative AI in your Web Apps
Unlocking Generative AI in your Web AppsUnlocking Generative AI in your Web Apps
Unlocking Generative AI in your Web Apps
Maximiliano Firtman
 
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Wonjun Hwang
 
On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...
On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...
On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Conte...
Ivano Malavolta
 
Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...
Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...
Enterprise Integration Is Dead! Long Live AI-Driven Integration with Apache C...
Markus Eisele
 
How to Install & Activate ListGrabber - eGrabber
How to Install & Activate ListGrabber - eGrabberHow to Install & Activate ListGrabber - eGrabber
How to Install & Activate ListGrabber - eGrabber
eGrabber
 
Bepents tech services - a premier cybersecurity consulting firm
Bepents tech services - a premier cybersecurity consulting firmBepents tech services - a premier cybersecurity consulting firm
Bepents tech services - a premier cybersecurity consulting firm
Benard76
 
Slack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teamsSlack like a pro: strategies for 10x engineering teams
Slack like a pro: strategies for 10x engineering teams
Nacho Cougil
 
IT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information TechnologyIT488 Wireless Sensor Networks_Information Technology
IT488 Wireless Sensor Networks_Information Technology
SHEHABALYAMANI
 
Config 2025 presentation recap covering both days
Config 2025 presentation recap covering both daysConfig 2025 presentation recap covering both days
Config 2025 presentation recap covering both days
TrishAntoni1
 

Group Replication in MySQL 8.0 ( A Walk Through )

  • 1. Group Replication In MYSQL 8.0 A walk through Presented by VIGNESH PRABHU S www.mydbops.com info@mydbops.com
  • 2. About Mydbops ● Founded in 2015, HQ in Bangalore India with 450+ customer base across the globe. ● Mydbops is on Database Consulting with core specialization on MySQL ,MongoDB & PostgreSQL Administration and Support. ● We have expert team with 30+ certified DBA’s providing full time support and currently managing 300+ servers on cloud. ● Mydbops was created with a motto of developing a Devops model for Database administration offering 24*7 expert remote DBA support. ● We help organisations to architect and scale systems in MySQL/Mongo by implementing the advanced technologies in industry which are completely open source. ● We are a leading solution provider in the market for all sort of cloud based deployments and management.
  • 3. Agenda ● Group Replication - Introduction ● Group Replication - Architecture ● Group Replication - Monitoring ● Group Replication - Modes ● Primary Member Change - Cases ● Primary Election Algorithm ● Conditions for using online UDF ● Communication Protocol ● Transaction Consistency
  • 4. Group Replication - Introduction ● High Availability Solution ● To create a fault tolerant System. ● Supports Server side failover. ● Client side failover is not supported by GR. ● Distributed functionality. ● Based on the GCS protocol and paxos algorithm. ● Split brain issue is avoided by Quorum.
  • 5. Group Replication - Introduction Services of Group Replication:- ● Group Membership (VIEW) ● Failure Detection (Detect the failure & update the VIEW) ● Fault-tolerance ( QUORUM condition n = 2 x f + 1. ) ● Monitoring ( performance schema tables ) All these Services are built into the GCS Protocol.
  • 6. Group Replication - Architecture
  • 7. Group Replication - Architecture Functionality of Each Component:- ● Capture :Keep track of all the local transactions. ● Applier :Remote transactions. ● Recovery :Manage the distributed recovery. ● Replication :Define the replication protocol & handle the conflict detection in the group. ● GCS & Xcom :GR Services & communication protocol.
  • 8. Group Replication - ( Commit )
  • 9. Group Replication - Monitoring Tables in Perf schema:- ● replication_group_member_stats ( info about certification process) ● replication_group_members ( info about the group members ) ● replication_connection_status ( info about the queue ) ● replication_applier_status ( shows state of replication worker thread ) Replication Channel:- ● group_replication_recovery - ( Channel for the recovery process) ● group_replication_applier - ( transactions from the other members or from the group )
  • 10. Group Replication - Monitoring Group Replication Server States:- ● Online :will accept the connection from the client. ● Recovering :Applying the events from the donor. ● Offline :Member doesn’t belong to any group. ● ERROR :Member is in the group but not working properly. “group_replication_exit_state_action” ● UNREACHABLE :It is mostly because of the network failure.
  • 11. Group Replication - Modes ● Based on the variable - (group_replication_single_primary_mode) ● Single Primary Mode ○ One member accept the writes. ○ group_replication_enforce_update_everywhere_checks (off) ● Multi Primary Mode ○ All the members can accept the writes. ○ group_replication_enforce_update_everywhere_checks (ON)
  • 12. Group Replication - Modes ● From MySQL version 8.0.13 we can change the modes in online Using the UDF’s ○ group_replication_switch_to_single_primary_mode() ○ group_replication_switch_to_multi_primary_mode() ● From Version 8.0.13 to 8.0.16, no check for the safeguard. ● From Version 8.0.17, GR has safeguard check. ○ (i.e) Lower MySQL version member will become the Primary.
  • 13. Primary Member Change - Cases ● Primary leaves the group . ● While Changing the primary member manually using UDF. ● While Changing the multi primary to single primary.
  • 14. Primary Election Algorithm First Election Check:- ● Ordering based on MySQL major Version & patch Version. Second Election Check:- ● Based on lowest member weight. ( group_replication_member_weight ). Third Election Check:- ● Based on the lowest server UUID.
  • 15. Conditions for using online UDF 1. All members must be in ONLINE state 2. No members can join in the group during a configuration change. 3. Only one configuration at once. 4. All members must be running MySQL 8.0.13 or higher
  • 16. Communication Protocol ● From MySQL 8.0.16, Configure different versions of MySQL members. ● Group must use same communication protocol Version. Reason:- ● From MySQL 5.7.14, allow compression of messages ● From MySQL 8.0.16, allow fragmentation of messages Functions:- ● group_replication_get_communication_protocol(); ● group_replication_set_communication_protocol(“version”);
  • 17. Transaction Consistency. ● Controlled by group_replication_consistency variable. ○ EVENTUAL ○ BEFORE_ON_PRIMARY_FAILOVER ○ BEFORE ( RO trx wait for preceding trx to apply ) ○ AFTER ( RW trx wait for preceding trx to apply in other members ) ○ BEFORE_AND_AFTER
  • 18. Email Us : Info@mydbops.com Thank you
  翻译: