Mysql replication @ gnugroup

What is replication?
Allows 2 or more databases to maintain state
between them
MySQL 3.23 and up supports asynchronous
replication
One server acts as a master for one or more slaves
Slaves pull data to be replicated from server’s
binary logs and execute the relevant statements
locally

3
MySQL Replication
Why? How?
1. High Availability Snapshots (Backup)
Possibility of fail-over 1. Client program mysqldump
2. Load-balancing/Scale- With log coordinates
out 2. Using backup
Query multiple servers InnoDB, NDB
3. Off-site processing
Don’t disturb master
Binary log
1. Replication
Asynchronous pushing to slave
2.Point-in-time recovery
Roll-forward

Replication basics…
 One master supports multiple slaves
 Each node on the network is assigned a unique identifying number
 Each slave attempts to connect to the master and fetches data to be replicated
 “Master” keeps logs of all changes – called
 “binary logs” or “binlogs”
 “Slave” connects to the master through normal
 MySQL protocol (TCP port 3306)
Master
Slave
Slave
Slave
Slaves

Clients perform data modification on master
server
INSERT, SELECT, DELETE, LOAD DATA
EXECUTE in MySQL 5.0 and above
Master SlaveClient
DATA DATA
INSERT
INTO …

Immediately following execution of command on master,
the command is written to the local binary log
Additionally, the master records its unique ID (to prevent
endless loops in circular replication scenarios) and the
timestamp for use with statements which use NOW(), etc.
Master SlaveClient
DATA
DATA
INSERT
INTO …
Binary
Log

If the slave is online, the command is transmitted to the slave in
parallel (well, immediately following) to being written in the local
binary log
Otherwise, when the slave next connects it will receive a list of all
pending statements from the master server’s binary log
The slave’s replication IO thread stores the command in the local relay
log
Master SlaveClient
DATA
DATA
INSERT
INTO …
INSERT
INTO …
Relay
Log
Replication
Thread

Once the data is received in the slave’s relay log, the slave
SQL thread executes the command locally, bringing the
slave up-to-date with the master
In MySQL 3.23, the IO and SQL threads were just one
thread. In later versions this was changed to boost
performance
Master SlaveClient
DATA
DATA
INSERT
INTO …
INSERT
INTO …
Relay
Log
Replication
Thread
DATA

Replication basics…(contd)
Replication works with all tables types
• Any “critical” reads must be done on the master –
replication is asynchronous, there may be a delay
• Master will rotate binary logs automatically for
every 1G of log records
• You must purge any old, unused, logs yourself

Terminology
Synchronous replication Master
• A transaction is not committed until the data MySQL
has been replicated (and applied) Server
• Safer, but slower
• This is available in MySQL Cluster
Replication
Asynchronous replication
• A transaction is replicated after it has been
MySQLcommitted
Server
• Faster, but you can in some cases loose
transactions if master fails Slave
• Easy to set up between MySQL servers

Terminology
Master MySQL Server
• Changes data
• Has binlog turned on Master
• Pushes binlog events to slave after slave has requested themMySQL
Server
Slave MySQL Server
• Main control point of replication
Replication• Asks master for replication log
• Gets binlog event from master
MySQL
Binary log Server
• Log of everything executed Slave
• Divided into transactional components
• Used for replication and point-in-time recovery

Simple Replication Setup
Modify my.cnf to include a unique server-id for
each node
On master server, ensure that log-bin (binary
logging) is enabled in my.cnf
On slave, configure login credentials on master,
either via my.cnf or CHANGE MASTER TO
statement
Copy initial data snapshot from master to slave
Configure initial binary log position on slave
Start replication with SLAVE START command

server-id – The unique server ID of this MySQL
• log-bin – Enable logging of changes to binary
logs
• log-slave-updates – Log updates that arrive on
the slave thread to the binary logs as well
(required if this master is also a slave of another
machine)
• binlog-do-db – Disables logging of any changes,
except to the specified databases
• binlog-ignore-db – Log all changes, as usual,
except for the specified databases

my.cnf
-------------
[mysqld]
server-id = 1
log-bin

ON MASTER SIDE
1 ) grant slave users on the master server :
Each slave must connect to the master using a standard
MySQL user name and password, so there must be a user
account on the master that the slave can use to connect. In this
scenario with existing slave, user for replication already exist on
the master server.

Permissions
Slaves need REPLICATION SLAVE permission on
master for basic usage
If LOAD TABLE FROM MASTER or LOAD DATA
FROM MASTER statements are used, slave will also
need SUPER and RELOAD privileges

Configuration - grants on master
GRANT REPLICATION SLAVE on
*.* TO ‘rep_user’@’slave-
host’ IDENTIFIED BY ‘this-is-the-password’

ON MASTER SIDE
2) Locking databases:
Take a section of mysql and lock the tables with the below
command.
USE exampledb; database name
mysql> FLUSH TABLES WITH READ LOCK;
Do not close the section. Kept it open

ON MASTER SIDE
3) Position Noting:
Use the below command and note down the position of master
SHOW MASTER STATUS;

ON MASTER SIDE

4) Backup the database:
Backup the databases using the following command
#> mysqldump -u root -p --databases trains >
/tmp/NNNNNN.sql

ON MASTER SIDE

5) unlocking databases;
On successfully compellations of backup activity up can unlock
databases on the activity section left open previously
mysql> UNLOCK TABLES;

my.cnf
-------------
[mysqld]
server-id = 2
master-user = someuser                ---> this setting not recommended in cnf file.
master-password = secret              ---> this setting not recommended.
master-host = ip.of.master             ---> this setting not recommended.

ON SLAVE SIDE

1) restore backup;
Restore the database
Mysql> . /tmp/NNNNNN.sql
NNNN is DB name

Restore the backup onto the slave
Master
Slave

ON SLAVE SIDE

2) Defining Position:
CHANGE MASTER TO MASTER_HOST='IP',
MASTER_USER='slave_user',
MASTER_PASSWORD='<some_password>',
MASTER_LOG_FILE='mysql-bin.NNNNN',
MASTER_LOG_POS=NN;

ON SLAVE SIDE

3) Start Slave:
start slave;

SLAVE mysql> SHOW SLAVE STATUSG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: vmware-mirimar
Master_User: someuser
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: vmware-mirimar-bin.000002
Read_Master_Log_Pos: 79
Relay_Log_File: vmware1-mirimar-relay-bin.000002
Relay_Log_Pos: 250
Relay_Master_Log_File: vmware-mirimar-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 79
Relay_Log_Space: 250
Seconds_Behind_Master: 0

More optional configuration on the slave
read-only
log-slave-updates
skip-slave-start

Master with Slave
Master
Slave

Master with Slave
binary
log
Master
TCP connection
Slave

Replication is independent of Storage Engines
You can replicate between any pair of engines
InnoDB to InnoDB
MyISAM to MyISAM
InnoDB to MyISAM
MEMORY to MyISAM
etc...
The binary log is not the InnoDB transaction log (or the
Falcon log, or ...)

Master with Many Slaves
Master
Slave Slave Slave Slave

Chain
Master/
Master Slave
Slave
log_slave_updates = 1

Chain - Server 2 goes down...
Master/
Master Slave
X

... Server 3 is still up, but out of sync
Master/
Master Slave
X

Each server has a unique "server_id"
server_id=3server_id=1
Master/
Master Slave
Slave
server_id=2
... and every event in a binary log file contains
the server id number of the server where the
event originated.

Ring
server_id=2
Master/
Slave
Master/
Slave
server_id=1 Master/
Slave
server_id=3

The ring topology is not a recommended
configuration
Master/
Slave
Master/
Slave
Master/
X

Pair of Masters
Master/ Master/
Slave Slave
The pair is a “special case” of the ring topology
used for
high availability.

The two most common topologies for
MySQL Replication
Master
Master/ Master/
Slave Slave
Slave
Slave
Slave

The "Relay Slave"
The master has to
handle only one
Master TCP connection.
Relay
Slave
log_slave_updates
Slave Slave Slave Slave Slave

And now introducing... the blackhole
storage engine
Master
engine = blackhole The relay slave
Relay manages replication
Slave
logs, but not actual
data.
Slave Slave Slave Slave Slave

Replication
Commands
A quick run-through of the commands

SHOW BINARY LOGS
Used on master
Requires SUPER privileges
Will display a list of binary logs on the server
Use it before using PURGE BINARY LOGS
mysql> SHOW BINARY LOGS;
+---------------+-----------+
| Log_name | File_size |
+---------------+-----------+
| binlog.000015 | 724935 |
| binlog.000016 | 733481 |
+---------------+-----------+

SHOW BINLOG EVENTS
Used on master
Requires REPLICATION SLAVE privileges
Show events in binary log
Also check mysqlbinlog utility
mysql> SHOW BINLOG EVENTS FROM 390 LIMIT 1G
*** 1. row ***
Log_name: slave-bin.000001
Pos: 390
Event_type: Query
Server_id: 2
End_log_pos: 476
Info: use `test`; create table t1 (a int) 1
row in set (0.00 sec)

SHOW SLAVE HOSTS
Used on master
Requires REPLICATION SLAVE privileges Shows list of
slaves currently registered with the master Only slaves
started with report-host option are visible
mysql> SHOW SLAVE HOSTS;
+-----------+-----------+------+-----------+
| Server_id | Host | Port | Master_id |
+-----------+-----------+------+-----------+
| 2 | 127.0.0.1 | 9308 | 1 |
+-----------+-----------+------+-----------+ 1
row in set (0.00 sec)

PURGE BINARY LOGS
Used on master
Removes log files before a certain log file or date
MASTER can be used in place of BINARY
Alternative is to use variable EXPIRE_LOGS_DAYS

SHOW SLAVE STATUS
Used on slave
Requires SUPER or REPLICATION CLIENT privileges
Shows some interesting information:
If the slave threads are running What
position the I/O thread read last What
position the SQL thread executed last
Error message and code, if thread stopped due to an error

SHOW SLAVE STATUS (5.1)
mysql> SHOW SLAVE STATUSG
*** 1. row ***
Slave_IO_State: Last_Errno: 0
Master_Host: 127.0.0.1 Last_Error:
Master_User: root Skip_Counter: 0
Master_Port: 10190 Exec_Master_Log_Pos: 0
Connect_Retry: 1 Relay_Log_Space: 102
Master_Log_File: Until_Condition: None
Read_Master_Log_Pos: 4 Until_Log_File:
Relay_Log_File: slave-relay-bin.000001 Until_Log_Pos: 0
Relay_Log_Pos: 4 Master_SSL_Allowed: No
Relay_Master_Log_File: Master_SSL_CA_File:
Slave_IO_Running: No Master_SSL_CA_Path:
Slave_SQL_Running: No Master_SSL_Cert:
Replicate_Do_DB: Master_SSL_Cipher:
Replicate_Ignore_DB: Master_SSL_Key:
Replicate_Do_Table: Seconds_Behind_Master: NULL
Replicate_Ignore_Table: Last_IO_Errno: 0
Replicate_Wild_Do_Table: Last_IO_Error:
Replicate_Wild_Ignore_Table: Last_SQL_Errno: 0
Last_SQL_Error:
1 row in set (0.00 sec)

START SLAVE and STOP SLAVE
Used on slave
Used to start or stop the slave threads Defaults to
affecting both I/O and SQL thread ... but individual
threads can be started or stopped START SLAVE
SQL_THREAD
START SLAVE IO_THREAD

RESET SLAVE
Used on slave
Removes all info on replication position
Deletes master.info, relay-log.info and all relay logs
Relay logs are unconditionally removed!
... even if they have not been fully applied

SET GLOBAL SQL_SLAVE_SKIP_COUNTER
Used on slave
Global server variable
Slave SQL thread shall not be running
Slave will skip events when starting
Useful when recovering from slave stops
Might leave master and slave with different data in tables
... so be careful when you use it

Use Cases, Part 1 - Basic Replication
Intensive Reads High Availability
Master
Master/ Master/
Slave Slave
Slave
Slave Slave

“Specialist" slaves - backups and reporting
Master
Slave
Slave
reportsSlave
Slave
Slave
backups

“Specialist" slaves - per-application
friends: 10 GB
Master
messages: 30
GB
Slave Slave
Slave
Slave
“message board” queries“friends list” queries

“Specialist" slaves - Blackhole Engine
Master
Slave
Slave
Slave
Slave “message board” queries
(friends table in black
“friends list” queries
hole)
(message table in black
hole)

Things to think about in basic replication
Initial snapshot of slaves
load balancing of clients
Failover of clients to new master

HA + Scale out?
Master/ Master/
Slave Slave
Slave
Slave Slave

Any better?
Master/ Master/
Slave Slave
Proxy
Master
Slave
Slave
Slave

Use Cases, Part 3 - Multiple Data Centers
secure
San Jose New Yorktunnel
rep
Active
Master
Master
wr wr
wr wr
Slave Slave
rd rd
app app
Slave Slave
( Jeremy Cole - MySQL Users Conf 2006 )

After Failover
secure
San Jose New Yorktunnel
rep
Active
Master
Master
wr wr
wr wr
Slave Slave
rd rdapp app
Slave Slave
( Jeremy Cole - MySQL Users Conf 2006 )

Internal Threads
Since MySQL 4.0, replication slaves run two threads
IO thread continuously receives updates from master
and writes to local relay log
SQL thread continuously executes statements in relay
log

IO thread isolation
Isolating IO thread means that slave won’t have to
wait for long-executing statements to finish executing
before retrieving data from master
Also, slave will continue reading data from master if a
statement creates a data conflict

SQL thread isolation
SQL thread isolation allows for replication in an
environment without a continuous link between
slave and masters
If master fails (or slave simply has no access), the
IO thread will try to reconnect endlessly (waiting
60 seconds between attempts)
SQL thread will continue processing relay logs
even while IO thread is unable to connect to
master

Master Thread
Additionally, the master server runs the Binlog Dump
thread
This thread is simply dedicated to scanning the binary
logs on the master and sending updates to the
connected slave
If this thread isn’t running, it means that replication
isn’t running – more accurately, that no slaves are
currently connected

Status files
2 status files for replication’s use
Their use is to record the state of replication between
server shutdown and startup
master.info records information about the slave’s
master server
relay-log.info records information about the local
relay logs

Information in master.info
Master log file
Read master log pos
Master Host
Master User
Password (will not be shown in SHOW SLAVE
STATUS)
Master Port
Connect Retry
In MySQL 4.1+, SSL options are stored if SSL is
used

Information in relay-log.info
Relay log file
Relay log pos
Relay master-log pos
Exec master-log pos

Backup master
 Master backups can be accomplished with
mysqldump
 Care must be taken to ensure the following 2
special considerations:
1. Consistent snapshot of master date (via lock tables
for MyISAM or single transaction for InnoDB)
2. Recording of binary log information, for use on slaves
(master-data)

Backup master files
If a file-system level backup is required, care should be
taken to manually record binary log name and position via
SHOW MASTER STATUS statement.
To ensure consistency between backup and binary log
position, the tables should be locked via FLUSH TABLES
WITH READ LOCK immediately before backup (and
SHOW MASTER STATUS)
LEAVE THE CLIENT CONNECTED!!!
After backup finishes, execute UNLOCK TABLES to release
the read lock

Backup slave
Same idea as master file system backup
Instead of recording position, it’s enough to backup
the master.info and relay-log.info files
Instead of acquiring global read lock, it’s enough to
STOP SLAVE before backup and START SLAVE once
backup finishes

DO
 Make sure server-id is set on all machines
 Enable log-bin on your master
 If a slave will also be a master, set log-
slaveupdates
 Take backups on a slave
 Architect your replication setup for growth!

DO NOT
Don’t use master-* lines in my.cnf; use CHANGE
MASTER instead
 Don’t take your backups on the master

Thank You!
For more information:
www.gnugroup.org

Mysql replication @ gnugroup

Recommended

More Related Content

What's hot (18)

Similar to Mysql replication @ gnugroup (20)

Recently uploaded (20)

Mysql replication @ gnugroup