Skip to content

Commit

Permalink
Make mysql replicas to work with MySQL 5.7 and 5.8
Browse files Browse the repository at this point in the history
This fixes moodlehq#80. It has been detected that MySQL main/replica
configurations have stopped working. My personal bet is that
some changes in the MySQL images have leaded to that, or maybe
they were working for MySQL 5.6 and never have worked to 5.7.

NVM, this commit just makes them back to work for both 5.7 and 8.0,
that are the currently "supported" MySQL databases.

Note that, after this PR, we'll be checking the MariaDB main/replica
configurations because they may need similar changes.

Also, while I've tested the solution locally, we are lacking any own
CI for this product, so I've created an issue about that:

moodlehq#81

Finally, here there are the explanations about the changes performed:

1. Moved the bin logs from /var/log/mysql to /var/lib/mysql . Reason
   being that the former directory doesn't exist anymore in MySQL 5.7
   and up (and also, that the binary logs are better with data than
   with logs).
2. Removed the 'multi_admin' user related stuff. Not needed at all.
3. Minor changes to moodle and root GRANT/IDENTIFIED stuff because
   of changes in MySQL 8 (already deprecated in 5.7). Use ALTER USER
   instead.
4. Use IDENTIFIED WITH mysql_native_password (the old password schema),
   because MySQL 8, when using the new hashed schema requires
   communications between the hosts to be encrypted. We don't need that
   extra complexity and while it can be disabled as part of the CHANGE MASTER TO
   command... it cannot be disabled for MySQL 5.7. SO the only option
   working in both versions is the used one (force old passwords schema
   for the replication user).
5. Remove the need of stopping and starting the temp server in the
   replica. All the commands can be executed with the temp server
   running and then the container entrypoint manageing itself will stop
   the temp server and start the final one.
6. Stop needing gosu completely. Now the replica conf files are made
   available via volume mount (like the standalone or main cases).

And that's all. The changes are small and subtle, but each one is
for a good reason, I think.
  • Loading branch information
stronk7 committed Aug 24, 2022
1 parent 3891a8f commit 94a6c51
Show file tree
Hide file tree
Showing 7 changed files with 12 additions and 38 deletions.
2 changes: 1 addition & 1 deletion runner/master/mysql.d/master/conf.d/master.cnf
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ read_buffer_size = 1M
innodb_flush_log_at_trx_commit = 1

server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
log_bin = /var/lib/mysql/mysql-bin.log
sync_binlog = 1
binlog-format = ROW

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
GRANT SHUTDOWN ON *.* TO 'multi_admin'@'localhost' IDENTIFIED BY 'multipass';
CREATE USER 'replication'@'%' IDENTIFIED BY 'replication';
CREATE USER 'replication'@'%' IDENTIFIED WITH mysql_native_password BY 'replication';
GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%';

GRANT ALL ON *.* TO 'root'@'%' IDENTIFIED BY 'moodle';
GRANT ALL ON *.* TO 'moodle'@'%' IDENTIFIED BY 'moodle';
ALTER USER 'root'@'%' IDENTIFIED BY 'moodle';

FLUSH PRIVILEGES;
7 changes: 7 additions & 0 deletions runner/master/mysql.d/slave/conf.d/slave.cnf
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[mysqld]
server-id = 2
relay-log = /var/lib/mysql/mysql-relay-bin.log
log_bin = /var/lib/mysql/mysql-bin.log

# List of databases to replicate.
binlog_do_db = moodle
7 changes: 0 additions & 7 deletions runner/master/mysql.d/slave/config/slave.cnf

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,6 @@ echo "==========================================================================

sleep 5

# We need to set this for some inane reason otherwise it fails to start mysql.
# This seems to be a bug with the entrypoint.sh
DATABASE_ALREADY_EXISTS='true'

docker_temp_server_stop

echo "Copying moodle.cnf"
gosu root cp /config/moodle.cnf /etc/mysql/conf.d/

echo "Restarting mysqld"
docker_temp_server_start mysqld

echo "Dumping ${DBHOST}"
mysql -u root -pmoodle -h ${DBHOST} -e "FLUSH TABLES WITH READ LOCK;" moodle
mysql -u root -pmoodle -h ${DBHOST} -e "SHOW MASTER STATUS;" moodle
Expand All @@ -38,14 +26,6 @@ echo "Current position is {$replfile} {$position}"
echo "Restoring into client"
mysql -u root -pmoodle moodle < /tmp/moodle.sql

echo "Stopping server to apply slave configuration"
docker_temp_server_stop
gosu root cp /config/slave.cnf /etc/mysql/conf.d/

echo "Restarting mysqld"
docker_temp_server_start mysqld

echo "Starting slave"
mysql -u root -pmoodle moodle << EOSQL
CHANGE MASTER TO
MASTER_HOST='${DBHOST}',
Expand Down
8 changes: 2 additions & 6 deletions runner/master/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -335,14 +335,10 @@ then
-e MYSQL_PASSWORD="${DBPASS}" \
-e DBHOST=$DBHOST \
-e DBHOST_SLAVE=$DBHOST_SLAVE \
-v $SCRIPTPATH/mysql.d/slave/config:/config \
-v $SCRIPTPATH/mysql.d/slave/conf.d:/etc/mysql/conf.d \
-v $SCRIPTPATH/mysql.d/slave/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d \
--tmpfs /var/lib/mysql:rw \
mysql:${DBTAG}

# Hack to make gosu work for all users on the slave.
docker exec -u root $DBHOST_SLAVE bash -c 'chown root:mysql /usr/local/bin/gosu'
docker exec -u root $DBHOST_SLAVE bash -c 'chmod +s /usr/local/bin/gosu'
else
echo "Starting standalone"
docker run \
Expand Down Expand Up @@ -395,7 +391,7 @@ then
-e MYSQL_PASSWORD="${DBPASS}" \
-e DBHOST=$DBHOST \
-e DBHOST_SLAVE=$DBHOST_SLAVE \
-v $SCRIPTPATH/mysql.d/slave/config:/config \
-v $SCRIPTPATH/mysql.d/slave/conf.d:/etc/mysql/conf.d \
-v $SCRIPTPATH/mysql.d/slave/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d \
--tmpfs /var/lib/mysql:rw \
mariadb:${DBTAG}
Expand Down

0 comments on commit 94a6c51

Please sign in to comment.