Comments on: ZFS Replication for MySQL data

By: Mark Callaghan

Mark Callaghan — Tue, 18 Nov 2008 21:01:25 +0000

Maybe I am being pedantic, but what you have described is a full backup followed by a sequence of incremental backups. I would not call this replication. It is still interesting.

By: Dave Edwards

Dave Edwards — Fri, 14 Nov 2008 19:26:53 +0000

Martin ‘MC’ Brown of MCslp Coalface takes on the matter of ZFS Replication for MySQL data.

By: Martin MC Brown

Martin MC Brown — Tue, 11 Nov 2008 08:59:16 +0000

Matthew:

Yes, AVS is the way to go for a more complete DRBD-like solution, and I’m working on an AVS blog post at the moment.

However, AVS is limited to OpenSolaris at the moment, and basic ZFS replication can be used on OS X and FreeBSD too.

By: Matthew Montgomery

Matthew Montgomery — Tue, 11 Nov 2008 01:26:35 +0000

The proper way to handle block device level replication in Solaris/ZFS is to use AVS

See: http://opensolaris.org/os/project/avs/

The idea behind AVS is very much the same as DRBD.

By: Nils Meyer

Nils Meyer — Mon, 10 Nov 2008 15:03:31 +0000

Keep in mind that DRBD works live, it’s not a stale snapshot that’s periodically replicated. Kind of like RAID over TCP. Your CAPTCHA is impossible to read.

By: Martin MC Brown

Martin MC Brown — Mon, 10 Nov 2008 07:24:57 +0000

It is true that the more snapshots that you keep around, there will be both some storage and performance overhead to keep them up to date.

But the point is that you shouldn’t ever be keeping snapshots long term.

In a typical backup setup, you’d keep limited snapshots around (say one a day), but most likely copy anything more than day old, and certainly a week old, off somewhere else.

In a replication environment, as I’ve described here, you need only keep the snapshots for as long as it takes to replicated them to the slave machine, plus enough snapshots for you to be able to create a suitable incremental for zfs send.

I replicate between two machines with a snapshot every 15 minutes, and keep one snapshot from the start of the day (literally at midnight) and keep only the last 4 (one hour) of snapshots beyond that. The rest are automatically destroyed.

Because of the way the snapshots work, that means I can replicate from the start of the day and any incremental up until the last 15 minute checkpoint, and still have all the changes.

Keeping those 5 snapshots up to date doesn’t make a particular impact on performance, as far as I can tell.

By: Arjen Lentz

Arjen Lentz — Sun, 09 Nov 2008 22:14:19 +0000

Useful.
From the above description, I do conclude that the more snapshots you have on your system, the more overhead there will be in the filesystem when block are changed, as the ZFS will need to spend time updating the info in the various snapshots.
So this would be something to keep in mind, otherwise it will start affecting performance, right?