Friday, September 11, 2009

PBXT 1.0.09 RC3 implements XA and online backup

I have just released PBXT 1.0.09 RC3. Besides bug fixes (details in the release notes), this version includes 2 Beta features:
  • XA/2-Phase Commit support
  • Native online backup Driver
XA support has been around MySQL for quite a while, and we all know of it usefulness, for example when sharding. So I was surprised to find a bug in the XA recovery: Bug #47134. Contrary to what is reported, the crash can also occur when using XA with just the default engines installed, so watch out for that one (the good news: the bug fix is simple).

Online backup is really cool! I have heard that it may soon be released in a coming version of 5.4, so lets hope that this is true.

In a little test, I did a backup of a 10GB database in 49.26 seconds! Admitedly this was on a system with 4 15K drives in a RAID 0 configuration. But that is still a fantastic, considering the tables are not even locked during this time!

The database itself took 19 min. 56 sec. to generate. A complete restore took only 14 min. 29 sec.

But, it gets even better....

I have been working on PBXT 1.1, where I have done a number of things to improve the I/O performance of the engine.

In the same test as above, run with PBXT 1.1, the time to generate the database was 9 min. 35 sec., and the time to restore was 6 min 18 sec! (Time to generate the backup was identical.)

PBXT 1.1 is available directly from Launchpad here: lp:~pbxt-core/pbxt/staging, if you are interested in trying it out. 1.1 also has full support for memory based tables.

The new release candidate (PBXT 1.0.09) can be downloaded from primebase.org/download. It is also available from Lauchpad as the rc3 series: lp:pbxt/rc3.

Please report bugs here.

Any feedback is welcome! You can use Launchpad questions or the PBXT mailing list for this purpose.

11 comments:

Mark Callaghan said...

Paul,

It is nice to see PBXT make so much progress.

Is there any chance that you can fix these restrictions on XA in MySQL while you are at it?

Hakan said...

Paul,

great to see that PBXT is moving forward in a fast pace.

I would love to see PBXT building on PPC.

Cheers,

Hakan

arjenAU said...

MySQL used to go kaboom if you had two engines with XA support, and I'm not sure the work was ever completed. How does this go now?
Would one need to disable InnoDB to make PBXT happy, or is it actually possible to a) have both active, b) XA enabled, c) run a consistent transaction covering both engines?

Paul McCullagh said...

Mark,

I took a look at those restrictions. There is a quite a bit of work to be done there. The most serious problem seem to be in conjunction with the binary log: "the transaction will be absent from the binary log even though it has been committed". That is probably the problem that most disturbs you. But, Google is a generous sponsor of such things. If they really need it, you and I know that there are very competent people in the community. You guys don't have to do it all yourselves! ;)

Paul McCullagh said...

Hakan,

Yup, I know that MariaDB got stuck on that one last. I will pull out my old PowerPC notebook, and see if it still boots... :)

Paul McCullagh said...

Arjen,

MySQL went kaboom pretty quickly when I enabled the XA functions in PBXT. So fast, that I was surprised myself.

First an assertion failed, and then it crashed due to a rather basic error: (char *) cast was missing from a pointer addition.

This made me wonder how much the code has been tested, so your concern is justified.

What I can say so far is, comment out the assertion, and make the bug fix, and then XA does work with PBXT, even with InnoDB enabled.

Next step would be to check if can really combine tables of both engines in a single transaction. But I have not tested this yet.

Mark Callaghan said...

Paul,

Google is generous and gives back to open-source. But I am not there anymore. I work at Facebook and blog at http://facebook.com/MySQLAtFacebook.

What is your interest in XA?

Paul McCullagh said...

Hi Mark,

Sorry, it's just taking me a while to get used to the idea that you are no longer at Google.

You and Google had become synonymous to me.

Well, Google's (significant) loss is Facebook's gain!

Congratulations on the new job! :)

The main reason for my interest in XA is because it's a requirement of a important sponsor of the engine.

Mark Callaghan said...

Paul,

That is a great reason to care about XA. Do they need XA and replication? If so, then you have to fix it so that PREPARE, restart server, COMMIT logs binlog events. Or just do replication via your storage engine as you have mentioned in the past.

Paul McCullagh said...

Marc,

These are significant problems with XA and replication that I was not aware of before. We will have to carefully analyze the consequences, and the effort to fix the problem.

As you say, engine level replication would solve this. So that may be a good alternative for us.

Shlomi N. said...

I'm anxious to test PBXT!