Just a quick update on the state of Drizzle’s transaction log as it’s been a while since I’ve mentioned it.

As I wrote earlier, we have already spent some time on basic tests of the transaction log structure – i.e. can we capture INSERT/UPDATE/DELETE/etc correctly?

Well, the next phase of testing is underway and we’re now beating up on things with the randgen!

At present, we are working with a single thread and throwing a large number of randomly generated queries at the log. We have the randgen set up so that it runs 20-30 queries per cycle and we run several hundred cycles. Once these queries have been executed, we make use of drizzled/message/transaction_reader to create SQL from the transaction log contents.

The example below assumes you’ve started the server via ./test-run –start-and-exit –mysqld=–transaction-log.enable and have created some tables to play with.  We call the transaction_reader like this:

drizzled/message/transaction_reader var/master-data/transaction.log

As an example, a query like:

UPDATE dd SET col_bigint=5 LIMIT 5;

Will show up as:

START TRANSACTION;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=3389;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=2329;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=3634;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=2369;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=3674;
COMMIT;

We then send this SQL to a validation server (just another instance of Drizzle) and then compare drizzledump files between the master and the slave.

So far, we’ve found a couple of new crashes, some minor issues with the transaction_reader program, and a couple of issues where the log actually fails to capture data (only UPDATEs have failed in this way so far). I’d like to give a special mention to David Shrewsbury and Joe Daly for their awesomely fast responses to the bugs I’ve found so far : ) We maintain a list of all  transaction log bugs we have found with the testing blueprint.  Most of these bugs are already closed (thanks Dave and Joe!).

Our next steps will be to tweak our single-threaded grammars a bit further, then we will move on to concurrent testing. We’ll be repeating the testing process I laid out above, except that we will let the multiple threads run to completion, say five to ten thousand queries apiece, and then replicate and validate. At the moment, we’re shooting for testing to be complete in time for next week’s milestone release.