Just a quick update on the state of Drizzle’s transaction log as it’s been a while since I’ve mentioned it.

As I wrote earlier, we have already spent some time on basic tests of the transaction log structure – i.e. can we capture INSERT/UPDATE/DELETE/etc correctly?

Well, the next phase of testing is underway and we’re now beating up on things with the randgen!

At present, we are working with a single thread and throwing a large number of randomly generated queries at the log. We have the randgen set up so that it runs 20-30 queries per cycle and we run several hundred cycles. Once these queries have been executed, we make use of drizzled/message/transaction_reader to create SQL from the transaction log contents.

The example below assumes you’ve started the server via ./test-run –start-and-exit –mysqld=–transaction-log.enable and have created some tables to play with.  We call the transaction_reader like this:

drizzled/message/transaction_reader var/master-data/transaction.log

As an example, a query like:

UPDATE dd SET col_bigint=5 LIMIT 5;

Will show up as:

START TRANSACTION;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=3389;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=2329;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=3634;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=2369;
UPDATE `test`.`dd` SET `col_bigint`=5 WHERE `pk`=3674;
COMMIT;

We then send this SQL to a validation server (just another instance of Drizzle) and then compare drizzledump files between the master and the slave.

So far, we’ve found a couple of new crashes, some minor issues with the transaction_reader program, and a couple of issues where the log actually fails to capture data (only UPDATEs have failed in this way so far). I’d like to give a special mention to David Shrewsbury and Joe Daly for their awesomely fast responses to the bugs I’ve found so far : ) We maintain a list of all  transaction log bugs we have found with the testing blueprint.  Most of these bugs are already closed (thanks Dave and Joe!).

Our next steps will be to tweak our single-threaded grammars a bit further, then we will move on to concurrent testing. We’ll be repeating the testing process I laid out above, except that we will let the multiple threads run to completion, say five to ten thousand queries apiece, and then replicate and validate. At the moment, we’re shooting for testing to be complete in time for next week’s milestone release.

We have now upgraded our lcov testing to take advantage of lcov 1.9‘s branch coverage.

While code coverage isn’t the be-all-end-all of testing, it is a very useful tool in helping us target areas that could use more tender, loving care (by which I mean beating them mercilessly with our test tools).  It doesn’t prove completeness of testing – it merely helps us move in that direction : )  The addition of branch-level coverage gives us another dimension to help us expand our testing.  We’re also making use of the –demangle-cpp option to produce neater function names for the function-level coverage.

You can check out the updated reports here.  We gather the code coverage of our test suite with every push to trunk and store the data for general analysis.

In other news, Brian has added two new functions for testing.  These changes were a part of another blueprint to remove print_stack_trace and stack dump. While I haven’t had a chance to do anything with them yet, I think these will be very useful and look forward to seeing what kinds of tests our community will be able to cook up : )

• Crash the server:

% ./drizzled/drizzled –plugin-add=crash_function

select crash();

• Shutdown the server:

% ./drizzled/drizzled –plugin-add=shutdown_function

select shutdown()

Finally, I have been playing with the syslog plugin and it is awesome!  It is easy to setup and is very easy to use.  The data produced will be very useful in testing as well.  An example produced via a randgen run for your enjoyment: