Hello, SkySQL!

So, as LinuxJedi so eloquently noted here, Rackspace and Drizzle are parting ways.  While they were kind enough to offer other opportunities with them, my preferences were similar to Andrew’s – to remain in the MySQL/Drizzle world.

I was fortunate enough that SkySQL had need of my services and am happy to announce that today marks my first day as a Senior QA Engineer with them.  I am very honored to join such a promising and skilled group and am very excited about the opportunities ahead.

My work will have me developing QA solutions (tests, code, whatever) for a wide variety of things.  Naturally, this includes SkySQL’s Reference Architecture where I will work on tests to ensure the delivered packages work perfectly for our customers (among many, many other things).  Additionally, I’ll be working with tools such as LinuxJedi’s modifications to Domas’ oh-so-tasty mydumper.  Basically, a little bit of everything ; )  One of the first things I’ll be doing is adapting dbqp to work with MySQL.  This will allow us to handle a wide variety of testing challenges from a single, adaptable platform in the MySQL world as well.  Drizzle will be replacing the legacy test-runner with this tool in the very near future.

Speaking of which, I’ve also been encouraged to continue contributing to Drizzle.  I have been speaking with Stewart recently about beefing up xtrabackup tests and putting the random query generator directly in the Drizzle tree.  It’s quite amazing to be in a position where I can collaborate on projects across company lines.  I also think it is particularly cool that Henrik notes SkySQL were the first to do a production install of Drizzle for someone!

With that said, I have plenty on my plate already and it’s time to get back to work : )

New dbqp feature – using pre-created datadirs for tests

Why would one want to do this, you may ask?  Well, for starters, it makes a great ‘canary-in-the-coal-mine‘ in regards to backwards compatibility!

For Drizzle, we’ve created some tables (via the randgen’s data generator if you are curious), saved a copy of the datadir, and then created a test case that uses said datadir for the test server.  The test executes some simple SQL queries to make sure we can read the tables properly.  This way, if we ever do something to either the server or .dfe format (data format exchange – had a most enlightening conversation with the team about this format’s history at the MySQL UC), we’ll have a broken test that cries about it.  From there, we’ll know we have to take some action.  The always-amazing Stewart Smith has also created some foreign key backwards compatibility tests, which I believe marks further progress towards the magical goodness that is catalogs!

We signal that we want to do this by using a .cnf file:

servers = [[]]


Each server is named s0,s1,sN.  If a server name is contained in the .cnf file, the test-runner will do the appropriate magic to use the specified datadir for that server.  The argument to load-datadir is the name of the directory that is intended for use in the test.  All datadirs are expected to live in drizzle/tests/std_data.  Tests that do use a .cnf file, like main.backwards_compatibility and slave.basic are skipped by test-run.pl automatically (you *can’t* run them via test-run.pl).

This is something that I don’t believe could be accomplished with the old test runner, or at least not *easily* done (see Rube Goldberg) ; ).  At some point, we will switch over to dbqp entirely and remove test-run.pl.  Seeing comments like this, makes me happy and think things are on track.

dbqp was created with the idea that it should be easy to express complex testing setups (multiple servers, using a preloaded datadir, etc, etc) and it looks like the incubation is starting to pay some benefits.  In addition to allowing this voodoo to happen, the code I’ve added to the test runner will allow us to start doing proper tests of the super Mr. Shrewsbury’s multi-master replication.  Joe Daly has also been doing some very promising work for hierarchical replication based on Dave’s tree.  I’ll be creating some example tests for these badass features soon.  The moral of the story is that by rethinking our test-runner, one tiny bit of code helps us move the ball forward on testing replication, backwards compatibility, and catalogs.

It’s honestly one of the best parts of working on the Drizzle project – being encouraged to experiment and rethink problems has enabled all sorts of innovation (but one example of Monty Taylor’s computing wizardry!) and cool features.  Thanks to this freedom to experiment, we now have even more ways of making sure we are producing quality code.

My view of QA is that we do help test, but that we also help other people answer their own questions about quality (via tools, documentation, examples, etc).  Ultimately, a test is a question – “Do you return the right answer for this query?”, “Can you survive a beating from the randgen?”, etc – and asking questions should be easy and informative.  QA shouldn’t be the sole province of some obscure priest class, but everyone’s playground.  When I see developers like Stewart writing interesting test cases and even contributing to the test tool itself, I’m even happier than when I find a bug (and finding bugs is quite awesome!).

Anyway, the code is proposed for a merge to trunk and documentation is available (under testing/writing test cases).  I hope that this makes trying to break things even more fun for people >: )