Testing Drizzle with the random query generator

As some of you may know, part of Drizzle’s testing process involves the use of the random query generator (aka randgen aka RQG).  Any patches that want to make it into trunk must survive several randgen tests, among other things.

One of the first things I did upon joining the Drizzle team at Rackspace was to take a look at the RQG tests running in Hudson.  My evaluation included tuning up the existing tests and modifying some MySQL test grammars for use with Drizzle.  It should be noted that randomness is a double-edged sword – it is extremely powerful for finding bugs, but it also requires a certain amount of diligence to produce useful, valid queries.  Once my initial work was complete, we unleashed the new RQG tests and found the following bugs:

At the time of this writing, 6 of these are in ‘Fix Released’ status and 3 more are in work.  I’d like to extend a personal “Thank you” to Prafulla Tekawade for his great work – he has supplied the patches for 5 out of the 6 fixed bugs (!).  Additionally, thanks for Toru Maesaka for his continued work on BlitzDB (which is part of our regular randgen testing).

If anyone is interested in learning more about using the random query generator, I encourage you to check out launchpad.  General information can also be found here and here

4 thoughts on “Testing Drizzle with the random query generator

  1. I used to wonder what is this “Random Query Generator”
    Now i understand it is a great stuff and right weapon in your hand.
    You are doing a great job of hunting down difficult to find bugs.

    • Well, it is a very cool tool. I’ll be writing up another blog soon to describe how you can use it to test upgrades / migration to Drizzle.

      Basically, if you can define the types of queries you want to run, you can set up a test that can be used for stress testing and for results verification (RQG allows for result set compares between 2-3 servers!). I imagine this could be useful, but we’ll see once I’m done writing : )

      Thanks for the reply.

  2. Pingback: Patrick Crews: Testing Drizzle with the random query generator | Weez.com

  3. Hi Patrick
    Thanks for mentioning my name :), It feels nice.
    I am planning to get more involved with drizzle development as time permits (especially in query processing modules).

    Prafulla

Comments are closed.