Seven Steps to Test Automation Success

In starting a new test automation group at work, I started doing some industry research to get some do/don't ideas. I really, really dislike the idea of GUI x,y coordinate automation; other GUI object-aware automation tools seem to be pricey and still seem like tons of maintenance and really distanced from testing the real meat of most apps. Most info that I've found on the topic has either dealt with things at the GUI level (probably since that's "easiest" to implement, despite being difficult to maintain), or at the unit level--while these focuses certainly have their merits, I was looking for info on the level in between those things; Google calls this the "medium" level.

I found this article by Brett Pettichord called "Seven Steps to Test Automation Success", where he talks about the difficulties in automating tests, and I really think he hits a lot of nails on the head. He argues that in developing automate tests, you should really treat your work like that of software development, then discusses his seven steps:

1. Improve the Testing Process
   
2. Define Requirements
   
3. Prove the Concept
   
4. Champion Product Testability
   
5. Design for Sustainability
   
6. Plan for Deployment
   
7. Face the Challenges of Success
   

In each step, Brett focuses on laying lots on the table--and a lot of times things that I hadn't really processed as stuff that needed to be on the table. I like how he consistently hints at making the app under test testable, thus driving the ease of use, not just for automaters & testers, but for users as well. I think he does a wonderful job describing how test automation teams can bridge the gap between both development and Black Box testing, as I think this is definitely a challenge that I'm going to face in building my team.

All in all, there's tons of meat in this article, so take it in sections or get a cup of coffee and get comfy, but either way I think this is going to be a reference of mine for quite some time now.

The BS7925-2 Standard for Software Component Testing

This doc is now 11 years old, but has some really good basic ideas for proceeding through a release cycle.  While on first glance the doc looks to be ginormous, it's not really that bad; the meat of it is really covered in the first 30%. Sections of interest:

• 2.1.1.8 mentions the order in which test activities should be done--it includes Component Test Specification (sandwiched between Planning and Execution)
• 2.3 "Component test specification".  In brief, tests are written here using the techniques that were determined doing Planning (read: tests weren't written during Planning, but the techniques were chosen then)
• 3 "Test Case Design Techniques".  Concisely describes popular and useful techniques for creating test cases for a component.
• 4 "Test Measurement Techniques".  These aren't criteria, but rather methods for helping to figure out progress--and maybe setting criteria based on this info.  They show how to do this for each test case technique type in section 3 (most are pretty obvious, but it's still nice to see on paper).

Grab some coffee and take a gander, here.  Oh ya, and in case you're familiar with this group, this doc was produced by the BCS SIGIST.  (seriously, why?  I guess the real question is why "British Computer Society Specialist Interest Group in Software Testing"?  Reminds me of Monty Python's Life of Brian...)

softwaretestingsucks.com

I'm not exactly sure how this got started, but there are actually a few decent articles/pages of info in here on the basics of testing:

• What is Quality?
• Life Cycle testing
• Testing Types
• Testing Techniques
• Testing Tools
• Certification Programs
• Testing Jokes

Beware--in that last one, the jokes are uber-cheesey, and mostly only remotely entertaining to the Test Geek... Check it out here.

Let's get mathy

Pairwise testing is a technique that was introduced to about 6 months ago and have really looked forward to getting the chance to use since then.  I re- ran across a list of tools to help generate lists of test cases to run according to the possible input data (variants).  When the time is right, I wanna try some of these out.