Seven Steps to Test Automation Success

In starting a new test automation group at work, I started doing some industry research to get some do/don't ideas. I really, really dislike the idea of GUI x,y coordinate automation; other GUI object-aware automation tools seem to be pricey and still seem like tons of maintenance and really distanced from testing the real meat of most apps. Most info that I've found on the topic has either dealt with things at the GUI level (probably since that's "easiest" to implement, despite being difficult to maintain), or at the unit level--while these focuses certainly have their merits, I was looking for info on the level in between those things; Google calls this the "medium" level.

I found this article by Brett Pettichord called "Seven Steps to Test Automation Success", where he talks about the difficulties in automating tests, and I really think he hits a lot of nails on the head. He argues that in developing automate tests, you should really treat your work like that of software development, then discusses his seven steps:

1. Improve the Testing Process
   
2. Define Requirements
   
3. Prove the Concept
   
4. Champion Product Testability
   
5. Design for Sustainability
   
6. Plan for Deployment
   
7. Face the Challenges of Success
   

In each step, Brett focuses on laying lots on the table--and a lot of times things that I hadn't really processed as stuff that needed to be on the table. I like how he consistently hints at making the app under test testable, thus driving the ease of use, not just for automaters & testers, but for users as well. I think he does a wonderful job describing how test automation teams can bridge the gap between both development and Black Box testing, as I think this is definitely a challenge that I'm going to face in building my team.

All in all, there's tons of meat in this article, so take it in sections or get a cup of coffee and get comfy, but either way I think this is going to be a reference of mine for quite some time now.

RANT: writing 1000 test cases is not that bad

I've heard a number of times over the past few months here at work things like: "...but if we do it like that, that means I'll be writing test cases forever!"  Testers: do you think developers say that when they're assigned to engineer some feature?  If they do, they probably won't be keeping their job long.  When assigned to engineer a feature, a developer goes through some process to figure out what it is they need to do/not do, then write the code that makes the thing do that, one line of code at a time, until the thing does what they think it should do.  What's so craaaazy about testing those lines of code in their different permutations??  Sure, there's merit in testing efficiently, but that's beside the point.  The point is, you do what it takes to get the job done--if that means writing 1000 new test cases in the next month, then that's what it means.

Testing is an Engineering discipline

In fact, as I understand it, in many companies Testing is the department that imposes discipline on Development--which shouldn't be.  SW Development Engineers have a set of steps that they go through when developing an application/component/whatever, which is different than the steps that SW Test Engineers go through--but each have their purposes.  It is not Test's job to ensure that SW Development Engineers have gone through their steps properly; it is however, Test's job to ensure that the product of Development's efforts is something suitable for what it was intended for. Next, SW Test Engineers--please please pleeeeeeaaassseee take some accountability for your job title!!  Check out wikipedia's entry for Engineering:

Engineering is the discipline and profession of applying scientific knowledge and utilizing natural laws and physical resources in order to design and implement materials, structures, machines, devices, systems, and processes that realize a desired objective and meet specified criteria. More precisely, engineering can be defined as “the creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.”

Engineering - Wikipedia, the free encyclopedia ...and notice the word "discipline".  Development Engineers can (and should be) hounded for bad formatting and not commenting their code--along those same lines of being disciplined, let the rest of us know what you're trying to accomplish when writing your tests!  Be explicit about what you're trying to test for, show steps for how to test for that, and be concise about what to expect.  Just as a screwy, uncommented piece of code that someone else wrote can drive a Development Engineer nuts, a screwy test can have the same effect on your fellow Test Engineers.  Please, learn to be disciplined in your work--you'll gain respect from your co-workers and probably realize that you missed a few things along the way.  It's not hard--it just takes discipline!