tag:blogger.com,1999:blog-91224518618114000482024-02-08T12:17:25.480-08:00Software PhilosophiaSoftware Test and Development nuggets with an occasional morsel of Philosophy. ...from an expert on neither.turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.comBlogger37125tag:blogger.com,1999:blog-9122451861811400048.post-75277816835227909322010-02-13T13:30:00.001-08:002010-02-13T13:30:05.152-08:00I think I moved blogs?<p style="clear: both">Hi all... I've been trying out tumblr.com for a month or so now for my blogging-ish outlet and have been really digging it. For my 7 followers out there, I think I've made the decision to post solely over there--check it at <a href="http://musicismyeverest.tumblr.com/" target="_blank">http://musicismyeverest.tumblr.com</a>. I've actually moved all my posts from here to there too!</p><br class='final-break' style='clear: both' />turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-5733534562266255222010-01-16T17:48:00.001-08:002010-01-16T17:48:52.896-08:00Seven Steps to Test Automation Success<p style="clear: both"><a href="http://lh5.ggpht.com/_tVHLoqwNJx4/S1Jsg3lr-rI/AAAAAAAABQc/n7cdDyDyNCw/s800/automation1.jpg" class="image-link"><img class="linked-to-original" src="http://lh3.ggpht.com/_tVHLoqwNJx4/S1JsgdwPGFI/AAAAAAAABQY/2F9sVblg-EM/s800/automation1-thumb.jpg" height="325" align="left" width="325" style=" display: inline; float: left; margin: 0 10px 10px 0;" /></a><br style="clear: both" />In starting a new test automation group at work, I started doing some industry research to get some do/don't ideas. I really, really dislike the idea of GUI x,y coordinate automation; other GUI object-aware automation tools seem to be pricey and still seem like tons of maintenance and really distanced from testing the real meat of most apps. Most info that I've found on the topic has either dealt with things at the GUI level (probably since that's "easiest" to implement, despite being difficult to maintain), or at the unit level--while these focuses certainly have their merits, I was looking for info on the level in between those things; Google calls this the "medium" level.</p><p style="clear: both">I found this article by Brett Pettichord called "<a href="http://www.io.com/~wazmo/papers/seven_steps.html">Seven Steps to Test Automation Success</a>", where he talks about the difficulties in automating tests, and I really think he hits a lot of nails on the head. He argues that in developing automate tests, you should really treat your work like that of software development, then discusses his seven steps:<br /><ol style="clear: both"><li>Improve the Testing Process<br /></li><li>Define Requirements<br /></li><li>Prove the Concept<br /></li><li>Champion Product Testability<br /></li><li>Design for Sustainability<br /></li><li>Plan for Deployment<br /></li><li>Face the Challenges of Success<br /></li></ol>In each step, Brett focuses on laying lots on the table--and a lot of times things that I hadn't really processed as stuff that needed to be on the table. I like how he consistently hints at making the app under test testable, thus driving the ease of use, not just for automaters & testers, but for users as well. I think he does a wonderful job describing how test automation teams can bridge the gap between both development and Black Box testing, as I think this is definitely a challenge that I'm going to face in building my team.</p><p style="clear: both">All in all, there's tons of meat in this article, so take it in sections or get a cup of coffee and get comfy, but either way I think this is going to be a reference of mine for quite some time now.</p><br class='final-break' style='clear: both' />turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-27542034617935897202010-01-16T16:15:00.001-08:002010-01-16T17:49:17.054-08:00Ruby Net::HTTP Cheat Sheet<p style="clear: both">I think this will prove to be a good resource--especially since I don't use this library that often. Hope you find it useful...<br /><a href="http://www.rubyinside.com/nethttp-cheat-sheet-2940.html">Net::HTTP Cheat Sheet</a></p><br class='final-break' style='clear: both' />turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-76583855310807892532009-11-04T11:48:00.000-08:002009-11-04T11:48:10.392-08:00Clean IRCing on OSX<p style="clear: both">I've been IRCing at work quite a bit lately, which has been pretty great. Every now and then I find it exciting to return back to old tech and start using it again--it's usually simple and complicated all at the same time, but for some reason I find that stimulating.</p><p style="clear: both"><div><u><br /></u>In any case... I used <a href="http://colloquy.info/" target="_blank">Colloquy</a> for a while, which is a great modern IRC client, but after dealing with some annoying bugs, downloading the source, rebuilding, and not getting any love, I decided to search for something new. And I found <a href="http://irssi.org/" target="_blank">irssi</a>. irssi is totally old school, but totally new school. I love it. There are a ton of getting started guides out there, but it was nice to find a Mac-specific one: <a href="http://quadpoint.org/articles/irssi">A Guide to Efficiently Using Irssi and Screen | quadpoint.org</a>; definitely worth checking out.<br /></div></p><p style="clear: both"><div><br />My irssi'ing got even better when combining with Blacktree's (makers of Quicksilver) <a href="http://docs.blacktree.com/visor/visor" target="_blank">visor</a>. Visor is just a simple HUD-type SIMBL plugin for Terminal.app that allows for key-combo hiding/unhiding. This is great for moving my IRC client (Terminal.app + irssi) on/off screen quickly, keeping my desktop clean so I can focus on my work.<br /></div></p><p style="clear: both"><div><br />Just thought I'd share with y'all...</div></p><br class='final-break' style='clear: both' />turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-39418146622969840062009-05-19T17:10:00.001-07:002009-05-19T17:10:53.539-07:00GTAC 2008 Keynote Address: The Future of Testing<div xmlns='http://www.w3.org/1999/xhtml'><p><object height='350' width='425'><param value='http://youtube.com/v/Pug_5Tl2UxQ' name='movie'/><embed height='350' width='425' type='application/x-shockwave-flash' src='http://youtube.com/v/Pug_5Tl2UxQ'/></object></p><p>Quite a thought provoking speech. I was fortunate to see this in person! Take an hour and check it out--it's totally worth it.</p></div>turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-90815438195736917782009-03-04T02:43:00.000-08:002009-03-04T02:44:30.615-08:00DensityThis ended my late night work marathon with a happy note:
<img src="http://imgs.xkcd.com/comics/density.png"></img>turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-61066298641310649072009-02-23T17:56:00.001-08:002009-02-23T17:56:41.665-08:00Hitler's nightly build fails<div xmlns='http://www.w3.org/1999/xhtml'><p><object height='350' width='425'><param value='http://youtube.com/v/Azl4nqLn4-Y' name='movie'/><embed height='350' width='425' type='application/x-shockwave-flash' src='http://youtube.com/v/Azl4nqLn4-Y'/></object></p><p>Oh man this is good. We should hire (no, not "heil") Hitler.... I love the part where the lady consoles the other lady saying, "Don't worry, he didn't mean that about Scrum." Genius.</p></div>turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-53635826911294457432008-09-26T19:38:00.001-07:002008-09-26T19:38:40.687-07:00Politics should be like Software DevelopmentI'm watching the presidential debates. Each candidate makes claims that the other said "this", then the other retorts with, "No, I said 'that'." Back and forth. So how do you know who's telling the truth, who's lying, who's twisting the truth, and who's misunderstood who?
How great would it be to have a list of things that each candidate stated at any given time, then when whoever gets elected, we can all see if they've actually done what they said they'd do. It'd be kinda like SW requirements for some project--you define them, your teams implement them in the software, then other teams make sure they were implemented as stated, then you all get together and talk about (and tell the rest of the company) the good things you did and the bad things you did.
Who in the public actually keeps our president accountable? I sure couldn't tell you what Dubbya claimed he did and if he did it. It just doesn't seem like there's much accountability for the things these guys say they're going to do.
Also, it seems like these guys don't know if the other is talking about strategy, tactics, or post-mortem topics. One attacks the other's strategies, then the first justifies with their tactics, then the first attacks back talking about the results of everything. Is it that unclear? It just seems like candidates thrive (or are told they'll thrive) on attacking the other, and in order to retort "politically", they work around the issues and talk about something else.
It's just annoying at how much of a TV show these campaigns have become--candidates can say whatever they want to say as long as it entertains the public. They work on our emotions, using planned out catch phrases and directed topics. But once the winner gets in office, who's there to keep them accountable?turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-3450018494246561382008-07-25T10:56:00.001-07:002008-07-25T10:56:22.120-07:00Vista auto-reboot-for-updates #4*sigh* I guess I'm used to this now, so it's not so surprising, although just as annoying. I was just setting up a meeting in Outlook when all of the sudden Outlook just started going in and out of focus, then *poof* everything closed and it was rebooting. Once again, thanks POS Vista for not considering that I was actually working on something before you decided to reboot--reboot without ever having asked me if that's what I wanted. I actually like how OS X assumes a lot for me, but never does it assume that I want to throw my work to the wind.
I really just don't get how this is good behavior. Judging by the frequency of my blogs on the topic, this happens about once every 3 - 6 weeks since April. Amazing. My opinion of MS continues to plummet...turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com1tag:blogger.com,1999:blog-9122451861811400048.post-90015277432401834112008-06-12T09:50:00.001-07:002008-06-12T09:50:58.846-07:00Vista auto-reboot-for-updates #3Yet again, I'm in the middle of my morning emailing, and all of a sudden, Vista decides to reboot without warning. How the hell could M$ ever decide this was a good idea? I really can't think of anything more frustrating than losing work. And oh ya... it's been "Configuring updates" for the past 10 minutes now... awesome. *sigh*turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-75575763134112006662008-06-11T10:02:00.001-07:002008-06-11T10:02:35.820-07:00RANT: writing 1000 test cases is not that bad<img src="http://jophan.org/mimosa/m30/jwc-yell.gif" /> I've heard a number of times over the past few months here at work things like: "...but if we do it like that, that means I'll be writing test cases forever!" Testers: do you think developers say that when they're assigned to engineer some feature? If they do, they probably won't be keeping their job long.
When assigned to engineer a feature, a developer goes through some process to figure out what it is they need to do/not do, then write the code that makes the thing do that, one line of code at a time, until the thing does what they think it should do. What's so craaaazy about testing those lines of code in their different permutations?? Sure, there's merit in testing efficiently, but that's beside the point. The point is, you do what it takes to get the job done--if that means writing 1000 new test cases in the next month, then that's what it means.turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-47931909865089775662008-06-04T10:08:00.001-07:002008-06-04T10:12:05.545-07:00Words (for all developers) To Live By<img src="http://i2.photobucket.com/albums/y1/lhomme77/knit3.jpg" width="400" />
In coming up with a presentation for work on characteristics of quality, I've done a lot of expanding my brain. Lots of defining, clarifying, quantifying, and a fair amount of reading. This morning I took another looks at Apple's ADC guide for Human Interface Guidelines--particularly the part on <a href="http://developer.apple.com/documentation/UserExperience/Conceptual/OSXHIGuidelines/XHIGCharGreatSoftware/chapter_4_section_1.html">Characteristics of Great Software</a>. I'm pretty sure they'd updated it since the last time I'd been there, as I noticed a link on that page to another page called "<a href="http://developer.apple.com/documentation/UserExperience/Conceptual/OSXHIGuidelines/XHIGDesignProcess/chapter_3_section_2.html#//apple_ref/doc/uid/TP40002718-BAJHCHDB">Know Your Audience</a>". I followed.
First nugget:
"It is useful to create scenarios that describe a typical day of a person who uses the type of software product you are designing."
Second nugget:
"Develop your product with people and their capabilities—not computers and their capabilities—in mind."
Third nugget:
"It is not your needs or your usage patterns that you are designing for, but those of your (potential) customers."
Read it.
Enough said.turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-87322067503021377502008-05-29T11:51:00.000-07:002008-05-29T12:03:52.032-07:00Vista & Its Own Mind #2Yet again, Vista presents the pleasant surprise of exiting all my work and rebooting, just to apply updates that it never told me about. I'm appalled for the second time now.
<span style="font-style:italic;">Update:</span>
And Outlook didn't even save the emails I had written but not yet sent... So freaking annoying.turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-48537623094211059322008-05-28T15:04:00.001-07:002008-05-28T15:05:30.069-07:00QC's: Maturity v. Stability<div xmlns='http://www.w3.org/1999/xhtml'><img align='left' src='http://www.engrish.com/image/engrish/personal-data-breaker.jpg'/><br/>ISO 9126 presents the concepts of "quality characteristics", which is proving to be both helpful and confusing here at work. We've had our longest group process meetings on this topic--just trying to figure out and agree on what each one of these means. One of the big problems is figuring out what some of these are <i>not. </i>We use many of these words loosely in this world of testing/development, thus setting ourselves up for confusion when talking about this stuff.<br/><br/><b>Technical Malapropisms</b><br/>Now, no one likes a vocal grammar Nazi around, but sometimes it really is important to talk about the distinctions between different things you talk about on a day to day basis. <br/><br/>One of the many bits of confusion came when talking about "Stability". Stability falls (as a sub-characteristic) under the main characteristic of "Maintainability", which can make sense if you take a minute to ponder. When we started trying to come up with examples of this, however, most of us came up with examples of "Maturity", which falls under "Reliability." It seems that in non-ISO conversations, stability is talked about in a way that encompasses <i>both </i>of these sub-characteristics. I did some brief google-ing to see if we were the only ones with this "problem" and found an abstract on <a href='http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VBS-44HYJ5B-8&amp;_user=10&amp;_rdoc=1&amp;_fmt=&amp;_orig=search&amp;_sort=d&amp;view=c&amp;_acct=C000050221&amp;_version=1&amp;_urlVersion=0&amp;_userid=10&amp;md5=5ad66080fe3bfba352ff1ea6f649e482' target='_blank'>Ecological Modelling: An inverse relationship between stability and maturity in models of aquatic ecosystems</a>. <a href='http://www.stsc.hill.af.mil/crosstalk/1997/08/product.asp' target='_blank'>This article</a> on software maturity models suggests that stability is a factory that makes up maturity. ISO says otherwise:<br/><br/><blockquote>Maturity:<br/><font face='Courier New'>The capability of the software product to avoid </font><font face='Courier New' style='background-color: yellow;'>failure as a result of</font> <font face='Courier New' style='background-color: rgb(0, 255, 0);'>faults in the software.</font><br/><br/>Stability:<br/><font face='Courier New'>The capability of the software product to avoid </font><font face='Courier New' style='background-color: yellow;'>unexpected effects from</font> <font face='Courier New' style='background-color: rgb(0, 255, 0);'>modifications of the software.</font><br/></blockquote><br/>Maturity deals with "failure as a result of faults"; Stability deals with "unexpected effects from modifications."<br/><br/><b>Failures v. Unexpected Effects</b><br/>Maturity deals with <i>failures</i>; Stability deals with <i>unexpected effects</i>. What's the difference? ...in terms of test cases passing I say, nothing. Both are the absence of passing; it doesn't matter what you call it. ...unless this means to say that a product is unstable if changes in module A unexpectedly effect the behavior in module B. One could then interpret "failure" as the inability for module A to meet specification for module A when testing module A; "unexpected effects" are then the inability for module A to meet specification for module A when testing module B.<br/><br/><b>Faults v. Modifications</b><br/>Maturity deals with <i>faults in the software</i>; Stability deals with <i>modifications of the software</i>. What's the difference? Key differences become possibly more confusing when you take a look at the definition and notes of the main characteristics for each sub-characteristic. Note 1 for "Reliability" (parent of Maturity) says:<br/><blockquote>"Wear or aging does not occur in software. Limitations in reliability are due to <u>faults</u> in requirements, design, and implementation. <u> Failures</u> due to these faults depend on the way the software product is used and the program options selected rather than on elapsed time."</blockquote>Interesting. A fault can be in the requirements. A fault can be in the design. A fault can be in the implementation. Lack of maturity, then, is failures in one of these fault areas. The 2nd sentence in the definition of "Maintainability" (parent of Stability) says:<br/><blockquote>"Modifications may include corrections, improvements or adaptation of the software to changes in environment, and in requirements and functional specifications."</blockquote>A modification requires that something exist already; a new "version" of that thing comes about after modifications are made. Stability, then, is strictly tied to how the application behaves in comparison to how it behaved in the last version. Lack of stability is evident, then, when the same test activities are conducted in two different versions of the application, both in the same environment, and failures occur in unexpected areas of the application.<br/><br/><b>Conclusions</b><br/>So one might be led to conclude that when testing version 2 of some app, you really could run test A and essentially be testing for both stability and maturity. But like many of these other ISO characteristics, the line is quite fine. The only way I can see it comes back to relate to one thing: what's effected. If you test A after changes were made in A, then you're testing maturity of A. If you test [B-Z] after making changes in A, you're testing stability of the app.</div>turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-84174557136828911722008-05-21T15:38:00.001-07:002008-05-21T15:38:30.151-07:00Sneaky<div xmlns='http://www.w3.org/1999/xhtml'><div style=''><img src='http://www.osnews.com/images/comics/skyos.jpg' style='max-width: 800px;'/><br/></div></div>turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-9425920822302316372008-05-14T23:08:00.001-07:002008-05-14T23:17:02.391-07:00Teach me to test<img src="http://www.bugbash.net/strips/bug-bash20070115.gif" align="left" width="450">
In seeking to appease my recent desires to get back in to writing code, I flirted with the idea of purchasing a book on Xcode and Objective-C so I could get crackin' on some Mac dev. With my other recent desire of cutting back on spending, I decided to check out <a href="http://cocoalab.com/?q=becomeanxcoder">cocoalab.com's free eBook on the topic</a>. While it covers the uber-basics of getting in to development, it also covers the uber-basics of Xcode, which is what I really need. It also does this on Leopard--something no books on the market can tout yet (so I've read).
So I blew through the first 6 chapters before having to attend my roomie's bday party, and am excited to get back to it ASAP.
It just occurred to me though, that while the book talks about debugging in Xcode, it barely talks about testing (well, so far). And then it occurred to me: most development books that I've ever read don't really make many suggestions about testing at all--much less about how or when to test the code you wrote. I realize that <a href="http://en.wikipedia.org/wiki/Test-driven_development">Test-driven Development</a> is really a <span style="font-style: italic;">suggested</span> technique, but it seems to me that if developers at least followed these concepts, they would find more success. Thus, if books taught how to test your code in equal doses at they taught how to write code, we might see a <a href="http://www.nist.gov/public_affairs/releases/n02-10.htm">reversal in the economy</a>. :-)turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-68995518912935047652008-05-05T16:57:00.001-07:002008-05-05T16:57:31.871-07:00A test case is a scientific procedure<img src="http://www1.pvsd.net/CMSWebsite/Joi/SciFair_CW_MP_07.jpg" style="margin: 10px 10px 0pt 0pt; float: left;" title="" alt="" /><font size="4"><span style="font-style: italic;">...so treat it as such!</span> </font> After reading through hundreds of test cases at work in the past couple of weeks, I'm getting extremely frustrated seeing test cases that don't state their point, nor how to know if what happens when running the test was good or bad (pass/fail). I've seen countless functional test case summaries that state how to do something (instead of what the point of the test is); ...countless test descriptions that state only the object under test (instead of how acting on that object will result in some expected outcome); ...and countless "expected results" sections with sentences that aren't even complete sentences, let alone describe how to determine if the test passed or failed.
We all did experiments in junior high, right? In its simplest form, a test case is just like that--make your hypothesis, plan the steps out you need to "run" in order to try to prove your hypothesis, get your gear together, run through your steps, then get a poster board and some spatter paint, write up your results (plus charts & graphs) and stick them to the board (make sure to not sniff the glue or you'll get in trouble! jk...).
Without even knowing a product, I should at least be able to understand what the point of a test case is when reading it. Next, the test case must have a goal of achieving something explicit, so that when I run through the steps I won't have to make any new decisions about injecting any new data from outside of my initial plan (the initial plan is a direct result of my hypothesis, so injecting new data may not coincide with what I'm trying to prove in the first place). Lastly, I should never see multiple possible expected outcomes as the result for a test--if there are 2 possible paths through a certain component, each path needs its own test case.
Be explicit with one (and only one) point to the test case. Please.turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-58297298032416729362008-04-21T17:18:00.001-07:002008-04-21T17:18:08.331-07:00Testing is an Engineering discipline<img src="http://www.daggettgymnastics.com/martialarts.jpg" ?="" />
In fact, as I understand it, in many companies Testing is the department that imposes discipline on Development--which shouldn't be. SW Development Engineers have a set of steps that they go through when developing an application/component/whatever, which is different than the steps that SW Test Engineers go through--but each have their purposes. It is <span style="font-style: italic;">not </span>Test's job to ensure that SW Development Engineers have gone through their steps properly; it is however, Test's job to ensure that the product of Development's efforts is something suitable for what it was intended for.
Next, SW Test Engineers--please please <span style="font-style: italic;">pleeeeeeaaassseee </span>take some accountability for your job title!! Check out wikipedia's entry for <span style="text-decoration: underline;">Engineering</span>:
<blockquote cite="http://en.wikipedia.org/wiki/Engineering">Engineering is the discipline and profession of applying scientific knowledge and utilizing natural laws and physical resources in order to design and implement materials, structures, machines, devices, systems, and processes that realize a desired objective and meet specified criteria. More precisely, engineering can be defined as “the creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.”</blockquote><cite cite="http://en.wikipedia.org/wiki/Engineering"><a href="http://en.wikipedia.org/wiki/Engineering">Engineering - Wikipedia, the free encyclopedia</a></cite>
...and notice the word "discipline". Development Engineers can (and should be) hounded for bad formatting and not commenting their code--along those same lines of being disciplined, let the rest of us know what you're trying to accomplish when writing your tests! Be explicit about what you're trying to test for, show steps for how to test for that, and be concise about what to expect. Just as a screwy, uncommented piece of code that someone else wrote can drive a Development Engineer nuts, a screwy test can have the same effect on your fellow Test Engineers. Please, learn to be disciplined in your work--you'll gain respect from your co-workers and probably realize that you missed a few things along the way. It's not hard--it just takes discipline!turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-63003441457306641112008-04-16T15:32:00.001-07:002008-04-16T15:41:25.714-07:00US FDA/CDRH: General Principles of Software Validation; Final Guidance for Industry and FDA Staff<img src="http://www.fda.gov/cdrh/images/cdrhlogonew-documents.gif" />I ran across this link while looking for a specific page in my del.icio.us test links, and while the products that this document is geared towards are a little different than what I deal with on a daily basis, the concepts are great. I particularly liked this section--especially the first sentence here--in the Verification & Validation section:
<blockquote cite="http://www.fda.gov/cdrh/comp/guidance/938.html">Software verification and validation are difficult because a developer cannot test forever, and it is hard to know how much evidence is enough. In large measure, software validation is a matter of developing a "level of confidence" that the device meets all requirements and user expectations for the software automated functions and features of the device. Measures such as defects found in specifications documents, estimates of defects remaining, testing coverage, and other techniques are all used to develop an acceptable level of confidence before shipping the product. The level of confidence, and therefore the level of software validation, verification, and testing effort needed, will vary depending upon the safety risk (hazard) posed by the automated functions of the device.
</blockquote>
There's also a great section called "Software is Different From Hardware", which points out some great subtle-but-huge differences between the two.
Section 5. Activities and Tasks has some good practical info on planning and test tasks--both Black Box and White Box (although not explicitly so).
<cite cite="http://www.fda.gov/cdrh/comp/guidance/938.html">
<a href="http://www.fda.gov/cdrh/comp/guidance/938.html">US FDA/CDRH: General Principles of Software Validation; Final Guidance for Industry and FDA Staff</a></cite>turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-88601797054092527782008-04-10T17:09:00.001-07:002008-04-10T17:09:05.550-07:00Vista annoyance<img src="http://files.acdsystems.com/english/newsletters/pics-tips/200701/EN/img/PICS-TIPS-lp-200701-article2-img-1.jpg" style="margin: 10px 10px 0pt 0pt; float: left;" title="" alt="" /> I love how Vista just decides to reboot and apply some updates while I'm in the middle of writing an email. No warnings, nothing. Just shuts down as if I said "Shutdown and kill all my apps in the process"... Nicely done M$. I love it when my OS gives me the finger.turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-6658411903928862322008-04-10T17:07:00.001-07:002008-04-10T17:12:23.156-07:00Apple's Characteristics of Great Software<img src="http://www.ars.usda.gov/is/graphics/photos/may05/D071-8i.jpg" align="left" />A friend & coworker of mine sent me a link to <a href="http://developer.apple.com/documentation/UserExperience/Conceptual/OSXHIGuidelines/XHIGIntro/chapter_1_section_1.html#//apple_ref/doc/uid/TP30000894-TP6">Apple's Human Interface Guidelines</a>, with the intent to give some insight in to testing the UI applications that I test for my job (thanks Mike). After stepping through a few pages in Part I, it clicked for me that a number of the "<a href="http://developer.apple.com/documentation/UserExperience/Conceptual/OSXHIGuidelines/XHIGCharGreatSoftware/chapter_4_section_1.html#//apple_ref/doc/uid/TP40002720-TPXREF101">Characteristics of Great Software</a>" that Apple lists are the same as some of <a href="http://en.wikipedia.org/wiki/ISO_9126">ISO 9126</a>'s quality characteristics.
After attending a <a href="http://www.cvbi.org/">CVBI</a> meeting this morning where the topic of conversation was "building quality in to software products", I'm really encouraged to see Apple not only list these characteristics that it thinks are important, but back up why they think they're important. One of the things that was discussed implicitly in this morning's meeting was that in order to end up with better quality products, developers (and testers) need to be taught at some point what quality is and how to get there. Apple uses this doc to clearly communicate some ways to raise the quality, not just on Apple applications, but on all applications.
Seeing this crossover between the quality characteristics that ISO setup and the ones that Apple setup really gives extra weight to these characteristics. Without having read too much in to the Apple doc yet, it looks like for the characteristics that Apple lists:
<ul><li>2 out of 7 (Ease of Use, Attractive Appearance) are related to ISO's Usability</li><li>1 out of 7 (Reliability) are related to ISO's Reliability</li><li>1 out of 7 (High Performance) are related to ISO's Efficiency </li><li>1 out of 7 (Interoperability) are related to ISO's Functionality</li><li>1 out of 7 (Adaptability) are related to ISO's Portability</li><li>Mobility seems to be a hybrid of ISO's Usability, Functionality, and Efficiency</li></ul>...interesting that in ISO terms, Apple is most heavily weighted on Usability...turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-27436654894268543172008-03-28T11:42:00.001-07:002008-03-28T11:42:49.448-07:00Quote of the day<img src="http://www.cs.wisc.edu/graphics/Gallery/Retarget/vitruvian_smaller.jpg" style="margin: 0pt auto 10px; display: block; text-align: center;" title="" alt="" />
"Never mistake motion for action."
-Ernest Hemingwayturboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-28923725625176528722008-03-27T16:55:00.001-07:002008-03-27T16:55:14.906-07:00The BS7925-2 Standard for Software Component Testing<img src="http://www.realityloop.net/blog/Thesamebutdiffernt_694/Front_3.jpg" style="margin: 0px auto 10px; display: block; text-align: center;" title="The Judean People's Front" alt="" height="200" />
This doc is now 11 years old, but has some really good basic ideas for proceeding through a release cycle. While on first glance the doc looks to be ginormous, it's not really that bad; the meat of it is really covered in the first 30%.
Sections of interest:
<ul><li>2.1.1.8 mentions the order in which test activities should be done--it includes Component Test Specification (sandwiched between Planning and Execution)</li><li>2.3 "Component test specification". In brief, tests are written here using the techniques that were determined doing Planning (read: tests weren't written during Planning, but the techniques were chosen then)</li><li>3 "Test Case Design Techniques". Concisely describes popular and useful techniques for creating test cases for a component.</li><li>4 "Test Measurement Techniques". These aren't criteria, but rather methods for helping to figure out progress--and maybe setting criteria based on this info. They show how to do this for each test case technique type in section 3 (most are pretty obvious, but it's still nice to see on paper).</li></ul>Grab some coffee and take a gander, <a href="http://www.ruleworks.co.uk/testguide/BS7925-2.htm">here</a>.
Oh ya, and in case you're familiar with this group, this doc was produced by the BCS SIGIST. (seriously, why? I guess the real question is why "British Computer Society Specialist Interest Group in Software Testing"? Reminds me of Monty Python's Life of Brian...)turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com2tag:blogger.com,1999:blog-9122451861811400048.post-82191277365018804512008-03-26T18:13:00.001-07:002008-03-26T18:14:49.117-07:00softwaretestingsucks.com<img src="http://www.softwaretestingsucks.com/vtesting.jpg" height="200" /><br>
I'm not exactly sure how this got started, but there are actually a few decent articles/pages of info in here on the basics of testing:
<ul><li>What is Quality?</li><li>Life Cycle testing</li><li>Testing Types</li><li>Testing Techniques</li><li>Testing Tools</li><li>Certification Programs</li><li>Testing Jokes</li></ul>Beware--in that last one, the jokes are uber-cheesey, and mostly only remotely entertaining to the Test Geek...
Check it out <a href="http://www.softwaretestingsucks.com/">here</a>.turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0tag:blogger.com,1999:blog-9122451861811400048.post-82561409749461007812008-03-19T13:32:00.001-07:002008-03-19T13:32:04.423-07:00Testing Removable MediaGuys on the UK's The Gadget Show cook with propane, freeze with dry ice, submerge in acidic liquids, drive over, and mortar-fire against a wall a number of types of removable media. Good stuff.
<object height="355" width="425"><param name="movie" value="http://www.youtube.com/v/NyOFIH-6WGs"><param name="wmode" value="transparent"><embed src="http://www.youtube.com/v/NyOFIH-6WGs" type="application/x-shockwave-flash" wmode="transparent" height="355" width="425"></object>turboladenhttp://www.blogger.com/profile/13370723531847481347noreply@blogger.com0