Apple's Characteristics of Great Software

A friend & coworker of mine sent me a link to Apple's Human Interface Guidelines, with the intent to give some insight in to testing the UI applications that I test for my job (thanks Mike). After stepping through a few pages in Part I, it clicked for me that a number of the "Characteristics of Great Software" that Apple lists are the same as some of ISO 9126's quality characteristics. After attending a CVBI meeting this morning where the topic of conversation was "building quality in to software products", I'm really encouraged to see Apple not only list these characteristics that it thinks are important, but back up why they think they're important. One of the things that was discussed implicitly in this morning's meeting was that in order to end up with better quality products, developers (and testers) need to be taught at some point what quality is and how to get there. Apple uses this doc to clearly communicate some ways to raise the quality, not just on Apple applications, but on all applications. Seeing this crossover between the quality characteristics that ISO setup and the ones that Apple setup really gives extra weight to these characteristics. Without having read too much in to the Apple doc yet, it looks like for the characteristics that Apple lists:

• 2 out of 7 (Ease of Use, Attractive Appearance) are related to ISO's Usability
• 1 out of 7 (Reliability) are related to ISO's Reliability
• 1 out of 7 (High Performance) are related to ISO's Efficiency
• 1 out of 7 (Interoperability) are related to ISO's Functionality
• 1 out of 7 (Adaptability) are related to ISO's Portability
• Mobility seems to be a hybrid of ISO's Usability, Functionality, and Efficiency

...interesting that in ISO terms, Apple is most heavily weighted on Usability...

The BS7925-2 Standard for Software Component Testing

This doc is now 11 years old, but has some really good basic ideas for proceeding through a release cycle.  While on first glance the doc looks to be ginormous, it's not really that bad; the meat of it is really covered in the first 30%. Sections of interest:

• 2.1.1.8 mentions the order in which test activities should be done--it includes Component Test Specification (sandwiched between Planning and Execution)
• 2.3 "Component test specification".  In brief, tests are written here using the techniques that were determined doing Planning (read: tests weren't written during Planning, but the techniques were chosen then)
• 3 "Test Case Design Techniques".  Concisely describes popular and useful techniques for creating test cases for a component.
• 4 "Test Measurement Techniques".  These aren't criteria, but rather methods for helping to figure out progress--and maybe setting criteria based on this info.  They show how to do this for each test case technique type in section 3 (most are pretty obvious, but it's still nice to see on paper).

Grab some coffee and take a gander, here.  Oh ya, and in case you're familiar with this group, this doc was produced by the BCS SIGIST.  (seriously, why?  I guess the real question is why "British Computer Society Specialist Interest Group in Software Testing"?  Reminds me of Monty Python's Life of Brian...)

FIFA.com - Football - Test Criteria

It's always refreshing to switch contexts for a subject you're immersed into.  I ran across FIFA's criteria for testing their balls, which is just good to take a look at.  Here's an excerpt from the write-up:

FIFA Quality Concept

The FIFA Quality Concept for Footballs is a test programme for Outdoor, Futsal and Beach Soccer footballs. Manufacturers have the possibility to enter into a licensing agreement for the use of the prestigious 'FIFA APPROVED' and 'FIFA INSPECTED' Quality Marks on footballs which have passed a rigorous testing procedure. As an alternative there is the possibility to use the wording 'IMS International Matchball Standard'. Footballs bearing this designation have passed the same quality requirements as 'FIFA INSPECTED' footballs. The use of this designation is however not subject to a licence fee and any association with FIFA is prohibited.

There are two levels of criteria for the three designations. Footballs applying for the category 'FIFA INSPECTED' or the technically equivalent 'IMS - International Matchball Standard' must pass the following six rigorous laboratory tests:

1. Weight
2. Circumference
3. Sphericity
4. Loss of Air Pressure
5. Water Absorption(replaced with,Balance' test for testing of Futsal balls )
6. Rebound Footballs applying for the higher 'FIFA APPROVED' mark must pass the six tests at an even more demanding level and must undergo an additional test:
7. Shape and Size Retention (Shooting Test)

FIFA.com - Football - Test Criteria

Truth and quality I

The Free On-line Dictionary of Computing defines quality as:

The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.  Not to be mistaken for "degree of excellence" or "fitness for use" which meet only part of the definition.

Put more simply, quality can be defined as how well something measures up to a standard. But what if the standard sucks? What if the quality of the standard is low? Example: the standard for an app says it only has to run 1 out of 5 times when you double-click its icon. If you test for that and the app meets the standard, you can say it's of good quality, right? ...right? It reminds me of testing apps whose functionality was written without specifications.  When I test the app and find something that doesn't make sense or "doesn't work", Development can say that it's not a bug--it's working as intended.  ...which is totally relative, but totally true.  It's just the intention that was flawed.  ...and the only way to coerce a change is to make some great and sneaky argument, or bring in some sort of exterior, already-defined standard that all of the sudden makes the given functionality look like crap. In the example above, the developer had set his own standard, to which I thought wouldn't be up to standards of the end user.  In the example of the 1/5 standard above, if you're like me, your brain makes a judgment call on the standard itself--probably without you realizing it.  In essence and in context to this post, you're impelled to hold the standard to some implied standard--a standard that's sort of like a code of ethics that pervades a culture.  You know that cutting in line at the Post Office is just a no-no, not because there are any signs there that say so--you just know.  Same idea.  In the case of the developer working without a spec, my code of ethics was just different than his. So in order to have Dev and QA teams be efficient, a general practice is to have both departments agree on what's acceptable and what's not. They define the standard as they see fit for their organization and customers. But the trick is: how do we know when the standards that we've set are good standards?  Where does the standard's standard come from? How do you know right off the bat that the requirement that the app only has to run 1/5th of the time really sucks? Some correlation can be found when looking at the study of Truth.  People have been studying concepts here for a couple thousand years, as opposed to the drop in the bucket we've spent on studying software.  And there are probably just as many theories on SW development and testing practices as there are on Truth.  So without going too in-depth to the topic, I believe there's some interesting discoveries to be made when considering the various theories of Truth.

ISO 9001 vs CMM

An basic high-level comparison between the two industry quality standards: What do ISO 9001 and CMM mean to your organization? (courtesy techrepublic.com)

ISO 9126 and Reliability

I've reading through the ISO 9126 Standard doc (in case you don't have ISO numbers memorized, it's Software engineering - Product quality) and amongst other interesting things thus far, I encountered a section titled "Quality model for external an internal quality" which outlines areas of an application that should be tested; test types, in essence. Of those test types, one was Reliability.  It defined Reliability as:

The capability of the software product to maintain a specified level of performance when used under specified conditions.

...but what really grabbed me was the first note under that definition:

NOTE 1  Wear or ageing does not occur in software.  Limitations in reliability are due to faults in requirements, design, and implementation.  Failures due to these faults depend on the way the software product is used and the program options selected rather than on elapsed time.

This blatantly points the finger to requirements, design, and implementation for when dealing with reliability. I think to some, this might seem to be a "duh" statement, but that whole note is quite enabling as a software test engineer.