Testing Computer Software, 2nd Edition
Cem Kaner, Jack Falk, Hung Q. Nguyen
ISBN: 0-471-35846-0


[chapter 2] 

Overview :- 

Don't ever aim to fully test anything. It's impossible. 

The point is to get things to work. 

Understanding the error is an acceptable fix. 

Testers should expect to find errors, but not anticipate them. 

A great programmer is less likely than a incompetent tester. 

[chapter 3] 

During design :- 

Evaluate requirements during planning, analysis and design. 

Prototype to quickly & cheaply clarify requirements and reduce risk. 

Walkthrough. 

Validate against requirements. Completely(?) 

During code :- 

Start intuitive, informal plan during development. Work through below steps several times then build them in to a formal, stricter set to be systematically applied each time modifications are made. 

Testing should be :- 


Incremental 
Static (e.g. code inspection by peers) & Dynamic (e.g. execute code) 
Regressive 
Consider Bebugging & Mutations (seeding bugs to evaluate testers & test data). 


Glass box (structural) can/does/should happen naturally as part of the programming process. Training developers to do this is worthwhile. 
Black box (functional) 
Plan test cases early. 

Typical stages :- 

Acceptance 
Stability 
Functional & System 
Compare with specification, usability, state transition, error recovery 
[we need to build a list of these specific to the BBC & WWW sites] 
Beta 
Integrity / Release 
Certification 
During Maintenance :- 
Don't under estimate post-launch maintenance. Schedule maintenance phases in to the products life cycle. 

[chapter 4] 

The 'quality' of software is fundamentally measured in human terms. Therefore, in testing for bugs we are looking to determine the *degree* of usefulness of the system to the user. 

Do not measure 'quality' against the specification unless you are certain the specification is correct. 

Types of Error :- 

[we need to build a relevant list from our history] 

[chapter 5] 

Reporting a Bug :- 

What the people who can assess/fix it need to know. 


Unique ID 
Error type 
Summary 
How to reproduce (step by step) 
Attachments 
Contact details 
Once submitted, how it should be dealt with. 


Assigned to 
Comments 
Status 
Priority (for project lead to decide) 
Resolution 
Analysing a bug :- 

Look for the critical (origin) point of occurance. 

A bug is reproducable only if you can explain how it occured. 

[identifty common techniques to reproduce, analyse, identify bugs] 

[chapter 7] 

Designing Test Cases 

Use equivalence classes to group tests. Then identify actual tests cases from the center and the boundaries of these classifications. 

Interactive programs will have visible state transitions. 

Capitalise on your expertise / intuition and speculate on what might go wrong. 

[chapter 12] 

Test Plan Documents 

Decide if the test plan will form part of the product or a development tool. 

Detail the objectives of the test plan. It should cover all technicalities, improve understanding of tasks to be undertaken, provide a structure for management & scheduling. 

Allow your tests to evolve throughout the project, but ensure the test strategy matches the project methodology. 

Start with broad overview of specification material, then add depth [pg 215-216] 

You will make lists, tables, outlines, matrices. 

Lists :- 

Things to be reviewed, not necessarily in great detail. 


List of all screens, dialogue boxes, reports, data files. 
List any variables - input & output. 
List features + functions that are visible to users 
List error messages (tie in to TRACE template) 
List program files & versions (esp. if in CVS) 
List compatible software + hardware + o/s 
List project deliverables (systems, documents...) 
Tables :- 

Good to show relationships. 


Input + output variables [pg. 227] 
decision tables & trees (around conditional aspects) 
boundary/equivalence chart. 
Outlines :- 


functional list 
step through (start simple) 
mark things as optional 
be sure to invoke all functions 
menu maps 
Matrix :- 

For data collection. 

Documentation :- 

Everything you do should be understandable. 


Personal notes (reminders, questions...) 
Notes for team (howto, purpose) 
Notes for testers (overview) 
Test script (for inexperienced, step by step) [pg. 244 pro's & con's] 
Notes for managers (results) 
IEEE Standard 829 (Software Test Documentation) - [pg. 246 - 252] 

Focus on fixing bugs, not form filling. 

[chaper 13] 

End production is trade-off between - Reliability Vs. Feature Depth Vs. Cost Vs. Release Date. 

[pg 365. appendix of possible errors]