The Tao Of Testing
Posted on 12 Dec 2000
by Martin Burns (MartinB)
Rated 4.37 (Ratings: 15)
- More articles in Site Development
- Your future business depends entirely on your professional reputation - good clients will always look for a reputation for delivering their requirements. Anything which enhances that reputation is A Good Thing.
- Once the system is handed over to the client, you will then have an audit trail of testing, documenting that the system is working. If it later fails, you have a backup to safeguard you against potential legal and reputational action from a panicking client.
- If you want to feel self-interested about this (and most of us do at some point), remember that the client should pay for all of this testing - it's all chargeable time, which will result in the client getting a better system at the end of it.
- Use Test Scripts
- Perform Tests
- Report and Fix Errors
- Usability Testing
- Usability testing should happen before a single element of the user interface (including information architecture) is fixed. Performing usability tests at this stage will allow you to change the interface reasonably quickly and cheaply - backing out an interface once it is coded is always going to be difficult. The best way to perform usability testing at this stage is to build a prototype of your proposed interface, and test that. Feedback from the testers will allow you to quickly amend your prototype and go through another iteration. Research shows that target="_foo" title="Opens in a new window">you only need to use five testers to perform the usability tests, and find 85% of the usability issues in each iteration. After a few iterations, you're unlikely to have substantive issues left.
- Unit Testing
- Typically, a system contains a number of pieces such as
- 'the bit which displays the product'
- 'the bit which puts the product into the shopping cart'
- 'the bit which which verifies the credit card and takes the payment'
- System Testing
- Once you have all your units behaving as expected, you need to string them together into a system, and test it in a semi-real environment, which is only different from the way it will finally operate in that you're not working with real users and live data.
- Integration Testing
- As eBusinesses become more complicated, there is a growing need for the systems you produce to be integrated with other systems, like the financial reporting system, the logistics system, the customer database and so on. The purpose of integration testing is to ensure that your system's inputs from and outputs to the other systems are as expected. This means that you will need to ensure that test data fed between the systems is not going to be mistaken for live data. That said, at some point you will need to put a real transaction through your test system as an end-to-end test. A useful (and popular with developers) way of doing this is giving the team working on the site an allowance to spend on the site as 'friendly orders', in return for reporting back any customer-facing inconsistencies in the entire process.
- Volume Testing
- Far too often, an eBusiness is a victim of its own success. From target="_foo" title="Launches in a new window">the Slashdot effect to title="Cahoot offered 0% interest on the 1st 50,000 credit cards they issued. You couldn't access their site for a week after launch. Launches in a new window">sheer stupidity of the Marketing department, if your system won't handle the loads put on it by users, you are going to lose both face and money. Larger eRetailers are now building their systems to handle over a thousand simultaneous users. While you may not be in that league, you need to simulate the loads you anticipate, plus leaving enough headroom for traffic growth. Get it wrong, and you may be facing title="They launched their telesales in Summer 2000, but the web site didn't go live until December 2000, because their volume testing failed. Launches in a new window.">a launch delay of months.
- Regression Testing
- Unless you are spectacularly lucky, your testing will highlight errors in your system. And there's a better than average chance that fixing those errors will introduce new errors. Regression testing is a matter of going back over your previous tests to ensure that:
- The bug you previously found has been fixed
- No new bugs have been introduced.
- User Acceptance Testing (UAT)
- Once you have what appears to you to be a working system, which meets all the requirements, the final piece of work you must undertake before you can ask for your cheque is User Acceptance Testing. This is essentially stepping through all the functionality with the client staff who are actually going to use the system. If your system fails UAT, yet meets the paper requirements, then you have an issue with your requirements documentation. You will need to resolve this with the client - has there been scope changes since the requirements doc was signed off? - before you can justifiably ask the client to sign off all your work and pay you.
- An ID number
- Status (new, in progress or resolved)
- Red (ie causes non-functionality in the system. Must get fixed before go-live). I've also seen this subdivided into "Red" and "Mother of Red".
- Amber (ie causes interference to user tasks. Should get fixed before go-live).
- Green (ie causes annoyance to users. Will get fixed if there is time before go-live).
- Patch ID which will resolve (or has resolved) the error
- An owner - a named individual who will take responsibility for ensuring that the fix happens. This need not be the person who actually fixes it.
- A detailed description of the error, including any error messages, and screenshots where appropriate.