Testing Your Site, pt.1

How many testers is enough when it comes time to comb your site prior to launch? Four? Six? Eleven? As many as you can get?

This depends upon whether you have already conducted usability testing. With the economy what it is, it’s probable you didn’t. And while that’s unfortunate, it’s also reality. So, you will want to fold some usability testing into your QA. Because even pseudo-usability testing is better than none.

With that in mind, it’s always better to err on the side of more testers… especially if they are volunteering their time and budget is not an issue.

Five testers is okay, though I prefer six as a minimum, seven is better, and eight better still Why? Well, not so much to catch mistakes (though there is that — no one catches everything, so the more testers there are, the better the chance of everything getting caught.) But it’s also that you need to make sure you are covering all the major platforms and browsers and versions, and five just might not be enough.

You also want more testers because inevitably there are testers who are just lame. Lovely people, but lame testers. I remember when we were testing Julia Quinn‘s site and Eloisa James, the queen goddess of the English language, submitted four solid pages of fixes (including catching an i-before-e issue on a header graphic that I am embarrassed to say none of us saw in production.)

Another tester said, merely: “Looks great! Love the color!” Again, lovely person, brilliant author, lame tester.

With only five testers, you don’t really have room for anyone to be lame. Or if you do, then the other four better inspect like drill sergeants. You see my point?

So keep hunting. If you can find another couple of testers, it really is in your best interest. Believe me, I understand the issues of trying to conscript people for this. But a big reason we like to have many testers is usability-related. Invariably there are one or two issues with the site that made perfect sense to us, but which confound over half your testers. Recognizing and calling out jarring elements is a primary job of your testers. Maybe you can dismiss the comment when when only one or two people are saying “What the H?”, but when it’s four or five people getting tripped up, then we know we need to change that element. If you only have five testers you are hoping that everyone does their job and calls out the awkward element that tripped them up. But no one bats 1000. Ever.

So choose many, and choose well. You want people who would tell you that you have spinach in your teeth or toilet paper stuck to your shoe. This is no place for “Well, I didn’t want to insult you.” Instill in your testers the value of their letting you know everything, especially if the element looks confusing. Don’t let them assume it’s their problem. If you don’t get it, call it out.

Oh, and your significant other can’t test. Neither can your mom. They are too close. More on this and other testing issues in a later post.

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·

5 thoughts on “Testing Your Site, pt.1

  1. A topic after my own heart. Better to put out a solid, vanilla product than a Ben & Jerry’s with pebbles instead of choc chip; boring before brilliance; quality before quantity, etc. etc. Perhaps it’s our judgmental human nature, but it’s easier to find faults in other people’s work than in our own. :)

  2. Thanks for the input, Keira, but we don’t really want to put out Vanilla. We well know what works in Vanilla (and actually, good, solid, known vanilla is a great starting place).

    We want to stretch for the unusual and unexpected. We want to include a lot of content without letting the visitor feel lost. And in doing this we will, from time to time, inadvertently not be smooth. This is where we want testers to call out the jarring effect. “This was weird to me.” or “This confused me.”

    In a perfect world, we get this during usability testing just by watching where the tester pulls a face or can’t figure out where to go. But in lean times usability is often chopped from budgets (unfortunately). And so we depend upon QA testers to call things out.

    But you are 100% right in that it is easier to find fault in other people’s work. This is why you can never test your own site, and your mom can’t test either.

    As ever, thanks for the input!

  3. I suppose I’m somewhat crazy, but I love testing sites. I enjoy seeing what new sites look like and what new bells and whistles are introduced. For the privilege of those early peeks at new sites, I try to be a thorough tester. One thing I always try to do is compare IE and Firefox. Sometimes the differences in display are perplexing, but the Waxies generally know how to magically synch them up. Now I have a Mac in the house, too (not mine … I’m still a PC person) so I can test on the Mac as well.

    I dream of the day when all browsers display pages identically! Ha.

  4. The one time I tested for you was a joy! I hope I gave you enough information to be fall in the lame category.

    One thing I love now, when testing sites, is I have an IE tester set. I can open a site in IE versions as old as 3.

    I’ve ran it on my own designs several times and caught numerous mistakes. The one that normally catches all the mistakes is IE 6.

    Like Candice, I dream of the day when they can all agree on what a coding string means. That or everyone would upgrade their pcs to the current versions.

Comments are closed.