As our clients have been planning their Spring schedules, we’ve had lots of discussions about testing. Whether a test is useful or useless depends largely on how it’s set up, so a review of best testing practices seems timely.
Isolate Variables and Use a Control. It’s amazingly common to change virtually every variable and call it a test, when in fact this approach is more of a crap shoot. To get useful information, you need to change one variable at a time and test it against a control. Knowing the percentage increase or decrease attributed to an isolated variable allows you to make intelligent decisions about future advertising.
Track Responses. I hate wasting money, and that’s exactly what happens if you don’t know which advertising produces what results. Some clients are sticklers about capturing key codes, and those clients make it easy for us to help them. Those that don’t capture source codes end up missing opportunities and/or spending money on advertising that may not be working, simply because we don’t have the data to know better.
Make Sure Your Results are Valid. Once test results start coming in, don’t make a hasty decision. Make sure you have enough response that results are statistically valid; usually a minimum of 100 responses per test cell is called for. And never ramp up your advertising without reconfirming your results with a somewhat larger test.
