One of the best things about direct marketing is that it eliminates the need for guesswork. Develop a theory about what will improve your marketing results, test it correctly, and you can move forward confidently when hard data supports your theory.
The problem is, many “tests” are poorly executed and can lead to erroneous conclusions. Here are a few tips to help you ensure you’re reaching valid conclusions:
• Test Against a Control. Trying something completely new without testing it against a proven control is not a test. All too often, marketers throw out the old because they’re tired of looking at it, and assume their customers are too. Don’t make that mistake. Always test the new against what’s worked in the past. Just because something is “same old, some old” to you, it doesn’t mean it’s stopped working.
• Isolate Variables. It’s surprisingly common for marketers to reach conclusions based on unscientific data. For instance, once company recently reported that a digest sized catalog far outperformed a full-size catalog. On further investigation, it turned out that they not only changed the catalog size and amount of selling space, but they had mailed the full size catalog in January, and the digest size in February, and that the list mailed in the two months were not comparable. Unless you test one variable at a time, you can’t know for certain what’s driving a difference in results.
• Get Statistically Valid Results. The more responses you have, the more confident you can be that you’ll get similar results the next time. For a reasonable degree of statistical significance, aim for 100 responses from any given test segment.
• Reconfirm Results. Direct marketers can tell you that quirky things can happen, and often do. Once you have test results, it’s prudent to reconfirm them with a somewhat larger test. Don’t blow out your budget on a major new change until further tests have confirmed their validity.
Keep these few principles in mind, and you’ll avoid the most common mistakes in testing.
