• Robert Walker

This "T" is for Testing

Updated: Jul 9


The Three T's of Marketing & Research

Small-to-medium sized businesses (SMBs) can use a simple reference model for their marketing and customer insights efforts. By focusing on what I call the "Three T’s" (targeting, testing, and tracking), your business operations will be continually guided and improved by staying focused on your core target, supported by continuous testing of new products and services, and by objectively monitoring your progress over time. Today let’s take a closer look at one of the these areas: testing.

Instill a Testing Mindset


Testing can involve a multitude of variables, but as an SMB you need to think about two fundamental dimensions when considering a test of any kind: tests that affect your brand, and tests to determine whether your ad spend is working.


My colleague Joel Rubinson makes this key distinction between what he calls performance vs. brand marketing. Each supports the other; they are not in opposition to one another. In simple terms, performance marketing is focused primarily on media allocation and the optimization of your ad spending (ROAS), while brand marketing is focused primarily on finding the best ways to communicate the fundamental premise of your brand, such as your brand's features, benefits, and desired end-user target.


It is in this context that I want you to think about two types of testing: what I will call brand concept testing of the brand promise (in various concept formats); and performance testing, the most commonly used form being A/B testing.


Brand Concept Testing


When we want to communicate the essence of a brand or an idea to a prospective customer, we do it with a stimulus known as a concept. A concept is an idea before it is marketed solely for testing purposes, so that we can understand consumer reactions to it. We test concepts to reduce the risk of making a mistake when launching an idea, to find the best way to describe an idea, and how to best communicate to our target audience.


A concept needs to communicate a compelling end-benefit or a solution to a problem using language that a target consumer can not only understand, but internalize and relate to emotionally. Concepts can differ significantly in their language, layout, image content, and other characteristics. The format of your concept will vary depending on the type of information you need for your brand and the type of test you are planning.

There are concepts written specifically for screening purposes that have very little detail or descriptive information (on purpose). There are concepts that go a little further, with more descriptive information, but still short of being fully developed. And then there are concepts that are close to finished advertising, much like you would see on a landing page or in print media. Here are three types of brand concepts that you should know:


Kernels are ideas or benefits presented as statements.

Kernels are used in screening tests to efficiently identify winners vs. losers. Kernels are evaluated on just a few measures (e.g., purchase intent, uniqueness, superiority), and each respondent sees all kernels, and each is assessed on all measures. Kernels can be attributes, benefits, or distinct ideas. This type of test is also called a "benefit screen".


If kernels are distinct ideas, the analysis focuses on the top performers and profiling them on demographics, geographical areas, or other methods such as attitudinal scoring. If the stimuli are end-benefits or positioning statements, we can use tools to identify underlying themes that might convey an even bigger thematic idea.


White card concepts are simply words on a page without high-quality images or fluffy language.

White card concepts are typically comprised of 4-8 sentences, factually stating the problem, usage situation, or need; they also state the end-benefit, solution, or final result provided. White card concepts can be existing products, stand-alone ideas, line extensions, or new uses and repositionings. They can include price, flavors, sizes, dosing, brand name, packaging information, and even a basic visual (i.e., a B&W drawing). Because the goal is to test the waters in a bit more detail, some diagnostic questions are included – but the number of questions is limited because we are typically evaluating multiple concept ideas.


Full concepts are used to capture more complete reactions, and when fine-tuning your messaging or language is essential prior to a launch or ad spending.

Full concepts often have the benefit of qualitative insights to develop language, positioning, tone, or emotion of an ideas that showed promise in previous screening work. Full concept testing can be done "monadically" (i.e., the respondent sees one idea at a time in its own cell of respondents), or in a sequential design.


Full concepts are longer, written to include all that might be conveyed in the time an ad is exposed (e.g., 15- or 30-seconds). They can also be more elaborate in their situational set-up or premise, use of demonstration cases, or other info.

Performance Testing


A/B Testing: you might think that A/B testing always provides a clear choice, but there is usually more to the story than the difference between two variables.

The A vs. B variants you are testing might be affected by a series of previous decisions made long before either A or B were evaluated head-to-head. For example, if A is your current campaign that includes search, PR, and Facebook ads, and B does not leverage the campaign you are now running, your test is already biased against B. Or, perhaps your objective was impressions, but one option delivered much higher conversions. So, interpreting results can quickly become more complicated than perhaps they first appeared.

But, for argument sake, let's assume that A and B start from the same point, and neither will be biased by previous advertising or spending level decisions. If so, A/B testing can be interpreted unbiasedly, and executed within any number of environments, such as CRM systems (HubSpot, Salesforce, etc.), dedicated A/B testing environments such as Central Control or Unbounce, and even some popular web hosting platforms provide an opportunity to conduct simple A/B tests.


Mechanically, the ad testing company you work with will develop two (or more) landing pages (A, B, C, etc.) and visitors to your site will be randomly redirected to one of those variants. Google Analytics and other web traffic statistics can be utilized to determine which variant is most effective at, for example: lowering bounce rates, achieving conversions, increasing CTRs, or other metrics you choose. A/B test designs can also revolve around content of the landing page, the overall site experience, or changes to ad spend, placement, location, context/environment, and more (see above brand concept formats).


Scratching the Surface


There are a multitude of different testing and design options for you to consider as an SMB. I have given you a taste, so get out there and test! Working with a marketing insights and research expert is your best guarantee that the type of concept and testing environment is designed, executed, and analyzed effectively. At Surveys & Forecasts, LLC we have worked with many different companies to help them develop optimal brand strategies and concepts, identify which execution best communicates their brand's proposition, and which marketing program is most effective for their limited ad dollars.


© 2016-2020 by Surveys & Forecasts, LLC