Modern medicine is evidence-based. The adoption of evidence-based medicine saw our lifespans double in the space of 50 years. That’s a remarkable achievement.

Sadly, evidence is missing in modern marketing practices. We rarely see any marketing discussion mention anything about significance, confounding variables, or correlation versus causation. Most blog and social media recommendations are anecdotal evidence masquerading as statistics. Overall, there is a disturbing lack of anything that one could fairly call evidence in recommending any particular marketing practice.

This becomes apparent when you start asking some simple questions about “statistically” justified marketing recommendations.

1. Are these numbers for B2B or B2C?

2. What is the impact of brand and market Leadership on these numbers?

3. What is the impact of fashionable trends on these numbers?

4. What is the sample size?

5. What is the breakdown by industry? Company size?

At this point, you are probably not getting your emails answered or your phone calls returned.

Some of these questions and related concepts have obvious ramifications- brand leaders are usually older, more mature, have larger budgets, and are more likely to optimize existing processes. So…any numbers hinting at a best practice and revenue growth needs to control for brand and market leadership. And any recommendation needs to be prefaced by “it depends”.

Generally, any statement like ’65% of best-in-class companies use X and grew an average of 20% more year over year then non-best-in-class companies’ is meaningless, or should be treated as meaningless without access to the support data and methods used. Without decent evidence and analysis there is absolutely no reason to make ANY kind of inference here between the use of X and growth. And even given that data and where there is a strong causal effect, there is the important question of whether the principle is applicable to other industries, markets, or business models.

In medicine, there is a very cool organization called The Cochrane Collaboration which aims to improve health care by providing systemic reviews of the quality of evidence being used to propose and select treatments- i.e. which studies are good, and which suck. Even in esteemed scientific circles peer-reviewed journals aren’t enough, and The Cochrane Collaboration aims to raise the bar by provide systemic evaluation of the quality of studies and the quality of how they are published. Is the data available? Can the study be repeated? These are other markers of quality are emphasized and graded.

Evidence-based medicine had doubled our lifespans. Would evidence-based marketing double our revenues? Is it even possible with so many confounding variables at play in the business world?

Test-Driven Marketing has been published in the October 2010 Pragmatic Marketing newsletter!

Pragmatic Marketing October 2010 Newsletter- Is Sales Becoming Marketing Tech Support?

Have a look and let me know what you think.

If you like this article, consider reading a few of our other marketing strategy posts:

When Being Customer Driven is Dangerous

The Dark Matter of Selling: The Missing 90% of the Sales Process

Please Stop Generic Enterprise Marketing!

and my personal favourite

Can Subject Matter Experts Destroy Your Company?

And please say hi-  leave some comments and if you liked what you’ve read subscribe to the Test-Driven Marketing RSS feed and follow @weerdlogic on Twitter.

Oh, and there is also a new Test-Driven Marketing LinkedIn
Group

Lawyers can teach a simple rule to product managers and marketers that will serve them well. It’s an evil sounding rule- one you might not believe at first. But bear with me.

Never Use One Word When Two Will Do.

Well, not all the time, but getting to specific meaning is very hard in most business environments.

Words are fluid and have multiple meanings. Said a fancier way, words are connotational, not denotational. That is, their meaning is tied to other meanings and context in very subtle and not-so subtle ways. Context matters, and when you capture requirements, you are removing those ideas from their living, operational context. There is no human mind in a job function, interacting with other humans, with lots of surrounding context.

For instance, in any company that does product development the meaning of the word product changes per department (merchandising, design, development, testing, sourcing, manufacturing, support) or per lifecycle stage (ideation, development, sourcing, testing, commercialization, etc) or combination of both.

Or even better, when making changes when is that product no longer that product? Some departments use form/fit/function- if it’s the same, it’s the same product. But if a purchased component would it it still be the same product if from different vendors?

Often, from a design perspective, yes. From a manufacturing and quality perspective, no.

All these departments use the word “product”, and never suffer from ambiguity or confusion in their day to day operations. Context is king, and the human mind is marvellous.

But all hell breaks lose once that context is gone, and requirements are put to paper. I’ve seen arguments rage for hours, and then repeat every few weeks, all due to ambiguity arising from this brave new paper reality.

Not convinced? Let’s look at an example using concise, well written English:

“There shall only be one product per product number. Product numbers shall not be reused”.

System is built, rolled out to designers. Or if lucky, they are brought into a requirements meeting before building the system.

“What? Are you nuts. We do 70 prototypes per final product and you are saying we can’t link them all to a final intended product number? And that we can’t change the product number to the final number?”

(BTW- this is one area where Agile methods start falling down- when slight increases in complexity tend to completely make over your domain model)

Okay. We see the problems. Let’s bring in our lawyer-mind, and revise the requirements using our new almost-evil rule:

“There shall only be one Manufactured Product per Manufacturing Product Number. Manufacturing Product Numbers shall not be reused”.

“Designers shall generate a unique Design Product Number per Design Product. There can be any number of Design Products per Manufactured Product, but only one released design. Once the Manufacturing Product is released, the design product can’t be changed without a change management process.

The new requirements make each domain object a compound term, and add time context such as “released” and control contexts such as when change control is required.

Now the problem with writing like this is that it can get carried away, and become unclear by it’s very complexity. This is where you need to match your approach to the problem you are trying to solve. Lawyers need to solve for provability in court, to the point of moving way past clarity into a unique language all it’s own. Similarly, requirements analysts need to ensure clarity, communication and the ability to be translated into specifications and features. If there is no effective communication, then the requirements can’t be validated.

The other issue to watch out for is the separation of requirements from design. Just because you call something “Manufactured Product” in requirements analysis DOESN’T mean you call it that in the software. Unlike written requirements, software introduces context back into the equation. Manufacturing is using the manufacturing tools, or only seeing the manufactured products. You can design the software to say “Product”. You just can’t analyze it that way.

PREVIOUS POST: Should You Trust a Blogless Marketer

ANOTHER POST: Do You Need Obsession-Driven Marketing?

MOST POPULAR POST (LAST 60 DAYS): Why Marketing is Becoming Like Software Development

2ND MOST POPULAR POST (LAST 60 DAYS): Is Sales Becoming Marketing Technical Support?

The Dawn of the Hyper-Literate Society

In Why Marketing is Becoming Like Software Development we discussed how marketing was becoming Test-driven and Agile, driven by the demands of the Internet business environment.

The basic reason for this was that the Internet has created hyper-literacy in buyers, radically changing the sales process in ways companies have been very slow to respond to.

Much like how the printed word extended the reach of general information to the masses, the Internet has extended the reach of specialized information to the masses. Literacy has been replaced by hyper-literacy, and this has radically influenced many of our institutions and behaviours. Patients know almost as much about their illnesses as their doctors, dieters are reading primary research on nutrition and arguing about causation versus correlation on diet websites. And buyers in B2B marketplaces are driving sales people crazy.

This raises the question- how much are sales and marketing changing? Will sales still matter?

New Channels of Communication Change The Game

Sales used to be the primary conduit for the customer to learn about a product or solution. There were essentially no other channels. A few brochures and white papers would be produced, but that was only in support of the sales interaction- that collateral wasn’t meant to sell, and never fit that closely to the customer’s specific reality. And this was the necessity based on reality- to do it any other way would have required the sales team dropping a file cabinet off at a customers site.

As a result, marketing wasn’t that important. Couldn’t be that important. Marketing worked on lead generation and trade shows because that was the best they could do through the limited customer communication channels available to them. Brochures were background value-add to the sales process, but often unread due to their being so generic.

That has changed, and companies are just starting to get it. The Internet has opened up many other channels for the customer to get information, and provides an encyclopedia of knowledge that allows him to find it.

When you hear about about “buying cycles” replacing “sales cycles” in sales strategy, this isn’t just an improved understanding of purchasing psychology. It’s something that has been enabled by a fundamental change in the environment.

Customers buy on their own terms because they can. And as a result, marketing has become far more important then it was 10 years ago, and the role of sales has changed – or should have changed- dramatically.

Marketing Has Become a Larger Part of Selling

The result of the Internet channel dominating the sales conversation (especially when we, the vendor aren’t present) means that Marketing has to become a much larger part of sales.

To do so, Marketing must do two essential things- support the Internet customer communication channel, and measure that channel through analytics.

Marketing Needs to Generate Content That Sells Rather then Supports Selling

Why? Again because the customer is consuming this information on his own via the Internet. If you aren’t providing it, someone else will.

This content also needs to be more specific and relevant to the buyer. There is no sales team present to interpret and communicate a deeper message- this is your chance to get your message through. Traditionally marketing content was marginally relevant due to no delivery channel to the customer (reference the awkward file cabinet point above) and a resulting generic message. That channel is now there, and better yet, it allows content to be filtered for the customer. It may be a huge virtual filing cabinet, but search engines and other technology allow the customer to quickly find what appeals to him, leaving him feeling like he’s reading a nice little on-demand magazine.

The Content Must Fit the Customer Need, No Interpreter Needed

Another challenge is that your sales team isn’t there to interpret the content or get feedback from the customer regarding the content. So, the content must be a near exact fit for the customer, necessitating a much different type of marketing collateral then has been previously the norm. This is the most significant challenge of the online marketing environment, and brings us to our next topic.

Marketing Needs to Do Lead Nurturing

Lead nurturing use to be clearly a sales function. It requires a soft touch, an understanding of the customer, and solution selling to present the information in a meaningful and manageable way.

This fails today for two reasons, one traditional and one purely a result of the modern marketing reality.

The Economics of a Sale Are Becoming More Challenging

A 30% cost of sale can’t be supported by most companies. It’s as simple as that. You can’t spend enough time with the customer to help with his education, and that means he’s on his own more often.

The Customer is On The Internet While You Are Busy Doing Other Things

The customer isn’t paused until the next sales meeting, waiting to be feted and wowed by your solution selling greatness. That’s now a small channel to his mind. He’s on the Internet, researching, browsing your website, browsing your competitors websites.

10 years ago the Internet was 4 years old from a business perspective and there wasn’t much content. The Internet was a small channel to the customer, smaller then the sales engagement.

5 years ago the Internet was 9 years old. There was some content, but it was still classic corporate brochureware. However, blogs were starting to take off, and more print industry articles and insights were available on the web. An expectation of useful information started to blossom in customers, and almost all of our customers were now active Internet users.

Today, the internet is 14 from a business perspective. It’s got attitude, and it thinks it knows everything. It doesn’t but it’s getting really close- at least outside of B2B marketing.

The expectation of content from people who use the Internet is now extremely high. It’s the first stop for information.

Marketing Needs to Do Analytics

The other side effect of the customer pursuing information on the Internet is that you aren’t there to interpret the customer reaction. Broader, deeper content is being put out to the Internet so the customer finds his knowledge from you, but what is his reaction?

That’s why analytics are essential. Without it, you are flying blind.

Online content allows you to measure your audience’s response and gain critical insight into customer outlook and behaviour. Five years ago this may not have been so certain- poorer content and a uncertainty if your buyers were represented in your website traffic made measurement hard.

Marketing Needs to Go Where The Customers Are, And Where They Are Listening

The Internet being the dominant channel of communication to customers clearly explains why there is now an exploding market of marketing automation vendors. Your customers are online, doing research, and becoming experts- your sales channel can’t compete. You can talk to them for an hour a week, but the Internet is there for them 7×24. They want to find answers, and they aren’t waiting for you.

This changes the value and purpose of content radically.

Is Sales Becoming Marketing Technical Support?

10 years ago content had to be presented face-to-face to customers because there were no other viable channels of communication.

Now, most content needs to be presented indirectly through the Internet because it is the most prevalent channel of communication. This also means that content (and marketing) end up doing more of the selling.

What does this mean for sales? Are they destined to become technical support for automated marketing efforts?

Not likely. Some radically new very early markets may be beyond effective marketing reach. And in all non-trivial markets relationships are still central to any sales or buying process, . In fact, properly managed, sales can focus on more value-add activities in marketing-sold markets, handling more accounts and driving more business. In most cases, the early soft touches guiding prospecting and lead nurturing need human contact. The flow of marketing content to the prospect needs guidance and filtering based on human contact. Facilitation, problem discovery, solution mapping- sales is becoming a lot more about consulting and project management. But the fact remains- that beast called the Internet is sitting there, humming away 7×24, always ready to hop in as a sales advisor to your hyper-literate prospect whether you participate or not. There is no stopping it, and this change in marketing is becoming a necessity, not a choice.

Software development is influencing marketing a lot these days with the emergence of Agile Marketing and Test-Driven Marketing- Agile and Test-Driven concepts being two concepts that have dominated software development for the last 5 years.

The reason is not a fad crossing over, but rather a more fascinating and powerful phenomena. As the Internet has become pervasive in our lives Marketing has become testable, much like software, as well as more impactful due to the amount of reading buyers now do during the buying cycle.

Agile software development embraces change and measurement, allowing responses to “market” signals that are leading indicators of failure. Signals include early customer feedback, early robustness feedback from automated tests, and ongoing feedback from regression testing, which captures failures due to the landscape changing.

But so what, you might ask? Marketing has become testable. Whoop-de-do. I’ve been marketing for twenty years, and we’ve done just fine without testing.

My answer? Marketing mostly sucked from a customer perspective. And this was effective and professional because there wasn’t much other option. Marketing wasn’t sales, and sales was the group responsible for strongly targeting messages to buyers. Marketing was background support. Marketing didn’t have access to the customer mind, by message or by volume of content (diversity in customers makes available volume of content critical for targeted messaging), and as a result marketing was largely product focused.

That’s no longer the case. Where sales was once needed to deliver targeted content, buyers now look for the content themselves online. And if they don’t find yours, they’ll find someone else’s. Last generation marketing that was background to the targeted sales message is now leading in the foreground, and landing with a awkward thunk.

In the new marketing environment, we need marketing to work well or we’ll fail. The Internet has changed how we sell, and marketing does a lot more of the selling because that’s where the customers are- at their computers researching their own needs. Selling has become buying facilitation, and if marketing doesn’t step up and do more of this type of selling with relevant targeted messaging, failure is almost unavoidable.

The other massive influence in the new marketing environment is that we can now measure if marketing is working. Considering that we now need marketing to work, that is very good thing.

Inbound marketing, Internet lead generation and Pay-Per-Click advertising (PPC) all test marketing through conversion rates on multiple website goals such as newsletter signup, RSS subscription, Facebook fans, white-paper downloads, requests for a demo, successful multiple touches, purchases, and so on. There are literally hundreds of amazing measurements and tests that can be applied- either on existing data or on data created from intentional tests.

This is enabling marketing to be test-driven, and in complex markets, it’s providing the tools for marketing to become agile. This is exactly what happened in software- the focus on feedback from customers and tests enabled software development to become agile. Indeed, the concept of Agile Software Development was somewhat meaningless until the feedback mechanisms were there.

Marketing has become like software. You run it, test it, and it either works or it doesn’t.

The bottom line is that today we must sell with marketing, and we can now measure our marketing. These two incredibly strong influences have combined in a way that will fundamentally change the entire business world.

And that means we can start applying some of the same processes- agile responses to market signals, testing for user adoption, testing for relevance, and actually building our marketing collateral so that it is in fact testable.

If you think marketing automation and automated lead nurturing are a fad, you better have lots of brand equity to ride on. Because if you don’t, you are not going to be able to build it in this brave new world.

For more on this topic, I recommend checking out some of the new wave of marketing automation vendors- Marketo, Manitcore, Eloqua , MarketBright or Hubspot, as well as the Analytics vendors such as Google Analytics, Yahoo Analytics, and Omniture and see what they are saying.

And if you are convinced and want more details, or unconvinced but open to the possibility, subscribe to this blog via the Test-Driven Marketing RSS Feed or subscribe with the simple email subscription form over there on the right sidebar. Our next post, Is Sales Becoming Marketing Technical Support?, goes deeper into the changing roles of Sales and Marketing.

There was a post yesterday by Seth Godin on testing and how B2B marketers often avoid the early testing work that can save so much money later:

“Business to business marketing is almost always better if you treat it like direct marketing. ”

and

“Get it right for ten people before you rush around scaling up to a thousand. It’s far less romantic than spending money at the start, but it’s the reliable, proven way to get to scale if you care enough to do the work.”

You can read the full post here- Getting to scale: direct marketing vs. mass market thinking.

Testing is a common theme today- analytics, metrics and data-driven activities are exploding in popularity- but adoption is  slow. Marketers simply don’t take the time to test.

Data-Driven Marketing, a new book on data-driven marketing by Mark Jeffery, spends a lot of time on the challenge of getting to a data-driven marketing process, with the first two chapters being about why it’s not being done and how to overcome that:

Chapter 1: The Marketing Divide- Why 80 Percent of Companies Don’t Make Data-Driven Marketing Decisions- And Those Who Do Are the Leaders

Chapter 2: Where Do You Start? Overcoming the Five Obstacles to Data-Driven Marketing

In chapter 2, when discussing starting with small experiments to test campaigns before rolling out, Jeffery says:

“Although the majority of marketers are aware of this approach, my research shows that the vast majority of marketing organizations, almost 70 percent, do not use experiments to pilot test marketing campaigns relative to a control group. Why? The answer is that most marketing organizations’ reward systems are based on activities, not results”

This is what Pragmatic Marketing calls “list based marketing” and actively tries to change with their excellent Effective Product Marketing seminar.

In Chapter 2 Data-Driven Marketing lists 5 specific obstacles:

1. Getting started

2. Causality

3. Lack of Data

4. Resources and Tools

5. People and Change

Jeffery’s core recommendation? Start small and use wins to driven recognition of value and garner more focus and investment in data-driven activities. Also, find and partner with like-minded people to gain more influence. He also makes the important point that planning for testing and measurability early on allows you to structure your campaigns for those goals without a significant increase in cost.

Avinash Kaushik, Googles Analytics Evangelist,  identifies similar issues in the world of web analytics:

“You’ll find that data or tool are not your problem, it really is your company culture (both in terms of using data or getting your site tech teams to do things to give you data)” from How to Choose a Web Analytics Tool: A Radical Alternative

And Avinash has a good post on Seven Steps to Creating a Data Driven Decision Making Culture. where he talks about making the data easily relevant to the needs of stakeholders and showing specific wins in those areas.

Taking the direct marketing practice of testing to other areas is compelling. David Ogilvy, in Ogilvy on Advertising, published 1985, spends a lot of time talking about testing and what advertising can learn from direct marketing. Reading this book suggests that much of the success of Ogilvy’s advertising business flowed from a strong product marketing mindset (knowledge of the product and customer) and a strong testing ethos- things that weren’t traditional in advertising at the time.

In short, everyone agrees they need to be more data-driven, barriers to getting there are largely cultural, and small wins are the way to shift the culture and gather the focus required to scale the resources and infrastructure needed to effectively get there.

And if you are having trouble getting momentum even towards getting analytics support even towards a small win? Maybe you can’t get access to that data?

Consider using external analytical infrastructure- tools like Hubspot’s website grader, the Compete website for competitive information (Keep Your Enemies Closer), and Alexa for website traffic information. Also, Google is more powerful the you think. Use Advanced Google search methods to analyze your site for inbound links (link:<yoursite.com>) and help you search your and competitors sites (site:<site.com>) to look for trends and statistics.

Malcolm Gladwell on spaghetti sauce | Video on TED.com http://bit.ly/9K6Amd

A great video on how testing uncovered hidden diversity that 20 to 30 years of asking questions didn’t and led to the creation of extra-chunky spaghetti sauce.

Another interesting example from the video- creating one coffee roast for a group would test an average of 60/100. Segmenting the group into 4 and creating their ideal coffee would test an average of 75 or 78/100.

The practical result?

According to Gladwell, “the difference between coffee that makes you wince, and coffee that makes you deliriously happy”. The same process taking into account diversity led to a product that was fantastic compared to a product that was bland and distasteful.

Bottom line- seeking universal principles can give you something no one cares for and severely undermines opportunities for success. Testing and segmentation provide the means for radically improving your performance.

This has been well proven in all kinds of business, where even slight change in operating model can segment a market quickly by making one segment happier. However, much of this segmentation happens by chance as vendors evolve accidentally in different directions.

Product marketing and marketing, if they can harness this principle, can also shift buyer responses from wincing to deliriously happy.

(Note: this post was originally titled ‘Are You Serving Extra Chunky Products and Messages?’)

I am making the assumption, based on effort and focus, that my articles are interesting and, hopefully, somewhat unique. But, in keeping with the theme of my blog, I need to test that.

Also, my goal is relevant readership. I don’t have any ads on this site and page views aren’t the main goal. A strong core of relevant readership means two things: 1) you have a position/brand that is coherent and meaningful, and 2) it should grow steadily by building out from that core.

Finally, it’s a new blog so I’m focused on RATIOS rather then COUNTS.

So, let’s get started.

The hallmark of any good testing approach is focus. If you test too many things at once, you don’t really test anything very well. And you can obscure or confound the measurement of your goals.

Since interest and value are my main goals, I’m going to measure comment density, the ratio of comments to visitors. This also gives me a comparative measure across blog posts. However, early comments drive more comment traffic, so that needs to be considered as well- some posts will, by random nature, get early posts that drive more posts. The questions raised by the post are essential as well- many long running comment threads are more about discussions within the comments then the content of the piece itself. Finally, the source of the traffic may effect the inclination to comment as well. Something that is tweeted may bring in more comment-inclined people the content found via search or regular readership subscription (email or RSS).

This brings up an important caveat regarding testing: testing without analysis is dangerous. And analysis is hard- often because it’s hard to get a good set of data that allows you to control for confounding data. Try asking “low fat or low carb” on a few diet forums to get a sense of that. In fact, “diet and the solution of the obesity epidemic” suffers from the same challenges as “marketing and the solution of the low growth/profit epidemic”.

Or consider borrowing from Gallup polls: “our market strategy is likely accurate to within 4 percentage points 4 out of 5 times”. Maybe we should consider similar framing of our testing analysis?

Even if we can’t measure statistical significance in many of our testing endeavours, keeping that top of mind will push us to think deeper about causality, correlation and the likelihood we have a good grasp of the reality in front of us. Look at the content in the following link:

http://en.wikipedia.org/wiki/P-value

Do you think that’s cool for marketing, or irrelevant? I hope you answered cool.

In a similar vein to comment density is subscription density. How many visitors subscribe to your feed? This is a great measure of content as well. However, this is also a reflection of brand – do people know you from somewhere else- and, interestingly, comment density. I know I’m far more likely to subscribe to a blog that has good and interesting comment traffic. It’s social proof of the blog quality, leading to my greater interest even if the current post doesn’t rock my world. It’s also significant content in itself. Have a look here:

http://www.marksdailyapple.com/definitive-guide-to-primal-supplementation/

While the post didn’t interest me so much, the comments did. They gave a sampling of social agreement on the premise of the post, though obviously with a bias that the readership is self-selected to be somewhat in alignment with the blog outlook. Vitamin’s mostly fail miserably from a metrics-based outcome valuation, so I personally save my money. But I like to track the topic.

As an analyst, also, note what is missing and the strength of the data. What is missing? Anything representing useful metrics to make a decision. What are the criteria that drive a choice to supplement? Just-in-case! Does that remind you of some marketing strategies? What would happen if we invested in food rather then supplements? What is the best investment for the mostly like improvement of the outcomes we seek?

Just-in-case is expensive because it’s often wrong. And it’s susceptible to change. Maybe marketing often fails for the same reasons diets do.

Let me stop before the comparison gets out of control. The point of the above example is to show the comment-effect of a blog on perceived value, real value and feed subscription rates. As a side point, I mention the value of not just looking for what’s there, but also for what’s missing. A keen eye to what’s missing can quickly uncover areas that are particular susceptible to confirmation bias.

As a side note, I also looked at Amazon to see the reviews of the blog principles book, and took that to be a possible measure of the interest and quality of the content. However, unless I can segment the Amazon reviews by 1) organic traffic via Amazon vs. 2) the self-selected pro-primal segment from the blog subscription base, and then control for the 3) early review pattern framing future reviews, I can’t really get a pure sense of the review rating meaning- especially comparatively to other books and ratings. Social media reach of blogs can have huge impacts on reviewer segmentation.

After comments and feed subscriptions, I would be looking at Tweets/Diggs/bookmarks, links, and traffic. It’s a new blog, so I put traffic last for now. I’m also content focused, so content quality, finding my audience, and testing for those is more important then traffic growth. Comments by virtue of being qualitative, offer more opportunity to gain a market understanding from which to strategize and understand traffic growth (a quantitative measure). Social media fills the same goal as comments, but isn’t as qualitative, and won’t be as prevalent at lower traffic levels.

Future measures might include things like inbound links and search engine ranking for certain keywords as described in books like Inbound Marketing and by vendors such as Hubspot (the authors of the book). They have a fun tool at http://blog.grader.com.

Ultimately, everything is related- traffic drives comments, which drive tweets, which, in turn, drives traffic and so on. But, initial spikes in traffic may not scale due to weaknesses in authenticity and focus. Hence, my approach on the initial focus and order of testing. However, over time measurement focus needs to change to highlight different aspects of a market and content. Testing only one thing will slowly reduce the value of information you are getting. Likewise, testing too much at once can be as bad as not testing at all due to confounding factors. In the end, a balance of qualitative and quantitative testing is critical, as each makes more sense in the light of the other, and that helps avoid misinterpretation.

The alternative to testing? Segment your market until it fits all your assumptions, biases, inclinations and beliefs. Otherwise known as do nothing. Sadly, confirmation bias often leads great volumes of work to the same conclusion as the do nothing process.

You may get lucky, and do well. However, alternatively, the market might not:

1. be big enough to generate enough revenue,
2. be coherent enough to develop mindshare and communication channels,
3. have enough urgency around the problem, either currently, or permanently, or
4. be stable enough to stay around for long.

In which case, you have a product and a market you can’t reach, or a market you can reach but doesn’t care, or a market that slowly disappears due to commodity pressures or encroachment from other solution types. Or you might have hit onto something that is simple, stable and meets your business growth desires. In which case, congratulations, and I invite you to read on for fun and future reference. You never know when the market might change.

So, even if you don’t test now, you will probably need to soon enough to 1)grow, and 2) stay relevant to the changing market.

And how to avoid the nasty influence of confirmation bias? That’s a topic for a future post and includes the importance of testing your message AND your assumptions.

8HFZYRUZJ62N

You Are Probably Wrong

The theme of this blog is “You Are Probably Wrong. Prove Otherwise”. My second choice was “Only The Paranoid Market Well”.

These are the premises behind TestDrivenMarketing.com- that reality is complex enough that we would do well to test our assumptions, and test them often. Some examples of tests would be win/loss analysis, measuring the value of marketing collateral or campaigns, or using response-testing for messages or website content.

For more about me, read the About section. It’s a deeper explanation of me the philosophy behind this blog.

Prove Otherwise

However, I believe that testing is a mindset, and can be applied qualitatively or quantitatively to a far broader set of the marketing journey, whether it be testing the message, testing market assumptions, or testing core concepts and beliefs at each stage of the marketing journey.

The hallmark of many great managers is their inclination to ask questions rather then tell people the answer.

You could think of them as not managing, but assumption-testing and analysis coaching. The question, in the end, is the most pervasive and valuable test. And learning to ask questions of your self and others in a productive, open-minded manner is the foundation of test-driven marketing. Indeed, you could think of a cross-functional team as largely beneficial as a structured testing-by-question environment.

In many ways, “test” simply means think harder, observe more closely, and find clever ways to find out about information sooner rather then later.

Marketing, ultimately, is about improved communication and shortening the learning curve for both our customer AND us.

And the best way to do that is to find out what works as specifically and and as fast as possible. As a parting thought, the upfront work in testing and planning leads us to this goal:

Strive to make planning hard and execution easy.

Where planning is collecting and processing inputs (win/loss, value metrics, constraints, project and launch plans), and execution is generating output (positioning, messaging, content, collateral, training).

Please Participate!

If you like or don’t like a post, please comment or ask questions! Also, if you want, you can:

Subscribe to this Blog via RSS

Follow Me On Twitter @NicholasVee

Join The LinkedIn Group

Visit Me on LinkedIn

Or, if old-fashioned, email me.

Thanks,

Nick Van Weerdenburg