Modern medicine is evidence-based. The adoption of evidence-based medicine saw our lifespans double in the space of 50 years. That’s a remarkable achievement.

Sadly, evidence is missing in modern marketing practices. We rarely see any marketing discussion mention anything about significance, confounding variables, or correlation versus causation. Most blog and social media recommendations are anecdotal evidence masquerading as statistics. Overall, there is a disturbing lack of anything that one could fairly call evidence in recommending any particular marketing practice.

This becomes apparent when you start asking some simple questions about “statistically” justified marketing recommendations.

1. Are these numbers for B2B or B2C?

2. What is the impact of brand and market Leadership on these numbers?

3. What is the impact of fashionable trends on these numbers?

4. What is the sample size?

5. What is the breakdown by industry? Company size?

At this point, you are probably not getting your emails answered or your phone calls returned.

Some of these questions and related concepts have obvious ramifications- brand leaders are usually older, more mature, have larger budgets, and are more likely to optimize existing processes. So…any numbers hinting at a best practice and revenue growth needs to control for brand and market leadership. And any recommendation needs to be prefaced by “it depends”.

Generally, any statement like ’65% of best-in-class companies use X and grew an average of 20% more year over year then non-best-in-class companies’ is meaningless, or should be treated as meaningless without access to the support data and methods used. Without decent evidence and analysis there is absolutely no reason to make ANY kind of inference here between the use of X and growth. And even given that data and where there is a strong causal effect, there is the important question of whether the principle is applicable to other industries, markets, or business models.

In medicine, there is a very cool organization called The Cochrane Collaboration which aims to improve health care by providing systemic reviews of the quality of evidence being used to propose and select treatments- i.e. which studies are good, and which suck. Even in esteemed scientific circles peer-reviewed journals aren’t enough, and The Cochrane Collaboration aims to raise the bar by provide systemic evaluation of the quality of studies and the quality of how they are published. Is the data available? Can the study be repeated? These are other markers of quality are emphasized and graded.

Evidence-based medicine had doubled our lifespans. Would evidence-based marketing double our revenues? Is it even possible with so many confounding variables at play in the business world?

The Dawn of the Hyper-Literate Society

In Why Marketing is Becoming Like Software Development we discussed how marketing was becoming Test-driven and Agile, driven by the demands of the Internet business environment.

The basic reason for this was that the Internet has created hyper-literacy in buyers, radically changing the sales process in ways companies have been very slow to respond to.

Much like how the printed word extended the reach of general information to the masses, the Internet has extended the reach of specialized information to the masses. Literacy has been replaced by hyper-literacy, and this has radically influenced many of our institutions and behaviours. Patients know almost as much about their illnesses as their doctors, dieters are reading primary research on nutrition and arguing about causation versus correlation on diet websites. And buyers in B2B marketplaces are driving sales people crazy.

This raises the question- how much are sales and marketing changing? Will sales still matter?

New Channels of Communication Change The Game

Sales used to be the primary conduit for the customer to learn about a product or solution. There were essentially no other channels. A few brochures and white papers would be produced, but that was only in support of the sales interaction- that collateral wasn’t meant to sell, and never fit that closely to the customer’s specific reality. And this was the necessity based on reality- to do it any other way would have required the sales team dropping a file cabinet off at a customers site.

As a result, marketing wasn’t that important. Couldn’t be that important. Marketing worked on lead generation and trade shows because that was the best they could do through the limited customer communication channels available to them. Brochures were background value-add to the sales process, but often unread due to their being so generic.

That has changed, and companies are just starting to get it. The Internet has opened up many other channels for the customer to get information, and provides an encyclopedia of knowledge that allows him to find it.

When you hear about about “buying cycles” replacing “sales cycles” in sales strategy, this isn’t just an improved understanding of purchasing psychology. It’s something that has been enabled by a fundamental change in the environment.

Customers buy on their own terms because they can. And as a result, marketing has become far more important then it was 10 years ago, and the role of sales has changed – or should have changed- dramatically.

Marketing Has Become a Larger Part of Selling

The result of the Internet channel dominating the sales conversation (especially when we, the vendor aren’t present) means that Marketing has to become a much larger part of sales.

To do so, Marketing must do two essential things- support the Internet customer communication channel, and measure that channel through analytics.

Marketing Needs to Generate Content That Sells Rather then Supports Selling

Why? Again because the customer is consuming this information on his own via the Internet. If you aren’t providing it, someone else will.

This content also needs to be more specific and relevant to the buyer. There is no sales team present to interpret and communicate a deeper message- this is your chance to get your message through. Traditionally marketing content was marginally relevant due to no delivery channel to the customer (reference the awkward file cabinet point above) and a resulting generic message. That channel is now there, and better yet, it allows content to be filtered for the customer. It may be a huge virtual filing cabinet, but search engines and other technology allow the customer to quickly find what appeals to him, leaving him feeling like he’s reading a nice little on-demand magazine.

The Content Must Fit the Customer Need, No Interpreter Needed

Another challenge is that your sales team isn’t there to interpret the content or get feedback from the customer regarding the content. So, the content must be a near exact fit for the customer, necessitating a much different type of marketing collateral then has been previously the norm. This is the most significant challenge of the online marketing environment, and brings us to our next topic.

Marketing Needs to Do Lead Nurturing

Lead nurturing use to be clearly a sales function. It requires a soft touch, an understanding of the customer, and solution selling to present the information in a meaningful and manageable way.

This fails today for two reasons, one traditional and one purely a result of the modern marketing reality.

The Economics of a Sale Are Becoming More Challenging

A 30% cost of sale can’t be supported by most companies. It’s as simple as that. You can’t spend enough time with the customer to help with his education, and that means he’s on his own more often.

The Customer is On The Internet While You Are Busy Doing Other Things

The customer isn’t paused until the next sales meeting, waiting to be feted and wowed by your solution selling greatness. That’s now a small channel to his mind. He’s on the Internet, researching, browsing your website, browsing your competitors websites.

10 years ago the Internet was 4 years old from a business perspective and there wasn’t much content. The Internet was a small channel to the customer, smaller then the sales engagement.

5 years ago the Internet was 9 years old. There was some content, but it was still classic corporate brochureware. However, blogs were starting to take off, and more print industry articles and insights were available on the web. An expectation of useful information started to blossom in customers, and almost all of our customers were now active Internet users.

Today, the internet is 14 from a business perspective. It’s got attitude, and it thinks it knows everything. It doesn’t but it’s getting really close- at least outside of B2B marketing.

The expectation of content from people who use the Internet is now extremely high. It’s the first stop for information.

Marketing Needs to Do Analytics

The other side effect of the customer pursuing information on the Internet is that you aren’t there to interpret the customer reaction. Broader, deeper content is being put out to the Internet so the customer finds his knowledge from you, but what is his reaction?

That’s why analytics are essential. Without it, you are flying blind.

Online content allows you to measure your audience’s response and gain critical insight into customer outlook and behaviour. Five years ago this may not have been so certain- poorer content and a uncertainty if your buyers were represented in your website traffic made measurement hard.

Marketing Needs to Go Where The Customers Are, And Where They Are Listening

The Internet being the dominant channel of communication to customers clearly explains why there is now an exploding market of marketing automation vendors. Your customers are online, doing research, and becoming experts- your sales channel can’t compete. You can talk to them for an hour a week, but the Internet is there for them 7×24. They want to find answers, and they aren’t waiting for you.

This changes the value and purpose of content radically.

Is Sales Becoming Marketing Technical Support?

10 years ago content had to be presented face-to-face to customers because there were no other viable channels of communication.

Now, most content needs to be presented indirectly through the Internet because it is the most prevalent channel of communication. This also means that content (and marketing) end up doing more of the selling.

What does this mean for sales? Are they destined to become technical support for automated marketing efforts?

Not likely. Some radically new very early markets may be beyond effective marketing reach. And in all non-trivial markets relationships are still central to any sales or buying process, . In fact, properly managed, sales can focus on more value-add activities in marketing-sold markets, handling more accounts and driving more business. In most cases, the early soft touches guiding prospecting and lead nurturing need human contact. The flow of marketing content to the prospect needs guidance and filtering based on human contact. Facilitation, problem discovery, solution mapping- sales is becoming a lot more about consulting and project management. But the fact remains- that beast called the Internet is sitting there, humming away 7×24, always ready to hop in as a sales advisor to your hyper-literate prospect whether you participate or not. There is no stopping it, and this change in marketing is becoming a necessity, not a choice.

Software development is influencing marketing a lot these days with the emergence of Agile Marketing and Test-Driven Marketing- Agile and Test-Driven concepts being two concepts that have dominated software development for the last 5 years.

The reason is not a fad crossing over, but rather a more fascinating and powerful phenomena. As the Internet has become pervasive in our lives Marketing has become testable, much like software, as well as more impactful due to the amount of reading buyers now do during the buying cycle.

Agile software development embraces change and measurement, allowing responses to “market” signals that are leading indicators of failure. Signals include early customer feedback, early robustness feedback from automated tests, and ongoing feedback from regression testing, which captures failures due to the landscape changing.

But so what, you might ask? Marketing has become testable. Whoop-de-do. I’ve been marketing for twenty years, and we’ve done just fine without testing.

My answer? Marketing mostly sucked from a customer perspective. And this was effective and professional because there wasn’t much other option. Marketing wasn’t sales, and sales was the group responsible for strongly targeting messages to buyers. Marketing was background support. Marketing didn’t have access to the customer mind, by message or by volume of content (diversity in customers makes available volume of content critical for targeted messaging), and as a result marketing was largely product focused.

That’s no longer the case. Where sales was once needed to deliver targeted content, buyers now look for the content themselves online. And if they don’t find yours, they’ll find someone else’s. Last generation marketing that was background to the targeted sales message is now leading in the foreground, and landing with a awkward thunk.

In the new marketing environment, we need marketing to work well or we’ll fail. The Internet has changed how we sell, and marketing does a lot more of the selling because that’s where the customers are- at their computers researching their own needs. Selling has become buying facilitation, and if marketing doesn’t step up and do more of this type of selling with relevant targeted messaging, failure is almost unavoidable.

The other massive influence in the new marketing environment is that we can now measure if marketing is working. Considering that we now need marketing to work, that is very good thing.

Inbound marketing, Internet lead generation and Pay-Per-Click advertising (PPC) all test marketing through conversion rates on multiple website goals such as newsletter signup, RSS subscription, Facebook fans, white-paper downloads, requests for a demo, successful multiple touches, purchases, and so on. There are literally hundreds of amazing measurements and tests that can be applied- either on existing data or on data created from intentional tests.

This is enabling marketing to be test-driven, and in complex markets, it’s providing the tools for marketing to become agile. This is exactly what happened in software- the focus on feedback from customers and tests enabled software development to become agile. Indeed, the concept of Agile Software Development was somewhat meaningless until the feedback mechanisms were there.

Marketing has become like software. You run it, test it, and it either works or it doesn’t.

The bottom line is that today we must sell with marketing, and we can now measure our marketing. These two incredibly strong influences have combined in a way that will fundamentally change the entire business world.

And that means we can start applying some of the same processes- agile responses to market signals, testing for user adoption, testing for relevance, and actually building our marketing collateral so that it is in fact testable.

If you think marketing automation and automated lead nurturing are a fad, you better have lots of brand equity to ride on. Because if you don’t, you are not going to be able to build it in this brave new world.

For more on this topic, I recommend checking out some of the new wave of marketing automation vendors- Marketo, Manitcore, Eloqua , MarketBright or Hubspot, as well as the Analytics vendors such as Google Analytics, Yahoo Analytics, and Omniture and see what they are saying.

And if you are convinced and want more details, or unconvinced but open to the possibility, subscribe to this blog via the Test-Driven Marketing RSS Feed or subscribe with the simple email subscription form over there on the right sidebar. Our next post, Is Sales Becoming Marketing Technical Support?, goes deeper into the changing roles of Sales and Marketing.

There was a post yesterday by Seth Godin on testing and how B2B marketers often avoid the early testing work that can save so much money later:

“Business to business marketing is almost always better if you treat it like direct marketing. ”

and

“Get it right for ten people before you rush around scaling up to a thousand. It’s far less romantic than spending money at the start, but it’s the reliable, proven way to get to scale if you care enough to do the work.”

You can read the full post here- Getting to scale: direct marketing vs. mass market thinking.

Testing is a common theme today- analytics, metrics and data-driven activities are exploding in popularity- but adoption is  slow. Marketers simply don’t take the time to test.

Data-Driven Marketing, a new book on data-driven marketing by Mark Jeffery, spends a lot of time on the challenge of getting to a data-driven marketing process, with the first two chapters being about why it’s not being done and how to overcome that:

Chapter 1: The Marketing Divide- Why 80 Percent of Companies Don’t Make Data-Driven Marketing Decisions- And Those Who Do Are the Leaders

Chapter 2: Where Do You Start? Overcoming the Five Obstacles to Data-Driven Marketing

In chapter 2, when discussing starting with small experiments to test campaigns before rolling out, Jeffery says:

“Although the majority of marketers are aware of this approach, my research shows that the vast majority of marketing organizations, almost 70 percent, do not use experiments to pilot test marketing campaigns relative to a control group. Why? The answer is that most marketing organizations’ reward systems are based on activities, not results”

This is what Pragmatic Marketing calls “list based marketing” and actively tries to change with their excellent Effective Product Marketing seminar.

In Chapter 2 Data-Driven Marketing lists 5 specific obstacles:

1. Getting started

2. Causality

3. Lack of Data

4. Resources and Tools

5. People and Change

Jeffery’s core recommendation? Start small and use wins to driven recognition of value and garner more focus and investment in data-driven activities. Also, find and partner with like-minded people to gain more influence. He also makes the important point that planning for testing and measurability early on allows you to structure your campaigns for those goals without a significant increase in cost.

Avinash Kaushik, Googles Analytics Evangelist,  identifies similar issues in the world of web analytics:

“You’ll find that data or tool are not your problem, it really is your company culture (both in terms of using data or getting your site tech teams to do things to give you data)” from How to Choose a Web Analytics Tool: A Radical Alternative

And Avinash has a good post on Seven Steps to Creating a Data Driven Decision Making Culture. where he talks about making the data easily relevant to the needs of stakeholders and showing specific wins in those areas.

Taking the direct marketing practice of testing to other areas is compelling. David Ogilvy, in Ogilvy on Advertising, published 1985, spends a lot of time talking about testing and what advertising can learn from direct marketing. Reading this book suggests that much of the success of Ogilvy’s advertising business flowed from a strong product marketing mindset (knowledge of the product and customer) and a strong testing ethos- things that weren’t traditional in advertising at the time.

In short, everyone agrees they need to be more data-driven, barriers to getting there are largely cultural, and small wins are the way to shift the culture and gather the focus required to scale the resources and infrastructure needed to effectively get there.

And if you are having trouble getting momentum even towards getting analytics support even towards a small win? Maybe you can’t get access to that data?

Consider using external analytical infrastructure- tools like Hubspot’s website grader, the Compete website for competitive information (Keep Your Enemies Closer), and Alexa for website traffic information. Also, Google is more powerful the you think. Use Advanced Google search methods to analyze your site for inbound links (link:<yoursite.com>) and help you search your and competitors sites (site:<site.com>) to look for trends and statistics.

I am making the assumption, based on effort and focus, that my articles are interesting and, hopefully, somewhat unique. But, in keeping with the theme of my blog, I need to test that.

Also, my goal is relevant readership. I don’t have any ads on this site and page views aren’t the main goal. A strong core of relevant readership means two things: 1) you have a position/brand that is coherent and meaningful, and 2) it should grow steadily by building out from that core.

Finally, it’s a new blog so I’m focused on RATIOS rather then COUNTS.

So, let’s get started.

The hallmark of any good testing approach is focus. If you test too many things at once, you don’t really test anything very well. And you can obscure or confound the measurement of your goals.

Since interest and value are my main goals, I’m going to measure comment density, the ratio of comments to visitors. This also gives me a comparative measure across blog posts. However, early comments drive more comment traffic, so that needs to be considered as well- some posts will, by random nature, get early posts that drive more posts. The questions raised by the post are essential as well- many long running comment threads are more about discussions within the comments then the content of the piece itself. Finally, the source of the traffic may effect the inclination to comment as well. Something that is tweeted may bring in more comment-inclined people the content found via search or regular readership subscription (email or RSS).

This brings up an important caveat regarding testing: testing without analysis is dangerous. And analysis is hard- often because it’s hard to get a good set of data that allows you to control for confounding data. Try asking “low fat or low carb” on a few diet forums to get a sense of that. In fact, “diet and the solution of the obesity epidemic” suffers from the same challenges as “marketing and the solution of the low growth/profit epidemic”.

Or consider borrowing from Gallup polls: “our market strategy is likely accurate to within 4 percentage points 4 out of 5 times”. Maybe we should consider similar framing of our testing analysis?

Even if we can’t measure statistical significance in many of our testing endeavours, keeping that top of mind will push us to think deeper about causality, correlation and the likelihood we have a good grasp of the reality in front of us. Look at the content in the following link:

http://en.wikipedia.org/wiki/P-value

Do you think that’s cool for marketing, or irrelevant? I hope you answered cool.

In a similar vein to comment density is subscription density. How many visitors subscribe to your feed? This is a great measure of content as well. However, this is also a reflection of brand – do people know you from somewhere else- and, interestingly, comment density. I know I’m far more likely to subscribe to a blog that has good and interesting comment traffic. It’s social proof of the blog quality, leading to my greater interest even if the current post doesn’t rock my world. It’s also significant content in itself. Have a look here:

http://www.marksdailyapple.com/definitive-guide-to-primal-supplementation/

While the post didn’t interest me so much, the comments did. They gave a sampling of social agreement on the premise of the post, though obviously with a bias that the readership is self-selected to be somewhat in alignment with the blog outlook. Vitamin’s mostly fail miserably from a metrics-based outcome valuation, so I personally save my money. But I like to track the topic.

As an analyst, also, note what is missing and the strength of the data. What is missing? Anything representing useful metrics to make a decision. What are the criteria that drive a choice to supplement? Just-in-case! Does that remind you of some marketing strategies? What would happen if we invested in food rather then supplements? What is the best investment for the mostly like improvement of the outcomes we seek?

Just-in-case is expensive because it’s often wrong. And it’s susceptible to change. Maybe marketing often fails for the same reasons diets do.

Let me stop before the comparison gets out of control. The point of the above example is to show the comment-effect of a blog on perceived value, real value and feed subscription rates. As a side point, I mention the value of not just looking for what’s there, but also for what’s missing. A keen eye to what’s missing can quickly uncover areas that are particular susceptible to confirmation bias.

As a side note, I also looked at Amazon to see the reviews of the blog principles book, and took that to be a possible measure of the interest and quality of the content. However, unless I can segment the Amazon reviews by 1) organic traffic via Amazon vs. 2) the self-selected pro-primal segment from the blog subscription base, and then control for the 3) early review pattern framing future reviews, I can’t really get a pure sense of the review rating meaning- especially comparatively to other books and ratings. Social media reach of blogs can have huge impacts on reviewer segmentation.

After comments and feed subscriptions, I would be looking at Tweets/Diggs/bookmarks, links, and traffic. It’s a new blog, so I put traffic last for now. I’m also content focused, so content quality, finding my audience, and testing for those is more important then traffic growth. Comments by virtue of being qualitative, offer more opportunity to gain a market understanding from which to strategize and understand traffic growth (a quantitative measure). Social media fills the same goal as comments, but isn’t as qualitative, and won’t be as prevalent at lower traffic levels.

Future measures might include things like inbound links and search engine ranking for certain keywords as described in books like Inbound Marketing and by vendors such as Hubspot (the authors of the book). They have a fun tool at http://blog.grader.com.

Ultimately, everything is related- traffic drives comments, which drive tweets, which, in turn, drives traffic and so on. But, initial spikes in traffic may not scale due to weaknesses in authenticity and focus. Hence, my approach on the initial focus and order of testing. However, over time measurement focus needs to change to highlight different aspects of a market and content. Testing only one thing will slowly reduce the value of information you are getting. Likewise, testing too much at once can be as bad as not testing at all due to confounding factors. In the end, a balance of qualitative and quantitative testing is critical, as each makes more sense in the light of the other, and that helps avoid misinterpretation.