I am making the assumption, based on effort and focus, that my articles are interesting and, hopefully, somewhat unique. But, in keeping with the theme of my blog, I need to test that.
Also, my goal is relevant readership. I don’t have any ads on this site and page views aren’t the main goal. A strong core of relevant readership means two things: 1) you have a position/brand that is coherent and meaningful, and 2) it should grow steadily by building out from that core.
Finally, it’s a new blog so I’m focused on RATIOS rather then COUNTS.
So, let’s get started.
The hallmark of any good testing approach is focus. If you test too many things at once, you don’t really test anything very well. And you can obscure or confound the measurement of your goals.
Since interest and value are my main goals, I’m going to measure comment density, the ratio of comments to visitors. This also gives me a comparative measure across blog posts. However, early comments drive more comment traffic, so that needs to be considered as well- some posts will, by random nature, get early posts that drive more posts. The questions raised by the post are essential as well- many long running comment threads are more about discussions within the comments then the content of the piece itself. Finally, the source of the traffic may effect the inclination to comment as well. Something that is tweeted may bring in more comment-inclined people the content found via search or regular readership subscription (email or RSS).
This brings up an important caveat regarding testing: testing without analysis is dangerous. And analysis is hard- often because it’s hard to get a good set of data that allows you to control for confounding data. Try asking “low fat or low carb” on a few diet forums to get a sense of that. In fact, “diet and the solution of the obesity epidemic” suffers from the same challenges as “marketing and the solution of the low growth/profit epidemic”.
Or consider borrowing from Gallup polls: “our market strategy is likely accurate to within 4 percentage points 4 out of 5 times”. Maybe we should consider similar framing of our testing analysis?
Even if we can’t measure statistical significance in many of our testing endeavours, keeping that top of mind will push us to think deeper about causality, correlation and the likelihood we have a good grasp of the reality in front of us. Look at the content in the following link:
http://en.wikipedia.org/wiki/P-value
Do you think that’s cool for marketing, or irrelevant? I hope you answered cool.
In a similar vein to comment density is subscription density. How many visitors subscribe to your feed? This is a great measure of content as well. However, this is also a reflection of brand – do people know you from somewhere else- and, interestingly, comment density. I know I’m far more likely to subscribe to a blog that has good and interesting comment traffic. It’s social proof of the blog quality, leading to my greater interest even if the current post doesn’t rock my world. It’s also significant content in itself. Have a look here:
http://www.marksdailyapple.com/definitive-guide-to-primal-supplementation/
While the post didn’t interest me so much, the comments did. They gave a sampling of social agreement on the premise of the post, though obviously with a bias that the readership is self-selected to be somewhat in alignment with the blog outlook. Vitamin’s mostly fail miserably from a metrics-based outcome valuation, so I personally save my money. But I like to track the topic.
As an analyst, also, note what is missing and the strength of the data. What is missing? Anything representing useful metrics to make a decision. What are the criteria that drive a choice to supplement? Just-in-case! Does that remind you of some marketing strategies? What would happen if we invested in food rather then supplements? What is the best investment for the mostly like improvement of the outcomes we seek?
Just-in-case is expensive because it’s often wrong. And it’s susceptible to change. Maybe marketing often fails for the same reasons diets do.
Let me stop before the comparison gets out of control. The point of the above example is to show the comment-effect of a blog on perceived value, real value and feed subscription rates. As a side point, I mention the value of not just looking for what’s there, but also for what’s missing. A keen eye to what’s missing can quickly uncover areas that are particular susceptible to confirmation bias.
As a side note, I also looked at Amazon to see the reviews of the blog principles book, and took that to be a possible measure of the interest and quality of the content. However, unless I can segment the Amazon reviews by 1) organic traffic via Amazon vs. 2) the self-selected pro-primal segment from the blog subscription base, and then control for the 3) early review pattern framing future reviews, I can’t really get a pure sense of the review rating meaning- especially comparatively to other books and ratings. Social media reach of blogs can have huge impacts on reviewer segmentation.
After comments and feed subscriptions, I would be looking at Tweets/Diggs/bookmarks, links, and traffic. It’s a new blog, so I put traffic last for now. I’m also content focused, so content quality, finding my audience, and testing for those is more important then traffic growth. Comments by virtue of being qualitative, offer more opportunity to gain a market understanding from which to strategize and understand traffic growth (a quantitative measure). Social media fills the same goal as comments, but isn’t as qualitative, and won’t be as prevalent at lower traffic levels.
Future measures might include things like inbound links and search engine ranking for certain keywords as described in books like Inbound Marketing and by vendors such as Hubspot (the authors of the book). They have a fun tool at http://blog.grader.com.
Ultimately, everything is related- traffic drives comments, which drive tweets, which, in turn, drives traffic and so on. But, initial spikes in traffic may not scale due to weaknesses in authenticity and focus. Hence, my approach on the initial focus and order of testing. However, over time measurement focus needs to change to highlight different aspects of a market and content. Testing only one thing will slowly reduce the value of information you are getting. Likewise, testing too much at once can be as bad as not testing at all due to confounding factors. In the end, a balance of qualitative and quantitative testing is critical, as each makes more sense in the light of the other, and that helps avoid misinterpretation.
