Posted by: Matt Shanahan

Subscriber loyalty is a relative measure.  More is better, and less is worse. To qualify the level of subscriber loyalty, you compare that subscriber’s loyalty metric to all your other subscribers.  And in the B2C world, this is a straightforward process.  Let’s face it an individual only has 24 hours/day of attention that can be devoted to your product.  So it’s fair to compare any metric directly between individuals.

And while metrics can be used to compare individuals, how do you determine the comparable level of loyalty for an organization? In the B2B world where we’re dealing with significant variances in sizes, attention capacity between two organizations can be enormously different which makes comparison of engagement trickier. How do you compare the level of engagement for a large global organization with 2,000 users to a medium-sized locally focused organization with 1,000 users? No longer do we have the built-in apples-to-apples comparison found in the B2C world.

In the B2B environment, metrics need to be normalized to account for the differences in subscribers’ sizes and contracts.  Incorporating firmographic information (e.g., # of employees, revenues) to compare against actual metrics (e.g., subscriber loyalty), lets us take into account organizational capacity, allowing metrics to be scored and normalized and therefore comparable.  This approach lets you determine how much organizational demand you have—essentially, loyalty at the organizational level. 

Getting back to our example, if that large, global company with 2,000 users out of 15,000 employees, it has significantly less demand than the medium company with 1,000 users out of 1,500 employees.  

By overlaying the actuals against the firmographics for the company, we normalize traditional metrics, making them useful in the B2B arena.  This approach let us extend our understanding from “are the individuals within the organization loyal” to “is the organization itself loyal?”

Firmographics, Metrics Firmographics, Metrics

Posted by: Matt Shanahan

I am never surprised when someone says they measure subscriber loyalty only to learn they are really measuring engagement.  How and why a subscriber engages can be a loyalty driver, and understanding these drivers can help paid-content providers create loyalty programs.

When it comes to measuring engagement, it’s hard to argue with the approach of measuring everything.  The incremental cost of collecting three or three-dozen quantitative measures is negligible. Why not have duration and frequency of visit, click-through rate, bookmark data, RSS subscriptions, downloads and blog comments metrics available at your fingertips?

My view gets a bit more controversial when we start talking about how to derive meaning from all the engagement data points that are collected. The general model today is to sift through the data points and, based on instinct and/or experience, determine what knob to dial up or down.  An organization might create a feature that helps improves content resyndication, roll out a program that increases downloads, or create a promotion that increases repeat visits.  All are fine ideas, but they are mostly guesses, and the approach is often shotgun.

I find it fascinating that all of these objective analytics are translated into activity through a mainly subjective lens.  Call me a stickler, but it seems we need an approach these analytics in a way that’s, well, a bit more analytical.

What’s missing is a quantitative platform for comparison—a normalized metric for figuring out which engagement metrics matter. That’s where subscriber loyalty comes in.  Once you have a quantitative way of understanding the subscriber loyalty, the engagement metrics come alive. Demand Rating™ is our metric for subscriber loyalty. It enables organizations to rank subscribers and group them into comparables sets to investigate differences. 

For example, you can group subscribers with high loyalty scores and then compare their engagement metrics to those with low loyalty scores. The delta between these groups will offer insight into engagement metrics that drive loyalty. Or, you might compare loyalty across segments —maybe geographical, organizational, or firmographic—to understand which engagement metrics are important to that particular segment.

As you can see, engagement metrics are loyalty metrics, but they can be highly correlated to each other.  Further, loyalty metrics quell a lot of controversy about engagement metrics because it’s the key for quantifying contribution of each engagement metric.

Demand Rating, Metrics, Subscriptions Demand Rating, Metrics

Posted by: Pete Horadan

I talked earlier about the insufficiencies of current tracking approaches and their impact on behavioral analytics.  Let’s now talk about how it should be done—how we do it here at Scout Analytics.  Basically, our technology is centered around a unique signature concept.  We analyze dozens of unique attributes and utilize patent-pending algorithms to derive a unique signature for the device, network, even the individual user (biometric). 

You can think of signatures sort of like a genome—while there are overlapping characteristics—each carries a unique and identifiable pattern and sequence.

Of course, this isn’t easy or everyone would be doing it.  As you can imagine, machines, networks and even people change in different environments and over time—you install a new version of Flash, or change to Silverlight.  The cast on your broken right hand changes how you use the computer.  You change network carriers.  The trick is to have algorithms that take into account enough factors and have enough intelligence to sense the change as a progression of a previous signature, rather than a different signature.

We have algorithms for recording and tracking a Devices Signature, a Network Signature and a Biometric Signature.  Each of these unique signatures works to improve the accuracy of tracking and context during analysis.  So next time I’ll give a little more insight into each.  The development of these techniques is something that truly sets us apart, and it’s what makes Demand Rating™ work.

Behavioral Analytics, Metrics Accuracy, Behavioral Analytics, Metrics

Posted by: Pete Horadan

Last week, we looked at the serious acurracy issues associated with using cookies to track visitor behavior and engagement.  This week we’ll explore the accuracy of another often-used factor, IP address. In short, IP address provides even less information about users than a cookie. 

The accuracy issue starts with the fact that one user may have many IP addresses.   For an example, let’s look at me. I access different sites (e.g., TechCrunch) from home through my personal broadband, while at my coffeehouse through their broadband, or through the network at my corporate office. When I’m on the road, I broaden my access points to include hotels and airports. When using IP address to count visitors, the information collected for me might look something like this:

IP Address                    Domain                         Location
24.16.13.198             comcast.net                Woodinville, WA  
208.54.4.23               tmobile.net                   Issaquah, WA
70.103.221.189    scoutanalytics.com              Issaquah, WA
12.204.178.67          marriott.com                  New York, NY

Of course this is incorrect. What appears to be four unique visitors, is really only me. 

The flip side is true as well, namely one IP address may have many visitors.  The common example is found when visitors behind an office firewall may all appear to come from the same address. In this case, paid-content providers have no visibility into individual visitors or their different behaviors. So, when trying to analyze unique visitor counts from IP addresses, the problem is almost intractable.  Does a given IP address represent one visitor, many visitors, or just a fraction of a visitor?

On their own, IP addresses have serious limitations in terms of establishing unique visitors or tracking behavior, seriously over- or under-estimating the true picture. Improperly used, they can lead to faulty analysis.  IP addresses are best used to provide hints which I’ll discuss more in future posts.

Behavioral Analytics, Metrics Accuracy, Behavioral Analytics, Metrics

Posted by: Pete Horadan

So we’ve been making the point all along that Demand Rating™ provides a new lens on subscriber loyalty. Realistically though, the usefulness of behavioral analytics or web analytics are gated by accuracy.  So how do you get accurate online metrics?  What’s really knowable?

Let’s start with usage tracking.  The traditional approach is through the use of tracking cookies.  A cookie is a small piece of data that is stored on a device by the application. Each new browser that accesses an application is given a cookie, and the application relies on that cookie to track user behavior. The issue, of course, is that the cookie method of tracking is wildly inaccurate, and it’s simple to see why.

At the highest level, tracking based on cookies don’t track individuals at all—they track cookies.   Users access applications through a variety of browsers and machines—from the office, at home, from their mobile devices.  Cookies can and are often reset by the user or even automatically by the browser whenever it shuts down.  It is practically never true that an individual visitor will utilize a single cookied browser for access.

In fact a recent study at Penn State shows that “cookies are about just as inaccurate in estimating unique visitors as unique network addresses,” overestimating audience size by 7 -30 times! So, do you really want to know that 90 cookies accessed your application?  Or that three people did?  The whole flawed cookie process results in a huge overstatement of usage and a total inability to track actual people. 

In a future article, I’ll talk about a better approach that avoids these problems.

Behavioral Analytics, Metrics Accuracy, Cookies, Metrics