Scout Analytics is not the first to monitor account usage and flag unlicensed use. There are commercial and homegrown account analysis tools out there. Most of these tools try to draw correlations between time (concurrency), IP address, and browser information.  A recent set of questions from customers were:

• How good can these tools be?
• What is the difference in performance from Scout Analytics?

Investigation
Using real world data, we looked at different techniques to analyze the IP addresses such as: total count, average interval, frequency, smallest interval, and others.  All the techniques computed an IP-related metric for an account (such as the total count of distinct IP addresses) and compared that measure to a threshold.  Next we compared the percentile of shared accounts above the threshold to the percentile of non-shared accounts above the threshold. This gives us an idea of how effective a measure is.

The best results for detecting unlicensed use from IP address was measuring the smallest interval between logins originating from different IP addresses.  Unfortunately, the technique has a high false positive rate associated with it.  To detect 25% of the accounts with unlicensed use, the technique would have a false positive rate of 10%!  Dropping the false positive rate to 1% would only allow 4% of the violators to be identified.  Overall the performance of IP address techniques is poor.

Thoughts
Why do IP address related measures perform so poorly?  IP address counts have a big range in values for a single-user account (e.g., mobile workforce). Some accounts used a single address, others used over 80! This range overshadows the vast majority of differences that might be caused by having multiple users in an account vs. only having a single user.

Account Sharing, Revenue Assurance, Unlicensed Use Unlicensed Use

No.  At least not very accurately.  Some online services have tried to use high frequency of access as a positive indicator of account sharing.  The following example illustrates the limitations of that metric.

Scout Analytics has been analyzing access patterns for one online service that has approximately 6,000 active users.  A recent regression analysis was performed where the percentage of shared accounts was correlated to the frequency of access as measured in standard deviations from the median.  The graph below shows that frequency of access was not a good indicator until more than 9 standard deviations above the median which represented < 1% of the shared accounts.

Account Sharing, Revenue Assurance, Unlicensed Use Unlicensed Use