The web today is a major revenue channel for almost all companies of all sizes – it is major sources of consumer intelligence – with every click customers communicate their desires and needs.  Enterprises and marketers need to focus on what consumer behaviors on websites are dictating and demanding.  The consumer web data streams can start with an enterprise’s website, but the tapestry of behavioral analytics can include chat, transcribed phone calls, email, wireless and other transactional databases.  At the lowest possible level consumer behaviors can begin to be captured at the Web sever log files – these are the most common formats:

• National Center for Supercomputing Applications (Common, Combined, and Separate)
• Custom Log File Format (information captured is defined by the user)
• W3C Extended (used by Microsoft IIS 4.0 and 5.0)
• IBM Tivoli Access Manager WebSEAL
• WebSphere Application Server Logs
• Sun^TM ONE Web Server (iPlanet)

For behavioral analytics log files can be used to track the movements and the return of website visitors.  Log files can reveal how visitors arrived at a web site, they can show which section of a site is most popular and an assortment of other web metrics.  Analog was one of the first log file analysis program back in 1995, soon after several commercial log analytics program came to market.  One of the first was WebTrends which added various graphs, tables and other business metrics, soon other firms followed, such as Accrue, WebSideStory, Coremetrics and dozens of others. Today these are the most popular log analyzers: 

Yahoo! Web Analytics – An enterprise site analytics tool that provides real-time insight into visitor behavior on their website.

There are limitations to log files which report more on browser activity rather than consumer behaviors.  There are also problems with search bots which crawl around sites creating traffic which is non-human, then there the problem of recognizing unique visitors who today are typically assigned dynamic IP addresses, one potential solution is assigning unique identification number cookies.  Log files can be configured to report on a these unique entity cookies: USERID=CustomerA;IMPID=01234 the advantage to this approach is that this value can be linked with other enterprise databases.  As we found out in the prior section a cookie is simply a small text file that is used for tracking consumer clickstream behavior.  A cookie typically contains the domain, a lifetime date, and a value, usually a randomly generated unique number for deep link tracking by the issuing website. JavaScript tags have become the most popular method of collecting web data streams, by simply adding a few lines of JavaScript code to a page all the data from visitors activities can be collect via a cookie:

function createCookie(name,value,days) {

     
                                    if (days) {

              var date = new Date();

             
                                    date.setTime(date.getTime()+(days*24*60*60*1000));

             
                                    var expires = "; expires="+date.toGMTString();

        }

       
                                    else var expires = "";

        document.cookie = name+"="+value+expires+";
                                    path=/";

}

function readCookie(name) {

       
                                    var nameEQ = name + "=";

        var ca = document.cookie.split(';');

       
                                    for(var i=0;i < ca.length;i++) {

               var
                                    c = ca[i];

               while (c.charAt(0)=='
                                    ') c = c.substring(1,c.length);

               if
                                    (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length,c.length);

        }

       
                                    return null;

}

function eraseCookie(name) {

       
                                    createCookie(name,"",-1);

}

JavaScript log files are easier to maintain than server log files and as the shift to the Web as the main channel for consumers, enterprises have shifted the responsibility for web analytics, such as log maintenance and reporting from internal IT shops to outside vendors.  Typically these web analytic vendors rely of JavaScript tagging to track and report on visitor activities. Most of which report on such web metrics as page views, number of visitors, top exit pages, and other measurements that provide little useful information for behavioral analytics. However, new vendors such as Lyris and Omniture allow marketers and enterprises to capture deeper web data which is required for the segmentation of visitor behaviors that provide much more insight than a count of web site ‘hits.’ 

An optional method of collecting web data is via a packet sniffer which are special types of hardware from such vendors as Clickstream Technologies and SiteSpec, some of which work with JavaScript.  Packet sniffer are an outsource solution to web analytics which sit between the consumer browsers and an enterprise’s corporate website, hence they can collect absolutely everything including passwords, name, addresses, and credit card numbers. One caveat about packet sniffers is that they cannot capture and report from Adobe Flash, Ajax or RIA files. The optimal solution is the merge of packet sniffers and JavaScript technologies for capturing and reporting on consumer behaviors.

As with all behavioral tracking cookies cannot be used to identify consumers: behavioral analytics is a reaction to behavior not individuals – privacy is not an issue.  Cookies allow an enterprise to recognize consumers when they return to their website by responding with targeted content or services.  For most purposes a website setting a cookie does not need to know the identity of the consumer - it just needs to remember the browser prior visit.  The important fact to remember most sites don’t know a consumer by their name rather they know him or her as: ID 511946jm33dd8839.  To learn more about cookies go to cookiecentral.com.

Marketers and enterprises have the option of writing personal information about consumers to a cookie gleamed from registration forms as long as it is stated in the website privacy policy.  If personal consumer information is stored in a cookie it should be encrypted to protect the privacy of the consumer.  The use of cookies for behavioral analytics is totally in control of an enterprise, so that their website may create a cookie with unique but anonymous content; or on the server side they may create a file that logs that anonymous content alongside any personal information that consumers have provided, such as their zip code, or the type of product, service or content they prefer via the completion of a registration, survey, contest or order form.