Category: Web Analytics

Not long before I was given an opportunity to present at the 2006 eMetrics conference in DC, I had been promoted to manage the combined web and business intelligence teams at USAC. The reason was simple — management by metrics was the only way to go if this non-for-profit organization responsible for the $8 billion Universal Service Fund was to be efficient. My thinking then was that web analytics and business intelligence were complementary. Today, my thinking is that they should not be viewed as separate disciplines anymore.

Web Analytics and Business Intelligence — Marriage Made in Heaven

The days of the separate tools are numbered. Big data vendors left and right are offering ways to integrate data from disparate sources. The skill sets should not be unique anymore as we are all to analyze the ways for optimizing an organization’s engagement with customers and constituents as well as its operational efficiency.

If Web Analytics is Business Intelligence, as the DA blog claims, why do we still view them as separate disciplines? If eConsultancy is inviting us to break down the silos between Web Analytics and Business Intelligence, why do we still cling to these obsolete boundaries?

True, there is data that originates on websites and data that originates in transaction systems but data is data. Do we need to split mobile analytics from web analytics? What about piping data from the internet of things? It is time this false dichotomy be put to rest.

What we need are skilled data analysts and flexible data analysis tools!

 

 

Last day of my WebTrends training. We continue with Custom Reports and will focus today on drill down reports.

With parameter analysis, we are limited to two parameters; with custom reports, also limited to two parameters. Drilldown dimensions shows you all the dimensions you would want to track down and can select for your custom report.

Another use for drilldown reports is when you have a lot of external information and you want to put it together. There is a pre-defined hierarchy for campaign attributes in WebTrends.

Using SmartView to track customer behavior

SmartView integrates actual view of the website in a browser with the WebTrends data. It is a great tool for novices to web analytics to give them a sense of what information is available.

For some of the functionality of SmartView you need to use session or persistent cookies or user ID and password — IP / User Agent sessionalization is not sufficient. Also, visitor history needs to be enabled.

SmartView compares the URI on the link to data. SmartView does not work with Javascript / pull-down menu links.

If you have two links on a page going to the same destination, i.e. top link and bottom link, you need to add the following parameter WT.svl to each link, i.e.

specials.asp?WT.svl=top
specials.asp?WT.svl=bottom

This would split the reported traffic between the unique links.

Install SmartView on each user’s machine and give them view access to SmartView. Access is based on profile basis.

Advanced SmartView reports are based on “page transitions”. These are combination of the page the visitor was on and the next one the customer visited.

Today is the third day of my WebTrends training and the first day of the advanced reporting training.

We will focus on :

– visitor history table usage in WebTrends
– configuring campaign tracking
– defining new dimensions, measures, filters to create custom reports, etc.

Clearing the confusion of hit metrics and visit metrics:
– Log file with 1 visit consisting of 3 page views, where each page view was to a different content group, nevertheless there is only one visit (although it was a visit to three different content groups)

Visitor History

To use visitor history you need to use persistent cookie, session cookie or persistent authentication (user login). By default, visitor history is not enabled and can be enabled by clickin on the Visitor History tab of the profile overview. If configurning the profile to export the visitor history, it would get stored in a specified directory on a daily, weekly or monthly basis.

Visitor history can include the following behavior categories:

– Campaign History
– Search Engine History
– Most Recent Search Engine Duration
– Visit History
– Purchase History
– Custom Visitor Segmentation
– Content Group Unique Visitor Tracking – new to 7.5
– Page of Interest Unique Visitor Tracking – new to 7.5

You can invent your own segmentation through tagging. Unique visitor would translate visits to visitors. The precision of unique visitor tracking is based on the cookie. The same visitor using different browsers from the same PC would be counted as two different unique visitors.

Since visitor history might grow, trimming can be handy. Visitor history trimming is based on several criteria — number of visits, lifetime value, most recent visit, most recent purchase, etc.

Visitor history gets updated based on the profile. If you want retroactive data you need to reanalyze.

Visitor history is important as a tool for visitor retention. It is cheaper to keep a customer happy than to get a new customer.

In version 8, Marketing Lab would enable the mashing of web log data and additional customer relationship information.

Visitor history can be used to understand visitor behavior:
– on which visitors should you spend marketing dollars?
– should some visitors be contacted more often than others?
– which campaigns generate the best visitors, etc?

Visitor History Table

– Visitor cookie ID (based on cookie sent by SmartSource or your own cookie parameter)
– Contains attributes on a per visitor basis:
– number of hits
– number of visits
– time of first visits
– time of last visits, etc.

Visitors tracked by IP address/User Agent will not be stored in the Visitor History Table.

Consider if you need to use visitor history because it slows down reporting and takes hard drive space. Try it by turning it on for a couple of profiles and see if you would get value from it.

Measuring Campaign Performance

To track paid (vs. organic search engine keywords), use the following tags:
WT.mc_ID=”marketing campaign code”&WT.srch=1

WT.srch=1 would automatically put the data into the paid-search category reports.

When tracking campaigns from multiple channels, the easiest is to have unique campaign ID. Wrap the campaign ID translation table which is limited to 26 columns (in custom reports). Why 26, because each column in Excel is associated with a letter. Possible values are:
– campaignID
– description
– offer
– creative type
– channel

The canned WebTrends campaign report uses 11 columns in the translation table. For the columns that are not used, you can put commas in the columns with no data.

The prefered method for tracking each campaign in WebTrends is to use the WT.mc_id parameter.

META tags:
1. Unique landing page (tagged page)
2. Pass WT.mc_id as part of the page URL

META tag syntax:

The META tags have to be above the SmartSource javascript code.
Campaign ID is a visit parameter.

Custom Reports

Custom Reports consist of one table and its associated graphs. Once created, reports are automatically added to:
– custom reports main screen
– custom report tab on profile
– reports pick list in Dashboards
– report pick list in Templates

Custom Reports can have a primary and secondary dimension, i.e. domains and URL with Query String. You can have up to 20 measures.

To create a Custom Report:

• Define the components:
1. Data sources first
2. Dimensions
3. Measures
4. Filters
• Custom Report
• Add the Custom Repor to profile

Translation files are easy but the downside is, they get stale. It is OK to sue translation table for things that would not change, for product categories for example, but for more dynamic information, like pricing, not.

Data Sources are associated with Dimensions and Measures.

When creating a Custom Report, make sure to check the box for “Exclude traffic where this dimension was unspecified”, otherwise the report would include those that did not mean the criteria for the dimension.

In a basic report you cannot turn on the Custom Report. Make sure to select advanced profile for custom reports.

Important Tip: To make sure that the Custom Reports show in every report, go to Report Designer > Templates and select to edit the Enterprise Complete View report. Click the Content tab and then click on the Enterprise Complete View V7.5 item on the left navigation. Add a new report and select from the Custom Reports list the following one: “Add a folder to my template that will include all current and future Custom Reports.” When you click on it, you will see “Auto-Populated at Report Time”.

Visitor History is a nice new feature of version 7.5 and up of WebTrends. It requires strong authentication, i.e. session cookies, persistent cookies but not IP address & user agent. Don’t turn it on if using IP address for authentication because it does make the profile larger and slows them down.

Running three segments of large amounts of data of 10 days each would run generally faster than 10 segments of 3 days each. But make sure you have the RAM and the hard drive in place before running large amounts of traffic.

Page files and download files can be defined on per profile basis. For example Flash files can be both downloads and page views. By default they would be under page views but it makes sense to configure the profile and assign swf (Flash files) as download files.

In Reports > Reports Headers you can customize the logos of the banner and replace the WebTrends Report Banner with custom images. The middle image is a spacer and it tiles it, so if the logo is large, do not use the middle image.

In Reports / Reports Types, turning all five reports types make sense — Yearly, Quarterly, Monthly, Weekly, Daily. If the reports start on June 1st, clicking on Quarterly, does not mean it will generate Quarterly reports for the first two quarters.

Parent-child profiles split your web traffic based on configurable rules, and create multiple report sets. This is useful for service providers and large enterprises. For example, a split based on domain can create a separate report for each domain.

When WebTrends analysis a log file it looks at the first hit, last hit and file size to compare and check whether it had run a report already.

Have a log file directory which keeps the recent log files.
Current directory for storing 1 – 30 days.
Another directory for storing 1 month – 6 months.
Another directory for storing older than 6 months.

Another approach is to use date macros in the log file paths:
ex%date-1%%yy%%mm%%dd%.log – for yesterday’s files
ex%date-2%%yy%%mm%%dd%.log – for 2 days ago
ex%date-3%%yy%%mm%%dd%.log – for 3 days ago

This always checks the last three days.

For special request profiles, the recommendation is to use a separate data source.

New to WebTrends 7.5 is the ability to schedule the download of reports. The smart thing to do is to design the template for the reports they need to see on a regular basis. In Scheduler > Scheduled Events > Schedule Report > Select profile, etc. Select a custom data range for the reports to be emailed.

Clear Analysis Data is different from Reanalyze. Make sure to have the new data source or editing the data source so reanalyzing would be meaningful.

To track the last three days of the SDC log files, use the following macro where sdc is the unique sdc identifier (usually 20 characters) if SDC stores logs on a daily basis:
sdc_%date-1%%mmm%_%dd%_%yyyy%.log – for yesterday’s log file
sdc_%date-2%%mmm%_%dd%_%yyyy%.log – for the day before yesterday
sdc_%date-3%%mmm%_%dd%_%yyyy%.log – for two days before yesterday

Session tracking: visits vs. visitors

Visit — all hits for any one visit
Visitor – anyone entering the web site

Sessionalizing by persistent cookie is considered more reliable than IP address & user agent. However, the same person visiting from work and home would be counted as two separate visitors.

To setup session analysis go to Web Analysis > Session Tracking. For example to track by parameter, we would use Parameter Match. WT scrolls you through grayed-out methods that are not used. Then in track sessions by parameter you would specify the parameter name.

If switching from 3rd party to 1st party cookie, you must redeploy the new tag to all your pages. Identify fallback method, most likely IP address and user agent.

Filters: All profiles by default come with all include filters. There are include and exclude filters. If there is not filter, all activity is included. There is no limit to the number of filters per profile.

To exclude spiders, create a new hit filter. From the browser drop down menu, select All Spiders and Robots [allspiders] which would include the browser.ini file which includes over 500 spiders.

Filters work from now on.

To analyze all campaigns, listing the Campaign ID in the profile would give you only number of visits but not a drill-down report. For “butterfly”-type drill-down report, create a separate profile filtering all entries from ?WT.cm_ID=*

Creating an include filter for the custom 404 files only would give a list of most failing pages which would be very valuable for web developers.

Multi-homed domain log file includes multiple data sources.

Remember, OR is more, AND is less.

If you are to email business users the reports they want to see, find out what they want and put it in a template.

I am attending WebTrends classes for technical professionals and will share some of my notes taken durnig the classes. The notes will not be complete nor very eloquent but will capture the most important information during the training.

Notes from day 1: WebTrends Essentials for Technical Professionals

Content groups can be defined two ways: through content words contained in URLs or through page tagging.
The page tag for content groups is WT.cg_n:
Set meta “WT.cg_n”=content “keyword” – for content groups (URL+WT.cg_n=”keyword”)
Set meta “WT.cg_s”=content “subkeyword” – for sub content groups

WebTrends by default truncates query parameters.

The report graphs and dashboards require the installation of Java which would require one to be administrator on their PC.

Daily reports saves hourly traffic. It makes sense to keep the daily reports for not more than 3 – 6 months.

Custom calendar range used daily reports. Comparative calendar range uses weekly reports.

/products/prices.asp
/products/prices.asp?ID=4
/products/prices.asp?ID=4&ID2=801350

WebTrends would look at it as three different page views although it is the same page. You would need to build either custom report or parameter analysis to avoid the URL truncation.

You can use translation table mapping query IDs to titles of the articles published.

Path analysis can give you view into 20 steps before and 20 steps after a defined URL. Scenario analysis would have the actual URLs and query parameters for each step and view the path analysis with query parameters.

In a form with multiple fields, you can tag each field to create a scenario analysis with one page only but multiple fields using the on-click event.

In the WebTrends Resource Center, very valuable information is in “Understanding Data Collection” under Implementation > “WebTrends Query Parameters” and “Tracking Downloads and Form Completions”.

With SmartSource Data Collection information about PDF downloads is not logged. You need to do on-click event to track PDF download.

To track Google paid search results, use the following tags:

WT.mc_ID=10001&WT.srch=1 where 10001 is the marketing campaign ID and WT.srch=1 tells WebTrends this is paid search and automatically updates paid search terms and phrases.

WebTrends cookies by default are set as persistent cookies with 10 year expiration. Cookies are the primary identifier for visitor history.

WebTrends_ID cookie contains customer IP address, time in milliseconds since 1974 and duration.

Rejection for 1st party cookies are around 12%, 3rd party cookies around 25%. Home users of AOL readily accept cookies in IE but Firefox does not accept 3rd party cookies by default.

Thirty minutes of inactivity ends a user session. Multiple users through a corporate firewall would generate a single session as long as there is less than 30 min between visitors. AOL can overstate page views by assigning randomly a shared IP address.

Usually traffic measurements based on cookies is better by 20 – 30%.

1st party cookies means that my site is issuing the cookie.

Set in the profile to save 180 daily reports and delete the older reports — rolling 180 daily reports. The feature to remove old files is disable by default.

Deleted Data Cleanup is under Scheduler > Scheduled Events. To enable the feature, unclick the check box for disabling the feature.

Keep quarterly and yearly reports for ever. Keep daily reports only for the last 6 months.

Standard size for a profile is about 1 MB per day. Recommendation no more than 30 custom reports per profile.

Version 8 will allow you to define what is an event. You can define what events you want to scoop out of the log file and automatically export into Oracle or SQL server and run queries there.

The proprietary database is non-querable; in version 8 you can export it for queries.

Comparative calendar view would compare week vs week or month vs month — likewise ranges.

Important: For custom calendar range the dates need to be contiguous. Visitors summary would show incorrect visitors summary in a custom calendar range because it sums up the daily visitors for each day into a total numver for the custom calendar range. You would need to reannalyze the data for the custom period if you want the total number of visitors.

Webtrends tracks annual visitors based on calendar year. So a visitor in December 2005 and a visitor in January 2006 would be consider as different even if they use the same IP address, same browser, same cookie.

Custom comparative view would allow you to compare two custom periods.

All custom reports add only dailies.

A template is a collection of reports — i.e. executive, marketer’s and webmaster’s reports. The dashboard always starts with the default template but I can choose a different template if given access to more than one templates.

Dashboards are not exportable. Reports are exportable into Word, Excel, PDF, or CSV format. You need to install the WebTrends Report Exporter.

Recommendation: Allow sharing bookmarks only among people who are knowledgable about WebTrends to avoid having the bookmarks list be longer than the content list.

You can bookmark queries.

By default of SDC rotates log files hourly, meaning it creates 24 files per day; the recommendation is to change to daily.

Create unique data source per server but use more than once data source per profile. It is better to read the SDC log files from the WebTrends server rather than from the SDC server.

WT.sdl is used exclusively to track multiple links from the same page, i.e. WT.sdl=top would be added to the top link and WT.sdl=bottom would be added to the bottom link.