WebTrends Training for Technical Professionals – Day 4: Advanced Reporting – Drilldown Reports

Last day of my WebTrends training. We continue with Custom Reports and will focus today on drill down reports.

With parameter analysis, we are limited to two parameters; with custom reports, also limited to two parameters. Drilldown dimensions shows you all the dimensions you would want to track down and can select for your custom report.

Another use for drilldown reports is when you have a lot of external information and you want to put it together. There is a pre-defined hierarchy for campaign attributes in WebTrends.

Using SmartView to track customer behavior

SmartView integrates actual view of the website in a browser with the WebTrends data. It is a great tool for novices to web analytics to give them a sense of what information is available.

For some of the functionality of SmartView you need to use session or persistent cookies or user ID and password — IP / User Agent sessionalization is not sufficient. Also, visitor history needs to be enabled.

SmartView compares the URI on the link to data. SmartView does not work with Javascript / pull-down menu links.

If you have two links on a page going to the same destination, i.e. top link and bottom link, you need to add the following parameter WT.svl to each link, i.e.

specials.asp?WT.svl=top
specials.asp?WT.svl=bottom

This would split the reported traffic between the unique links.

Install SmartView on each user’s machine and give them view access to SmartView. Access is based on profile basis.

Advanced SmartView reports are based on “page transitions”. These are combination of the page the visitor was on and the next one the customer visited.

WebTrends Training for Technical Professionals – Day 3: Advanced Reporting

Today is the third day of my WebTrends training and the first day of the advanced reporting training.

We will focus on :

– visitor history table usage in WebTrends
– configuring campaign tracking
– defining new dimensions, measures, filters to create custom reports, etc.

Clearing the confusion of hit metrics and visit metrics:
– Log file with 1 visit consisting of 3 page views, where each page view was to a different content group, nevertheless there is only one visit (although it was a visit to three different content groups)

Visitor History

To use visitor history you need to use persistent cookie, session cookie or persistent authentication (user login). By default, visitor history is not enabled and can be enabled by clickin on the Visitor History tab of the profile overview. If configurning the profile to export the visitor history, it would get stored in a specified directory on a daily, weekly or monthly basis.

Visitor history can include the following behavior categories:

– Campaign History
– Search Engine History
– Most Recent Search Engine Duration
– Visit History
– Purchase History
– Custom Visitor Segmentation
– Content Group Unique Visitor Tracking – new to 7.5
– Page of Interest Unique Visitor Tracking – new to 7.5

You can invent your own segmentation through tagging. Unique visitor would translate visits to visitors. The precision of unique visitor tracking is based on the cookie. The same visitor using different browsers from the same PC would be counted as two different unique visitors.

Since visitor history might grow, trimming can be handy. Visitor history trimming is based on several criteria — number of visits, lifetime value, most recent visit, most recent purchase, etc.

Visitor history gets updated based on the profile. If you want retroactive data you need to reanalyze.

Visitor history is important as a tool for visitor retention. It is cheaper to keep a customer happy than to get a new customer.

In version 8, Marketing Lab would enable the mashing of web log data and additional customer relationship information.

Visitor history can be used to understand visitor behavior:
– on which visitors should you spend marketing dollars?
– should some visitors be contacted more often than others?
– which campaigns generate the best visitors, etc?

Visitor History Table

– Visitor cookie ID (based on cookie sent by SmartSource or your own cookie parameter)
– Contains attributes on a per visitor basis:
– number of hits
– number of visits
– time of first visits
– time of last visits, etc.

Visitors tracked by IP address/User Agent will not be stored in the Visitor History Table.

Consider if you need to use visitor history because it slows down reporting and takes hard drive space. Try it by turning it on for a couple of profiles and see if you would get value from it.

Measuring Campaign Performance

To track paid (vs. organic search engine keywords), use the following tags:
WT.mc_ID=”marketing campaign code”&WT.srch=1

WT.srch=1 would automatically put the data into the paid-search category reports.

When tracking campaigns from multiple channels, the easiest is to have unique campaign ID. Wrap the campaign ID translation table which is limited to 26 columns (in custom reports). Why 26, because each column in Excel is associated with a letter. Possible values are:
– campaignID
– description
– offer
– creative type
– channel

The canned WebTrends campaign report uses 11 columns in the translation table. For the columns that are not used, you can put commas in the columns with no data.

The prefered method for tracking each campaign in WebTrends is to use the WT.mc_id parameter.

META tags:
1. Unique landing page (tagged page)
2. Pass WT.mc_id as part of the page URL

META tag syntax:

The META tags have to be above the SmartSource javascript code.
Campaign ID is a visit parameter.

Custom Reports

Custom Reports consist of one table and its associated graphs. Once created, reports are automatically added to:
– custom reports main screen
– custom report tab on profile
– reports pick list in Dashboards
– report pick list in Templates

Custom Reports can have a primary and secondary dimension, i.e. domains and URL with Query String. You can have up to 20 measures.

To create a Custom Report:

  • Define the components:
    1. Data sources first
    2. Dimensions
    3. Measures
    4. Filters
  • Custom Report
  • Add the Custom Repor to profile

Translation files are easy but the downside is, they get stale. It is OK to sue translation table for things that would not change, for product categories for example, but for more dynamic information, like pricing, not.

Data Sources are associated with Dimensions and Measures.

When creating a Custom Report, make sure to check the box for “Exclude traffic where this dimension was unspecified”, otherwise the report would include those that did not mean the criteria for the dimension.

In a basic report you cannot turn on the Custom Report. Make sure to select advanced profile for custom reports.

Important Tip: To make sure that the Custom Reports show in every report, go to Report Designer > Templates and select to edit the Enterprise Complete View report. Click the Content tab and then click on the Enterprise Complete View V7.5 item on the left navigation. Add a new report and select from the Custom Reports list the following one: “Add a folder to my template that will include all current and future Custom Reports.” When you click on it, you will see “Auto-Populated at Report Time”.

WebTrends Training for Technical Professionals – Day 2

Visitor History is a nice new feature of version 7.5 and up of WebTrends. It requires strong authentication, i.e. session cookies, persistent cookies but not IP address & user agent. Don’t turn it on if using IP address for authentication because it does make the profile larger and slows them down.

Running three segments of large amounts of data of 10 days each would run generally faster than 10 segments of 3 days each. But make sure you have the RAM and the hard drive in place before running large amounts of traffic.

Page files and download files can be defined on per profile basis. For example Flash files can be both downloads and page views. By default they would be under page views but it makes sense to configure the profile and assign swf (Flash files) as download files.

In Reports > Reports Headers you can customize the logos of the banner and replace the WebTrends Report Banner with custom images. The middle image is a spacer and it tiles it, so if the logo is large, do not use the middle image.

In Reports / Reports Types, turning all five reports types make sense — Yearly, Quarterly, Monthly, Weekly, Daily. If the reports start on June 1st, clicking on Quarterly, does not mean it will generate Quarterly reports for the first two quarters.

Parent-child profiles split your web traffic based on configurable rules, and create multiple report sets. This is useful for service providers and large enterprises. For example, a split based on domain can create a separate report for each domain.

When WebTrends analysis a log file it looks at the first hit, last hit and file size to compare and check whether it had run a report already.

Have a log file directory which keeps the recent log files.
Current directory for storing 1 – 30 days.
Another directory for storing 1 month – 6 months.
Another directory for storing older than 6 months.

Another approach is to use date macros in the log file paths:
ex%date-1%%yy%%mm%%dd%.log – for yesterday’s files
ex%date-2%%yy%%mm%%dd%.log – for 2 days ago
ex%date-3%%yy%%mm%%dd%.log – for 3 days ago

This always checks the last three days.

For special request profiles, the recommendation is to use a separate data source.

New to WebTrends 7.5 is the ability to schedule the download of reports. The smart thing to do is to design the template for the reports they need to see on a regular basis. In Scheduler > Scheduled Events > Schedule Report > Select profile, etc. Select a custom data range for the reports to be emailed.

Clear Analysis Data is different from Reanalyze. Make sure to have the new data source or editing the data source so reanalyzing would be meaningful.

To track the last three days of the SDC log files, use the following macro where sdc is the unique sdc identifier (usually 20 characters) if SDC stores logs on a daily basis:
sdc_%date-1%%mmm%_%dd%_%yyyy%.log – for yesterday’s log file
sdc_%date-2%%mmm%_%dd%_%yyyy%.log – for the day before yesterday
sdc_%date-3%%mmm%_%dd%_%yyyy%.log – for two days before yesterday

Session tracking: visits vs. visitors

Visit — all hits for any one visit
Visitor – anyone entering the web site

Sessionalizing by persistent cookie is considered more reliable than IP address & user agent. However, the same person visiting from work and home would be counted as two separate visitors.

To setup session analysis go to Web Analysis > Session Tracking. For example to track by parameter, we would use Parameter Match. WT scrolls you through grayed-out methods that are not used. Then in track sessions by parameter you would specify the parameter name.

If switching from 3rd party to 1st party cookie, you must redeploy the new tag to all your pages. Identify fallback method, most likely IP address and user agent.

Filters: All profiles by default come with all include filters. There are include and exclude filters. If there is not filter, all activity is included. There is no limit to the number of filters per profile.

To exclude spiders, create a new hit filter. From the browser drop down menu, select All Spiders and Robots [allspiders] which would include the browser.ini file which includes over 500 spiders.

Filters work from now on.

To analyze all campaigns, listing the Campaign ID in the profile would give you only number of visits but not a drill-down report. For “butterfly”-type drill-down report, create a separate profile filtering all entries from ?WT.cm_ID=*

Creating an include filter for the custom 404 files only would give a list of most failing pages which would be very valuable for web developers.

Multi-homed domain log file includes multiple data sources.

Remember, OR is more, AND is less.

If you are to email business users the reports they want to see, find out what they want and put it in a template.

Notes from the WebTrends Training for Technical Professionals

I am attending WebTrends classes for technical professionals and will share some of my notes taken durnig the classes. The notes will not be complete nor very eloquent but will capture the most important information during the training.

Notes from day 1: WebTrends Essentials for Technical Professionals

Content groups can be defined two ways: through content words contained in URLs or through page tagging.
The page tag for content groups is WT.cg_n:
Set meta “WT.cg_n”=content “keyword” – for content groups (URL+WT.cg_n=”keyword”)
Set meta “WT.cg_s”=content “subkeyword” – for sub content groups

WebTrends by default truncates query parameters.

The report graphs and dashboards require the installation of Java which would require one to be administrator on their PC.

Daily reports saves hourly traffic. It makes sense to keep the daily reports for not more than 3 – 6 months.

Custom calendar range used daily reports. Comparative calendar range uses weekly reports.

/products/prices.asp
/products/prices.asp?ID=4
/products/prices.asp?ID=4&ID2=801350

WebTrends would look at it as three different page views although it is the same page. You would need to build either custom report or parameter analysis to avoid the URL truncation.

You can use translation table mapping query IDs to titles of the articles published.

Path analysis can give you view into 20 steps before and 20 steps after a defined URL. Scenario analysis would have the actual URLs and query parameters for each step and view the path analysis with query parameters.

In a form with multiple fields, you can tag each field to create a scenario analysis with one page only but multiple fields using the on-click event.

In the WebTrends Resource Center, very valuable information is in “Understanding Data Collection” under Implementation > “WebTrends Query Parameters” and “Tracking Downloads and Form Completions”.

With SmartSource Data Collection information about PDF downloads is not logged. You need to do on-click event to track PDF download.

To track Google paid search results, use the following tags:

WT.mc_ID=10001&WT.srch=1 where 10001 is the marketing campaign ID and WT.srch=1 tells WebTrends this is paid search and automatically updates paid search terms and phrases.

WebTrends cookies by default are set as persistent cookies with 10 year expiration. Cookies are the primary identifier for visitor history.

WebTrends_ID cookie contains customer IP address, time in milliseconds since 1974 and duration.

Rejection for 1st party cookies are around 12%, 3rd party cookies around 25%. Home users of AOL readily accept cookies in IE but Firefox does not accept 3rd party cookies by default.

Thirty minutes of inactivity ends a user session. Multiple users through a corporate firewall would generate a single session as long as there is less than 30 min between visitors. AOL can overstate page views by assigning randomly a shared IP address.

Usually traffic measurements based on cookies is better by 20 – 30%.

1st party cookies means that my site is issuing the cookie.

Set in the profile to save 180 daily reports and delete the older reports — rolling 180 daily reports. The feature to remove old files is disable by default.

Deleted Data Cleanup is under Scheduler > Scheduled Events. To enable the feature, unclick the check box for disabling the feature.

Keep quarterly and yearly reports for ever. Keep daily reports only for the last 6 months.

Standard size for a profile is about 1 MB per day. Recommendation no more than 30 custom reports per profile.

Version 8 will allow you to define what is an event. You can define what events you want to scoop out of the log file and automatically export into Oracle or SQL server and run queries there.

The proprietary database is non-querable; in version 8 you can export it for queries.

Comparative calendar view would compare week vs week or month vs month — likewise ranges.

Important: For custom calendar range the dates need to be contiguous. Visitors summary would show incorrect visitors summary in a custom calendar range because it sums up the daily visitors for each day into a total numver for the custom calendar range. You would need to reannalyze the data for the custom period if you want the total number of visitors.

Webtrends tracks annual visitors based on calendar year. So a visitor in December 2005 and a visitor in January 2006 would be consider as different even if they use the same IP address, same browser, same cookie.

Custom comparative view would allow you to compare two custom periods.

All custom reports add only dailies.

A template is a collection of reports — i.e. executive, marketer’s and webmaster’s reports. The dashboard always starts with the default template but I can choose a different template if given access to more than one templates.

Dashboards are not exportable. Reports are exportable into Word, Excel, PDF, or CSV format. You need to install the WebTrends Report Exporter.

Recommendation: Allow sharing bookmarks only among people who are knowledgable about WebTrends to avoid having the bookmarks list be longer than the content list.

You can bookmark queries.

By default of SDC rotates log files hourly, meaning it creates 24 files per day; the recommendation is to change to daily.

Create unique data source per server but use more than once data source per profile. It is better to read the SDC log files from the WebTrends server rather than from the SDC server.

WT.sdl is used exclusively to track multiple links from the same page, i.e. WT.sdl=top would be added to the top link and WT.sdl=bottom would be added to the bottom link.

Web analytics consolidation – would NetTracker stay on track?

This week, one of the leaders of enterprise marketing management solutions, Unica Corporation, acquired Sane Solutions, the maker of NetTracker web analytics software, for $26M – $28M. As quoted in TechTarget, Elana Anderson, VP and research director at Forrester Research, stated “For the online, interactive buying sector, Web analytics is the fundamental repository. If I’m Omniture, Coremetrics or Web Side Story, I’m thinking, ‘now Unica has moved into my turf.'”

This happens only weeks after WebSiteStory announced its purchase of Visual Sciences, bringing two of the leadings players in the web analytics market place together, while Google acquired the blog analytics solution MeasureMap,following its purchase of Urchin Software (now offered for free as Google Analytics).

This flurry of merger & acquisition activities in the web analytics market space, no doubt, confirms the growing need to maximize the ROI of web marketing and the role web analytics providers should play in solving this problem, but the approaches of certain vendors require some analysis:

Some of these acquisitions are pretty obvious in their business sense — Google’s purchase of Urchin is targeting the AdSense subscribers and MeasureMap will most likely empower users of Blogger, while the merger of WebSideStory with Visual Sciences aims to create a power player covering a wide range of businesses, from WebSideStory’s stronghold in medium-sized e-commerce sites to Visual Sciences’ footprint in the intelligence community and larger corporations.

The fate of NetTracker within Unica is less predictable, though. It almost begs comparison with another acquisition of a web analytics maker – the venerable NetGenesis which was among the traffic analysis pioneers in the early days of the web, by a data mining vendor, SPSS. As a January 2006 report by Forrester Research reasoned, “Today, the dilutive effect of the SPSS acquisition is fully evident: NetGenesis has positioned itself as the firm to turn to for “predictive Web analytics.” Having abandoned the more mundane aspects of Web analytics to the likes of Omniture, WebSideStory, and WebTrends, the vendor is hanging its future on this esoteric niche that few customers are ready for.”

Surely, as businesses learn to gain insight from web traffic and improve on their business intelligence capabilities, the trend will be towards vendors offering bundled solutions, but there is so often a gap between wishful marketing strategy and the ability of clients to absorb too much information at a time, that I wonder whether NetTracker will be sidetracked by the other Unica offerings, not unlike the way NetGenesis faded away within data miner SPSS.

How much is search engine marketing worth to you?

Nielsen/NetRatings updated its report on the search engines’ market share in the U.S. [PDF]:

  • Google – 48.2% (in January 2006) vs. 47.1% (in January 2005)
  • Yahoo! – 22.2% (in January 2006) vs. 21.2% (in January 2005)
  • MSN Search – 12.8% (in January 2006) vs. 12.8% (in January 2005)

comScore Networks, on the other hand, measured lower but growing market share for Google in January 2006:

  • Google – 41.4% (in January 2006) vs. 35.1% (in January 2005)
  • Yahoo! – 28.7% (in January 2006) vs. 31.8% (in January 2005)
  • MSN Search – 13.7% (in January 2006) vs. 16.0% (in January 2005)
  • AOL – 7.9% (in January 2006) vs. 9.6% (in January 2005)
  • Ask.com – 5.6% (in January 2006) vs. 5.1% (in January 2005)

These percentages are based on an estimate that 85% of the total U.S. internet user base, or 146 million people, visited a search engine at least once in January 2006.

While the shares of the main search engines haven’t changed much since last year, the total search traffic in January 2006 has grown by 39% (according to Nielsen/Netratings) or by 11% (according to comScore) since a year ago. Whether you use the Nielsen/Netratings’ numbers (5.7 billion searches) or the comScore’s ones (5.48 billion searches in a single month), an average of 3 searches per user on a typical day (based on the 60 million daily search engine users in the U.S., estimate by the Pew Internet & American Life Project) begs the question: do search engine marketers overpay for their traffic, and if so, what is there to do?

The growing importance of search engines as a marketing channel would explain the growth of search engine marketing (SEM) spending only partially. When you compare the SEM annual spending growth, estimated at 277% by MarketingSherpa at a recent web seminar, to the 39% growth of search engine traffic, it becomes obvious that as many as the search engine users are they are not enough — too many of the search engine marketers are competing, and overpaying, for too few eyeballs.

While the fear of the competition might be a big factor for the keyword bidding craze, my guess is that the biggest reason why marketers so willingly part with their dollars is because they do not measure the return on their marketing investments. This is where the importance of web analytics and organic search engine optimization come. One enables the other. Unless you grow (organically), you cannot sustain your business. Unless you measure, you cannot manage. Unless you manage, you spend your online marketing budget at your own risk.