June Dershewitz on Web Analytics

Functionalism Basics

In this post I'll describe the basic building blocks of functionalism. As you follow along, think about how you can apply these techniques to your own web site measurement and optimization projects.

Functionalism Defined

Functionalism is a methodology for web measurement. It includes:

• A way to understand what each part of a web site is supposed to accomplish.
• A specific set of measurements to determine how well the function is being performed.

Here's the cool part - functionalism is appropriate for any type of site: commerce, lead gen, media, branding, support, etc. 

Back when I was an on-staff web analyst at a large company with a very diverse collection of web sites - none of which had direct and obvious ties to revenue - I used to joke that, "I wish we just sold shoes." No offense to analysts who measure commerce, but you people have it easy. If I had known about functionalism back then, my group's web analytics projects would have been far more structured and consistent than, sadly, they were.  

So much for my flashback. Here in the present, functionalism can give you and your web analytics team a framework for measuring every component of every diverse web site that you own. Not only can it enforce business-wide consistency in basic measurement, it can also put some much-needed structure around the process of doing web analytics. 

The Approach

Functionalism has 2 major components:

• A set of page types that map to content on your site 
• A set of KPIs that map to each of the page types

Once these components are in place you can take any page on your site, give it a page type classification, then measure how well it's performing using the KPIs associated with that page type. In this way, functionalism provides a method for deciding when and how to use any KPI that you define.

Page Type Classification

Page types are designed to capture the natural function of a page. For instance, you'd call a page a "Router" if its function is to move the visitor to specific places on the site, and you'd call a page a "Convincer" if its function is to convince someone to buy your product or service.

As I classify pages I often refer back to a definitive list of the page types in my repertoire. It looks like this:

• Engagers: Pages whose primary purpose is to interest the visitor and get them to do something/anything on the site.
  
• Routers: Pages whose function is to move visitors into specific places on the site. One of the differences between an Engager and a Router page is that the latter is built with the expectation that the visitor has come to the page looking for a particular type of information/service. Search should often be treated as a special class of Router page.
  
• Convincers: Pages whose function is to “sell” the visitor on a product or service.
  
• Explainers: Pages whose job is to help the visitor understand some aspect of a product or service.
  
• Informers: Pages whose primary objective is to provide basic news and information about a product or industry. This differs from Explainers in that content is not specific to your own particular product or service.
  
• Billboards: Pages providing various content to visitors, but whose primary business objective is to display third-party advertisements on a Cost-per-Impression basis.
  
• Sponsors: Pages or series of pages which are themselves revenue-producing, whose content is mostly provided by a third-party who pays for inclusion within your website.
  
• Closers: Pages that are supposed to get visitors to enter a conversion process.
  
• Converters: Pages that are part of whatever is necessary to gather information/agreements and get a finished lead/sale/transaction.
  
• Re-Assurers: Pages built to re-assure the visitor about some potentially problematic issue or concern (privacy policies are a common example).
  
• Tools: Pages that are designed to collect or provide information as part of a nonsales process (like checking an account status or finding a location). There are many different kinds of tools and for a tool-rich site tool pages need to be subdivided into more granular types.
  
• Completers: Thank you pages – designed to signal the completion of a process and – in some cases – drive to additional engagement.

This is the Cliff Notes version of the full page type listing that you can find on page 19 and following in Gary Angel's freely-available Functionalism White Paper.

Make a list like this for yourself. Keep in mind that, as long as you have defined the right page types, every page on your site can be classified. Don't feel bound by the list I've provided here - it's totally fine to create your own page types.

Page Types & Associated KPIs

Within the framework of functionalism, each page type is associated with one or more KPIs. In this way, the measure of a page's effectiveness is tailored to the function it performs.

For instance, a "Router" page will be associated with KPIs that measure how well it moves visitors to specific places on the site. Typical router KPIs will measure the "next page" behavior for that page - the percent of clicks on body content, percent of clicks on navigational links, exit rate, etc.

Similarly, a "Convincer" page will be associated with KPIs that measure how effectively it convinces a visitor to buy. The most common convincer KPI is percent of next page clicks on call-to-action links - you can call this "Conversion Rate" as long as your business audience is clear on the definition.

Here's a sample mapping:

  

As with page types, the KPIs can be customized to suit your needs. Pick KPIs that 1) make sense as a measure of functional performance, 2) are meaningful to your business audience, and 3) are actually obtainable from the web analytics tool that you use.

KPIs are tool-neutral and designed to be implemented in a range of solutions including Omniture, Webtrends, Google Analytics or any other tool. So as long as you've got a basic, valid visitor behavior tracking in place on your web site, you can use functionalism today without having to customize your implementation.

That's the basic setup. Once you've defined how you'll measure each page, you can use it to compare pages of one type to one another, to assess the effectiveness of redesigns, and to provide focus for ongoing optimization. I'll cover examples in future posts.

Functionalism is NOT ...

Now that I've outlined what functionalism is, here's a quick note on what it's not. Functionalism is not designed to provide any subjective analysis of pages: It includes no subjective design rules, no business specific classifications, no way to “score” pages subjectively. 

It simply provides a framework for performance measurement.

[This post is part of my series on Functionalism. In my next installment I will walk you through a how-to.]

Functionalism Works Because People "Get It"

Back in one of Gary Angel's original blog posts on functionalism, he wrote this about why functionalism works so well compared with other web behavioral analysis techniques:

"When we explained [Functionalism] to channel marketers and site designers, they got it immediately. We could see the lights go on. 'Ah – I get it. This page is supposed to move people here and it isn’t doing the job and I can even understand how you proved it isn’t doing the job.'"

In short, functionalism works because people "get it."

In web analytics - a discipline often plagued by complexities and misunderstandings and so many "huh?" looks - it's refreshing to find something that people "get" so readily. As we strive to popularize and evangelize web analytics it really helps to have such palatable material. Try it, you'll like it.

So obviously you can see that I'm a convert. Here's how I got my start with functionalism:

Not long after I joined Semphonic, I took on a client project that involved a functional website analysis. I set out by reading Gary's white paper and reviewing some other previous projects and talking to my new colleagues about it. And my initial impression was, as in the quote at the beginning of this post, "Yeah, I get it. That makes total sense."

I felt as if I had been doing this type of analysis all along, but this just helped put a name and a structure around the activity.

These days, when I explain functionalism to new colleagues and new clients, I often say that the analysis part is very very simple, it's the methodology for the analysis that benefits from some study. Functionalism is a methodology for web measurement. It takes a bit of discipline to apply this methodology to an otherwise unstructured analysis task, but it pays off in the end because it gives a common language for all analysts and also all intended recipients of the analysis.

Are YOU ready to get started? Great. In the next post I'll explain the basics of functionalism.

[This post is part of my series on Functionalism.]

Functionalism Blog Post Series

How do you actually do web analysis?

I'm not speaking rhetorically here. I mean, if you're a web analyst, and you set out to make sense of the web site behavioral data that we all collect in web analytics applications like Omniture SiteCatalyst and Webtrends and Google Analytics, how do you frame a typical analysis project?

When I do web analysis I often turn to functionalism.

The concept of functionalism was developed several years ago by Gary Angel, Semphonic's founder and CTO. In essence, it is a framework for doing web site behavioral analysis. First, web pages are categorized into types depending on their function, and then specific KPIs are applied to each page type. I'll explain more later.

Although functionalism was Gary's "baby," over the past few years it has become the common language we all speak at Semphonic. It is now one of our web analytics consulting product offerings, and at any given time we've got someone in the office doing functional analysis. We're spreading the word through training, too; I recently taught a class on functionalism at the X Change 2009 Think Tank.

Since I believe functionalism is so useful for web analysts, I have decided to write my own blog post series on it.

I will begin by telling you how I got started with functional analysis and explaining why I think it's worth the effort. In subsequent posts I'll lay out the basics of functionalism, give a number of examples, and describe how you can run a functional analysis project of your own. I'll file everything in the Functionalism category on this blog.

As background material I recommend reading Semphonic's white paper on functionalism, as well as Gary's own blog series on the topic:

• Functionalism White Paper (pdf)
• Gary Angel's Blog: Functionalism and Web Analytics Part I
• Gary Angel's Blog: Functionalism and Web Analytics Part II
• Gary Angel's Blog: Functionalism and Web Analytics Part III
• Gary Angel's Blog: Functionalism and Web Analytics Part IV

 

Mobile Analytics Whitepaper: Written by Me, Available Free

I'm pleased to announce that - as a joint effort between Semphonic and our partner, Tealeaf - I have published a whitepaper on mobile analytics and customer experience management (CEM). You can get the whitepaper at Tealeaf's site or write to me directly (june@semphonic.com) if you want to know more.

Working with Tealeaf has given me a new appreciation for the value of CEM, and I believe they've got a very unique offering when it comes to mobile web usage analysis. So have a read through the whitepaper. I welcome your feedback either privately or as comments on this blog.

Book Review: Click by Bill Tancer

I love data. No, no, I mean it. When I saw Bill Tancer speak at a conference a little over a year ago he closed with that sentence, in fact a clever plug for his corporate blog: ilovedata.com. I gladly took the bait. Recently, when I heard that Tancer had published a book about his beloved data, I snapped that up, too.

The book is Click: What Millions of People Are Doing Online and Why it Matters. It is, chapter by chapter, a collection of analyses focused on a single data set, namely the search and clickstream behavior of 10 million US-based internet users. 

On the surface Click has genuine pop culture appeal, with coverage of topics like porn and fashion and phobias and rock bands. In that respect it's readable by anyone who uses the internet and wonders what we - the collective "we" - are doing online. At the same time I believe there's extra value for those of us who work with web activity data for a living. 

As an analyst there is nothing quite so satisfying as taking some data, asking and refining and asking questions, turning the data over and over until interesting observations pop out. Tancer writes about his observations, but he also writes about the processes leading up to his observations. In no way is Click meant to be an instruction manual for data analysts, and yet it does a pretty good job of describing the act of analysis. If you want to know what it's like to be a data analyst, read this book.

Some reviews I've read assert that Click is a plug for Hitwise and its own slice of competitive intelligence data. I disagree. Taking a step back, I think that smart analysts can draw from the book and relate it to whatever data they happen to have at hand. In particular, site-side web analytics data from tools like Omniture and Google Analytics, while offering a fundamentally different view of the internet, can still be used effectively to uncover trends in search behavior a la Tancer's approach. Simply ask and refine and ask questions, then present your observations with the same enthusiasm that Tancer does in Click.

Now that I've written this review I can hand off the book to my fellow Semphonic data analysts.  There's a waiting list. We all love data.

The Dreaded 404 Not Found: Three Ways to Find Broken Links

Every so often I get a request along the lines of, "Hey web analyst, run me a broken links report." Although simple enough, this task is not without its nuances. Here are 3 tactics you can use to find broken links on your own site when you (inevitably) get asked:

1) Set forth a spider

You may be the resident expert in whatever web analytics tool your company uses, and when someone asks you about broken links they may expect that you'll use your web analytics tool to find broken links for them. However, it's not your only choice! Spiders are worth considering for this task, as well; they'll identify a slightly different set of broken links than you get from your web analytics tool.

Here's how I like to think about the difference: web analytics tools tell you which URLs visitors actually try and fail to view. Spiders tell you which URLs throw errors regardless of whether any visitor clicks the link or not, and regardless of whether the target URL lives on your site or elsewhere on the internet.

Newbie explanation: spiders are software programs that crawl pages on your site and scan for certain things like broken links. If your site employs a lot of JavaScript and forms and Flash and web 2.0 whiz-bang, you'll need to find a spider that can handle the complexity. There are basic free tools (like the W3C Link Checker) and fancy not-so-free tools (like this one from Accenture) and plenty of options in between.  If you haven't done so already, go test-drive a spider.

2) Analyze your logs

Back in the old days, when nearly every web analyst dealt with log files, it was actually quite easy to get a broken link report because it comes standard with log-processing solutions. These days, with so many of us using JavaScript tags alone for data collection, error tracking takes a bit of customization that you must initiate.  

If your web analytics tool uses log files, now is the time to locate your broken link report. If your tool uses JavaScript tags, it's worth checking to see if broken link reporting has been set up. If, for whatever reason, you don't have broken link reporting yet, there's still hope! You can still get something useful from your raw logs.

Normally I'd say don't mess with your logs - it's a slippery slope and before you know it you'll find yourself writing your own web analytics application from scratch. Don't go there! However, logs can actually be useful for error reporting, at least as a once-off project. The basic information about a broken link is contained within a single row, so there's no need to sessionalize multiple rows into visits.

Here are a few guidelines for log handling, should you have the need. Since log files are quite large your task will be easier if you pick a fairly short, recent date range for analysis. Next, you (or your sys admin) can filter down the logs so you just get the rows with status code 404. Pop your data into Excel (it fits in Excel, right?) and make sure you can identify what each column contains. The columns you'll be most interested in are requested URL (the broken link itself) and referring URL (the page on which the broken link appeared, if applicable). Finally, make a pivot table and identify the most common broken links and their sources.

3) Have a custom 404 page, and tag it

Allow me to demonstrate how to tell if any site has a custom error page: http://june.typepad.com/foo Oh no, a broken link! Now I know what my 404 page looks like, and by viewing source I can see if it's got a web analytics tag on it (as of this writing it does not). Try this on your own site. It will work like a charm unless you actually have a page called /foo.

If you discover that you haven't got a custom 404 page, it's time to make one - it's simply good form.  For inspiration, here's a practical and amusing post on how to create an effective error page.

As an extra step for those who use a tag-based web analytics tool, you will need to make sure your custom error page is properly tagged.  The specifics will depend on the tool you use, so go to your solution's document library and read the fabulous manual.  In the end you will get error reporting that's just as good as log file solutions, and a lot less effort than parsing raw logs.

Now, after trying out the 3 tactics mentioned here, take all of the problems you've unearthed and submit them as bugs. It will take far longer to fix the errors than it did for you to find them in the first place.  Mission accomplished!

Photo credit: Meteorry

Web Analytics Data Reconciliation How-To Guide

I suspect that most experienced web analysts have done at least one data reconciliation project during the course of their tenure.  For something so common, however, it rarely gets discussed. 

Sure, it's not sexy like Angelina Jolie, but even Plain Jane likes a little attention now and then.

Data reconciliation is an important foundational activity because, when done well, it will inspire people to have confidence in the data that you share with them. Data quality will never be perfect, but it should be good enough for everyone to feel that they can make sound business decisions based on what's available.

Enough pep talk.  If you're on the brink of your first data reconciliation project, here's what to do:

1) Identify your two data sources

The need for data reconciliation arises when you have two separate systems that provide similar sets of data.  One of these sources - let's call it "Primary Source" - will necessarily be your standard web analytics application.  The other one - let's call it "Secondary Source" - can be one of several things, namely:

• An upstream system, like campaigns (banner, search, email)
• A downstream system, like commerce or downloads or form submissions
• A parallel system, such as when you migrate from one web analytics tool to another.  In this case I'd advise you to break your project into smaller chunks according to individual reports you care to reconcile.

2) Learn how your primary source gets collected

Read the documentation and talk to your internal tech team.  Be clear on the scope of the data you're collecting - ie exactly which pages are tagged, or exactly which log files are processed.  If you're using page tags, know whether the tag is placed at the top or the bottom of the page (this will affect when the tag fires, which in turn affects the level of data loss to some extent).  Make note of any special filters, transformations or business logic used here.

3) Learn how your secondary source gets collected

If your secondary source is a parallel web analytics system, repeat the process you followed in step 2, above. 

If it's an upstream system you're stuck with whatever documentation and lore you can glean regarding how that works. 

If it's a downstream system you'll need to identify the group within your business that owns that system, then grill them on how they do data collection and how they transform the data into the metric you're trying to reconcile.  There's a lot of variability here, especially if your downstream system is homegrown, so be sure to do a thorough investigation.  As in step 2, make note of any special filters, transformations or business logic used here.

4) Compare data sets

Pick a sensible date range and granularity level, then pull corresponding data from both sources.  A good default would be daily totals for a month.  If you're dealing with really high volume you may want to isolate a subset of your data based on some attribute that you can reliably pull from both sources, like a single URL (for downloads) or a single product (for commerce).

Now put your data sets side by side in Excel and calculate the delta.  Compare the trends over time and see if you can explain the differences.  Ask yourself, are you comfortable with the differences you see?  If not, consider fine-tuning the way you pull data from your primary and/or secondary source in order to account for those differences.

Reality check: you're never going to get a perfect match.  This is a good exercise, but know when to say when. Do not obsess!

5) Document and share your findings

This is the most important step.  Write a report about what you've done and what you've found.  Now go talk to people - give a verbal presentation of findings to your web analytics colleagues and your concerned data stakeholders. 

At this point you should be able to speak with confidence about the differences in the two data sources, and your goal should be to pass this confidence on to the people around you.  Save your report for future reference, as newcomers are likely to ask the questions you've already answered.

6) Plan to revisit if necessary

If reconciliation is part of tool migration, you are now done.  Good work.

If your secondary source is an upstream or downstream system, plan a periodic audit to make sure your findings are still valid.  If your systems are stable you can get away with doing this maybe once a year, but if you have any appreciable changes - like a major site redesign or a shopping cart overhaul - you may wish to do another quick round of reconciliation at that time.

Where to Put Integrated Data: 7 Helpful Questions

Web analytics data integration goes both ways.  When you marry clickstream data with other business data, you can put the combined result either inside or outside your web analytics application.  The trick is, if you can put it either place, how do you decide which place is best? 

Here are 7 questions to consider as you make your decision:

1. Is this a once-off or will you need an ongoing feed?  Say you're working on a deep-dive analysis project, or you're preparing a data set to use for data mining.  You're probably pulling activity from a discrete period of time.  If so, integrate outside your web analytics application, where there's less overhead for a one-time task.  If, on the other hand, you're going to want this integrated data to be available at moments notice for all eternity, you're best off integrating wherever you can most easily automate your feed, which brings me to my next point:


2. How much effort will it take to automate, in vs. out?  Call me lazy or call me practical, sometimes the right answer is the the easiest one (that's Occam's Razor, right?). The major commercial web analytics vendors have built-in integration tools, like Coremetrics Connect and Omniture Genesis.  If the data you need to integrate falls within the realm of what your web analytics application can handle, use the wizard and take a feed in.  If, on the other hand, you want to integrate custom data that's not wizard-able, take a feed out instead - but make sure you've got IT resources to help you automate the load into the destination system.


3. Which analysis tools do your data consumers prefer to use?  Maybe you've got a favorite data visualization application (like Tableau), or predictive modeling software, or another business intelligence tool that people at your company like to use.  Yes?  Then integrate your data in a place where it will be easy to get at using that tool, most likely outside your web analytics application.  If you plan to use Excel you have more of a choice, because most web analytics vendors have Excel plug-ins.  You could integrate within your web analytics application and then feed it to Excel, or, if your data set is small enough, you could integrate by VLOOKUP()-ing right there inside Excel.


4. Are your data consumers already active users of your web analytics application?  If so, you'd be doing them a favor by putting the integrated data where they're most likely to use it.  On the other hand, if they spend all day working with some other business data system, put it there instead.  It could be the factor that determines whether the integrated data ever gets adopted in practice by the people who are expected to use it. 


5. Will you need reporting components that web analytics applications handle especially well, like browser overlay and pathing?  This will depend on whether your web analytics application actually lets you display integrated data in browser overlay and pathing reports.  If so, and if you can imagine actually using these reporting components, try to integrate inside your web analytics application.  Although web analytics applications are not as robust, generally speaking, as other data analysis tools, they manage to do a good job of presenting clickstream-specific data. 


6. Are you hoping to integrate data that can actually be gathered at collection time?  Maybe the extra business data you want to integrate is something you'll be able to assign to a custom variable in your web analytics application at collection time.  If so, you'll be able to integrate without any after-the-fact joining.  If your integration data doesn't surface until further downstream, though, you can't use this approach.
   


7. Do you need to store your integrated data behind the corporate firewall?  This isn't so much a technical issue as a legal one.  If the data you want to integrate involves personally-identifiable information and you're using a hosted web analytics solution, go re-read your site's privacy policy.  Chances are you will need to store the integrated data behind your own corporate firewall.  If you host your own web analytics application on-site you may still be able to integrate inside it, otherwise you'll need to pull a feed out.

So, depending on your situation it's perfectly reasonable to join data in both directions - inside and outside your web analytics application.  Strive to find a solution that's practical, easy, legal, and most likely to make your data analysts happy.

How to Prep for Site Redesign Measurement

I've been blogging for a few months now and it's been really fun, especially when it comes to choosing topics to write about.  So far I've posted about career development, events I attend and host, data integration, user-generated content, ... heck, I've even posted a picture of my family's pet chicken.

Now, in an attempt to say more about what I actually do at Semphonic, I thought I'd tell you about one project I'm currently working on.  Without giving out any identifying details about my client I'd like to share some tidbits that will be generally useful if you find yourself in a similar position.

So I'm doing a site redesign analysis project.  My client has recently made some major structural, interface and content changes to their site, and I'm comparing visitor behavior (and outcomes and sentiment) before and after the  launch of the new site. 

Redesign analysis is something I've done numerous times throughout my career in web analytics.  Simple fact:  sites get redesigned over and over.  Analyzing the impact of redesign is mostly about picking the right things to measure and then coming up with an interesting story to tell project stakeholders.  Let it be clear that I'm talking about sweeping changes to a whole site that happen at the flip of a switch, not incremental changes made by way of testing. 

Sometimes a redesign launches and I - as the measurement person - have looked back and said, "D'oh!  I should have thought of that in advance!"  In the remainder of this post I'll list out 3 ways I prepare for a site redesign measurement project.  Setting up these things ahead of time will allow you to avoid some known pitfalls.

Redesign Prep #1: Take Screenshots

What to do:
Take "before" screenshots of all the major pages on your site and put them aside for safekeeping.  Yes, you may be able to dig archived pages out of a content management system, but screenshots are a more reliable bet.

Let me just say I absolutely love the screen capture utility called SnagIt, especially the scrolling window functionality.  I recently discovered that SnagIt can also preserve links on a web page, which is quite useful. 

Why to do it:
Because you'll need to know what the site looked like prior to redesign, and  you may want to include these screenshots in your analysis presentation.

Redesign Prep #2: Decide on Timing

What to do:
Take out your calendar, pick "pre" and "post" windows of time to analyze, then pick dates a little farther out from that when you'll be able to share your analysis findings. 

It's a bit of a balancing act.  Invariably site owners want to know immediately whether or not their effort was a success, but from the analysis standpoint it's advisable to wait until a sufficient period of time has passed before you draw any conclusions about the impact of redesign. 

In my current project I'm doing a 2-week quick check-in followed by a 1-month post final analysis - that way I'm able to give some immediate feedback while I wait for the data to roll in and then use the longer time period in my final presentation.

Why to do it:
Because being proactive rather than reactive about data analysis is simply good form, plus it gives redesign project stakeholders some dates to look forward to.

Redesign Prep #3: Grab Data

What to do:
Figure out  if there's any historical data you won't be able to get after the site launches.  This varies depending on what tool you're using, but browser overlay and "Next Page" reports are the most commonly affected.   

What I mean is, you may only be able to view the browser overlay for the period of time when your page looks exactly like it does today.  For dates in the past when the page looked different, your browser overlay report may be unintelligible.

Know what's fleeting, then go in and grab data for your "pre" time period(s) while the data is still available.  If, like me, you choose to analyze two different windows of time - a short period and a long period - be sure to collect both of these snapshots.  Be thorough.

Why to do it:
Because you don't want to be faced with gaps in data as you pull together your analysis. 

So, how do you prepare for measuring redesign?  Anything you'd like to add to what I've mentioned here?  I welcome your comments.