June Dershewitz on Web Analytics

Notes from the Tableau Customer Conference

Last week I attended the Tableau Customer Conference in Las Vegas. I'm not going to tell you what I did in Vegas (because, as they say, what happens in Vegas ....). Instead, I thought I'd share my conference notes. Fascinating.

If you fancy yourself a data analyst, you probably know and love Tableau. It's like Excel on steroids. I've used the product off and on since 2006; although I'm not a power user at the moment, there are some pockets of great expertise within my company.

Given my background, I attended the conference with two main objectives: 1) meet analysts who are doing "cool stuff" with data, and 2) figure out where Tableau fits in the broader business intelligence ecosystem. Here's what I found out.

Analysts doing "cool stuff" with data
The show featured case studies from diverse industries, including:

• Intel, HR/staffing group
• Electronic Arts gaming product, operational efficiency and product development
• General Motors, supply chain
• DePaul University, admissions and enrollment
• Securities, real-time decision support for stock traders

Although the subject matter varied quite a bit from group to group, the common thread was a "do it yourself" approach to each initiative and an emphasis on self-service. Often the Tableau administrator was also the subject matter expert for the data at hand, and the output served their business group directly - no heavy process, no data warehousing middleman.

Tableau's position in the business intelligence ecosystem

• Grassroots. Tableau usage definitely seems to be more "grassroots" than "institutional" - analysts love to get quick access to their data, but it does lack the structure that many of us are used to seeing within traditional data warehousing.
• Decentralized. Deployment at some institutions is totally decentralized – groups will take it and use it when they have a need, as opposed to big, centrally-managed BI initiatives.
• Versus other BI tools. Tableau has not totally replaced tools like Microstrategy in most institutions. There's room for both types of tools. Some consider Tableau as a prototyping tool for dashboards that are later built in Microstrategy. I ran into a former colleague who serves as a Microstrategy administrator at a very large company. And yet, there he was at the Tableau conference. No irony. He sees them as complimentary products.
• Presentation. Some practitioners say that Tableau is actually changing the way that they interact with their executives – less static Powerpoint, more exploratory data sharing sessions.
• Beauty and the Beast. Tableau can produce some beautiful output, but there's no accounting for taste. If you create heinously ugly Excel charts, you will probably create only slightly less ugly Tableau output. Believe me, I saw it.
• Again, grassroots. I saw numerous examples of individual contributors leading the way, rather than centralized BI/DW groups. Broader corporate socialization about Tableau often bubbled up from groups that demonstrated success.

Keynote notes

• Tableau leadership and tech teams – showcased new features in Tableau 7. Collectively thumbed nose at traditional BI/DW companies (by, for example, eschewing bloated mission statements).
• Guy Kawasaki – broadly appealing, though his content was certainly not tailored to the subject of the conference. Made a number of references to "Mac vs PC" debate, but perhaps not realizing that Tableau runs on PC only.
• Cory Doctorow – spoke on data privacy. Philosophy a bit at odds with analysts in the audience, but worth surfacing as valid concern.
• Stephen Few - whose talk I missed, but whose contributions to the industry I respect a great deal.

Some technical tidbits

• Hadoop support – new in Tableau 7. As I see it, it's just another data source, but connection allows for some control over data modeling
• Google Analytics support – no direct connection available, must use an intermediate layer like Analytics Canvas

10 Must-Have Analytics Tools for Small Business: June's Picks

I was recently one of several people interviewed for an Inc.com story about Web analytics tools for small business.  My fellow contributors included industry thought leaders such as Bryan Eisenberg and Eric Peterson. The resulting article provided an excellent summary of our collective opinion. I’ve chosen to publish my personal top 10 list here so you can see the full range of my own recommendations.

Taken together, the tools on my personal list cover the full palette of Web analytics: quantitative, qualitative, optimization and competitive analysis. I've also thrown in a couple of tools you can use to measure emerging media such as social and mobile.

Every tool on my list is either low-cost or free.  Think of them as starter tools for each type of measurement category. 

If you’re having trouble getting funding for expensive-but-important tools, try this approach: use low-cost tools to showcase some early successes with limited budget, then win over management and use that as leverage to get funding.  (Credit goes to Avinash Kaushik on this clever solution to the budget issue.)

Without further ado, here are my top picks for broad-coverage, low-cost, high-impact analytics tools for small business.

Category: Quantitative

1) Google Analytics
See where your visitors come from, what they do on your site, and how often they come back. Setting up goals allows you to perform the essential analysis function of tying behavior to outcomes.

2) Crazy Egg
Generate heatmaps that show you exactly where your visitors are clicking.

Category: Qualitative

3) 4Q Survey
A simple 4 question survey that allows you to monitor online task completion.

Category: Optimization

4) Google Website Optimizer
Multivariate and A/B testing tool. Run different versions of content live on your site to determine which one produces the highest conversion rate.

Category:  Competitive

5) Compete
Compare your own site traffic alongside your competitors.

6) Google Insights for Search
Compare search volume trends within your industry.

Category: Emerging Media

7) Bitly
Shorten and share URLs through social media platforms, then track traffic volume over time. Adding a "+" to the end of any bitly URL will allow you to see stats for that link - so you can use it as a competitive tool, too.

8) Facebook Insights
Dashboard that shows usage and demographic info for your Facebook Page.

9) Percent Mobile
Simple tool that allows you to see the percent of visitors to your Web site who use mobile devices.

Bonus Addition

10) Brain Power
You won't get anywhere with these tools unless you actually have someone to use them and interpret results - so make sure you have a dedicated analyst who can feed recommendations back to your business.

How to Get Bounce Rate for Google Website Optimizer A/B Test Variations

Suppose you’re planning to run a simple A/B test on a Web page and you’d like to use bounce rate as your measure of success. Can you do it with Google Website Optimizer (GWO)?

I’ve had this issue crop up a number of times recently, so I decided to research it. Here’s what I found out.

Can I set bounce rate as a conversion goal within GWO?

No. GWO itself does not report bounce rate, nor does it allow bounce rate to be set as a conversion goal. It’s possible to approximate bounce rate as a goal by coding every single link click on every single experiment page as a conversion event, but that gives you exit rate, not bounce rate - plus the setup can take a lot of effort.

Can I get bounce rate for GWO A/B test variations from Google Analytics or Omniture?

Yes. Although bounce rate isn’t reported within GWO, you can get it from Web analytics tools such as Google Analytics and Omniture. Integrating the data turns out to be very easy, and in fact requires zero coding as long as all experiment pages contain your standard Web analytics page tags.  

Here’s why it’s easy: As GWO serves variations in an A/B test, visitors are redirected to unique URLs (indexA.html vs. indexB.html vs. indexC.html, for example). This behavior was contrary to my initial assumption that all visitors got the same URL, as is the case for GWO multivariate tests. Since URLs are unique in an A/B test, you can simply view the pages report within your Web analytics tool and filter on (for example) index*.html to see one row per variation.

Do I need to pass any custom variables from GWO to my Web analytics tool?

No. You don’t need to pass any custom variables to get reporting for an A/B test.

If you're running a multivariate test, on the other hand, you must pass the GWO variation ID to your Web analytics tool as a custom variable, since the URL remains the same for all variations. If you’re interested in this technique, there are good discussion threads on this topic in the GWO support forum.

If I optimize for bounce rate outside of GWO, do I still need to set up a goal within GWO?

Yes, but it can be a simple placeholder. When you create a GWO A/B test you’re required to provide a conversion goal before you can launch. However, if you're planning to optimize for bounce rate or some other metric in your standard Web analytics tool, you can spoof a GWO conversion goal by specifying a dummy page somewhere on your site. In effect, you're simply using GWO to manage the serving of page variations.

To prove that it can be done, here are some screen shots from a test I set up:

Now it's your turn. Got any tips, tricks, admonishments?

How to Compare 2 Lists of URLs Using Excel

Sometimes I need to compare 2 lists of URLs and find out how much they overlap. Which URLs are only in list #1? Which URLs are in list #2? Which URLs are in both lists?

Here’s an example: say I’ve got one list of URLs from my Web analytics tool (like Content>Top Content in Google Analytics) and another list of URLs from a sitemap-generating spider. Comparing the 2 lists lets me see which pages are actually getting traffic on my site versus which pages are reachable by spider. Although most URLs that get traffic are also spiderable, it’s also likely that some URLs get traffic but aren’t spiderable (like thank you pages) and some URLs are spiderable but get zero traffic.

I've developed a quick and easy way to do this kind of comparison using Excel. You can download it here. I’ve included step-by-step instructions for you to follow. There are 8 steps:

1. Paste in your first list of URLs
2. Standardize them
3. Pivot to remove duplicates
4. Paste in your second list of URLs
5. Standardize them
6. Pivot to remove duplicates
7. Concatenate the standardized, deduped lists and add a "magic" code
8. Pivot again to get results

All the details are in my example spreadsheet.

If you're a programmer I’m sure you have a fancier way to compare lists. However, the beauty of my method is that anyone can do it as long as they have a copy of Excel and a basic understanding of pivot tables.

Speaking of pivot tables, if you've never created one I encourage you to learn right this minute. It's something that every single Web analyst should know how to do. If I ever interview you for an analyst job, even an entry-level one, I will quiz you on pivot tables.

What do you think? Do you have an even easier method? And, as a Web analyst, why have you needed to compare lists of URLs?

Preserving Content Hierarchies in a Migration from Webtrends to Omniture

Blog posts are like shoes: some are sexy and some are practical. This post falls on the far side of practical, not unlike orthopedic clogs.

Here’s the premise: Say you’re currently tracking your site with Webtrends and you’re considering a switch to Omniture.  If you’re happy with the content hierarchy logic that you’ve built into your Webtrends implementation, how much of it will be portable to Omniture? Here are the most important factors to consider:

1) Webtrends Profile = Omniture Report Suite

The Webtrends profile and Omniture report suite both represent the top-level segments for a site. Assuming a given Webtrends profile genuinely maps to a visitor segment that requires full reporting it should be recreated as an Omniture report suite.

In some Webtrends implementations I’ve seen (especially log-based systems), administrators will create lots of profiles simply to obtain unique visitor counts for partial slices of populations. If this is the case it may be possible and appropriate to get the required metrics in Omniture without recreating every single profile as a report suite, since Omniture has more unique visitor reporting flexibility. Omniture also offers a free report suite roll-up feature; there’s no direct match for this in Webtrends.

2) Webtrends Directory Report ≈ Omniture Channel

The Webtrends Directory Report lists out visits by first-level directory based on URL structure. The Omniture s.channel variable serves a similar function, although it’s more likely to be based on business rules which may or may not match the literal file path. If the site has a clean URL structure this would be a simple and easy translation.

3) Webtrends Content Groups ≈ Omniture Hierarchy

In Webtrends most of the business-level content hierarchy reporting is managed using content groups. In my experience, ALL usable Webtrends implementations have at least a few content groups defined. The closest analogy in Omniture would be the s.hierN variables.

Keep in mind that Webtrends content groups go 2 levels deep (i.e. content groups and sub-groups) while Omniture hierarchy reporting goes 5 levels deep. Another important difference is that Webtrends has no strict limit on the number of content groups you can create, while a standard Omniture contract includes just 5 hierarchy variables (each with 5 levels). A migration would involve not only a mapping of content groups to hierarchy variables but a reassessment of whether the 2-level structure designed within Webtrends makes business sense now that you've got 5 levels at your disposal.

4) Other Considerations

In general, Omniture implementations tend to be more complex – and more flexible – than Webtrends implementations. Anyone considering a migration should take their Webtrends setup, translate functionality where parallels exist, then look to the additional Omniture features available to them.

What do you think? Am I right? Am I wrong? Am I missing something? I welcome your feedback, especially if it’s practical.

Flickr Stats vs Other Photo Sharing Analytics

Just after the public launch of Flickr Stats back in 2007, I posted a positive review of the tool, musing that it could serve as a good model for other social media measurement systems.

Fast forward to 2010, I am no longer so content with Flickr Stats. I'm convinced we can do better. Perhaps my attitude is influenced by the recent buzz about how anyone can track stats on their Facebook Fan page using Google Analytics or Webtrends. Or perhaps I've just grown tired of the limitations of Flickr Stats.

Is Photobucket any better, or Picasa, or something else? This post serves as a summary of my quest to find the best photo sharing analytics. Here's what I found.

Flickr: A shortcoming and a workaround

For paying Flickr Pro customers, Flickr Stats currently reports a host of metrics including the number of views over time, favorite/comment activity, top content, referring domains/URLs and search keywords.

The thing I used to like best about Flickr Stats was the all-time referrers report, since it provided insight into my photostream's findability over time. Unfortunately, about a year ago Flickr chose to scale back the referrer stats it reported. Referrer information is now only available by day for the last month. Judging from the volume of support forum comments on this subject, I'm not the only Flickr member who's unhappy with this change.

As a workaround, one Flickr member has created an app called F*Stats that scrapes Flickr Stats and stores historic referrer information in a local database. This seems like a complicated solution, but right now it's the best way for Flickr members to retain their referrer info.

[March 3, 2010, 4pm Pacific. Update: Several hours after I published this blog post, Flickr announced the release of their shiny new Flickr Stats API. Also, members can now download CSV files containing all historic referrer data. Ah, timing.]

Photobucket: Easy social media sharing and pretty good stats

Less than a year ago Photobucket launched a stats dashboard for all members, with limited metrics in the free version and more in the Pro version. In terms of functionality, I'd rank the stats dashboard just below Flickr Stats. It does appear to provide referrer information over time, but it only reports referring domains - not referring URLs or search keywords.

On the positive side, the ability to easily share Photobucket content on Twitter, Facebook and other social media sites - and then get referrer stats - is an excellent feature and may be reason enough to use the service.

Picasa: Perhaps Google Analytics integration on the way?

Activity tracking on Google's Picasa photo sharing site is currently very limited - you simply get the number of views of each photo. However, there's quite a bit of user feedback regarding the desire to track Picasa with Google Analytics. In this wish list of most desired Picasa features, Google Analytics integration is one of the top requests. Come on Google, grant our wish.

Other Alternatives: Again, with GA integration

Although I've covered 3 of the most popular photo sharing sites, there are plenty of others to choose from. A quick search turned up 2 less well-known sites that currently allow Google Analytics tagging: SmugMug and PhotoSwarm.

What do you think? Are there better solutions that I have overlooked? Or, are photo sharing stats not worth anyone's time? I welcome your feedback.

3 Great URL Shorteners with Analytics

URL shorteners trim down regular links so they can be shared more easily on Twitter and other social media platforms. My favorite side benefit of URL shorteners is that every link click can be tracked as it's redirected to its final destination.Not all URL shorteners actually offer analytics, though, so if you need tracking you must choose wisely. To help you along, I recommend these 3 URL shorteners with analytics:

1) bit.ly

It's the Twitter default, and therefore very popular. Full-featured analytics are available to whoever creates a short URL and any other person who cares to look. The trick to seeing stats for any bit.ly link: add a "+" sign at the end of the short URL (for example, http://bit.ly/djG9TM becomes http://bit.ly/djG9TM+). Bit.ly also offers a Pro account with an enhanced dashboard for private use by link publishers.

• Best for you if: you want to stick with a popular choice and you don't mind sharing your stats with the world.
• Site: bit.ly
• Screenshot:

 

2) cli.gs

This URL shortener has many of same features as bit.ly, but stats for URLs are not public. In other words, you must be logged in to view your own stats, and you cannot view stats for URLs you did not create. Another differentiator: cli.gs displays geolocation data as a map overlay rather than a chart - nice if you're a "visual" person.

• Best for you if: you need full-featured analytics and privacy.
• Site: cli.gs
• Screenshot:

 

3) ow.ly

This tool takes on the problem from a different angle: the URL shortener and the associated reporting interface are embedded within Hootsuite, a social network management client. It takes a bit of work to set up an account and learn how to use it, but the stats are really well done. You get analytics for individual messages as well as a summary roll-up. Another very useful feature: an optional form helps you append campaign codes to your URLs before you shorten them.

• Best for you if: you're looking a comprehensive system for managing posts OR you want help appending tracking codes as you shorten URLs.
• Site: hootsuite.com
• Screenshot:

 

These 3 tools are my top picks. For a more complete review of URL shorteners (both with and without analytics), see Danny Sullivan's post from April 2009.

Review of All Android Analytics Apps

Out of curiosity as much as necessity, I recently evaluated all of the mobile web analytics applications that run on my Motorola Droid phone. As of January 2010, a search for "analytics" in the Android Market turns up the following apps:

1. Analytics Widget (free)
2. mAnalytics (free/paid)
3. DroidAnalytics (free 24-hour trial/paid)
4. Mobile GA for Android (free)

I have installed and reviewed each of these apps:

Analytics Widget

This very simple 1x1 widget sits on your mobile desktop and displays either page views or visits for the Google Analytics profile of your choice. The default (and only) date range is "today" - but this seems appropriate, since it functions as a ticker. The widget auto-refreshes periodically, or you can click to refresh it on demand. I now click it compulsively whenever I see it.

• I like: Very simple, basic and easy to use.
• Wish it had: A period-over-period percent change, or even a little sparkline to go along with the single number. The fact that I've had 11 page views today (on my lowly blog) is meaningless until I can compare it to past performance. Beyond that I'd like to click through to more detailed data. If I see a spike in traffic I might want to drill down and look at referrers or search terms.
• Bug factor: It force-quit on me a few times but always recovered on its own. There's currently a note on the Android Market page for this app indicating that this is a known problem, so hopefully it will be fixed soon.
• Stars: 4 out of 5
• Verdict: I'm going to keep it on my desktop (so I can continue to refresh it compulsively).

mAnalytics

Full-featured application (no widget) displays 7 top-line metrics from Google Analytics including visits, bounce rate, % new visits and more. It also provides detail reports for referrers and search terms. All data appears in tabular form, plus very basic charts for visit and page view trends. Date range choices include several presets (today, yesterday, last 7 days, last 30 days) as well as the option for a custom date range.

• I like: Just a little detail. It's an appropriate amount of data to display on my small mobile screen without feeling cluttered.
• Wish it had: A widget. Actually I wish I could merge Analytics Widget with mAnalytics so I could see a little overview on my desktop and then click through to detail. Other than that, it would be nice to see more of the reports and the drilldowns that I can get in the fixed web version of Google Analytics. More charts would also be a welcome addition. Turns out that there are more detail reports and more charts in the paid (vs free) version; if I develop a burning desire for these features I will buy the Pro license.
• Bug factor: No issues.
• Stars: 4 out of 5
• Verdict: I'm going to keep it. I might even spring for the Pro version. If another free app turns up with more of my desired functionality, though, I may replace it.

DroidAnalytics

This paid app has a 24-hour trial so you can try before you buy. It includes many of the same metrics and reports as the mAnalytics app, plus a whole lot more. There are basic trend charts for 6 main metrics - page views, visits, visitors, bounce rate, average time on site and pages per visit. Additionally, there are 9 detail reports listing out these 6 main metrics for every row of data. I found all of the data a bit overwhelming, but if you think you need all of this detail on-the-go, it's possible to find what you're looking for in the app.

• I like: As much detail as anyone would want on mobile. The settings are a nice touch; it's possible to customize your view quite a bit. I also like the fact that the developers have posted version notes; it's clear that this app has a good development path ahead of it.
• Wish it had: A slightly better user interface. I got the hang of things eventually, but it took more time than I'd like to figure out the icons and get a handle on what exactly was being displayed.
• Bug factor: I noticed several minor interface glitches. For example, rotation from portrait to landscape does not appear to work, and in one instance the date format changed on its own accord after I viewed (but did not edit) the settings.
• Stars: 3.5 out of 5
• Verdict: Although I liked this app, it's not sufficiently better than the free apps for me to keep it at this time. I am going to check back in a couple of months, though, because the developers are bound to include more features and fixes in future releases.

Mobile GA for Android

The screenshots of this app looked very promising but I was unable to get it working on my Droid. I tried a few times with no luck.

• I like: The product screenshots.
• Wish it had: Better recovery from errors.
• Bug factor: Terminal case.
• Stars: 1 out of 5
• Verdict: I'll try reinstalling it in a few months to see if the problems have been resolved.

Conclusion

I was surprised by the variety in functionality across these applications. For the time being I will keep using the Analytics Widget and mAnalytics in combination with one another; they both contain features I like, with a proper amount of detail for mobile.

It's also interesting (but not surprising) to note that Google Analytics is currently the only web analytics tool with apps in the Market. How long will it take before Omniture and Webtrends apps make their debut? There are already iPhone apps for both.

Speaking of the iPhone, here's a recent post from Lars Johansson in which he reviews all Google Analytics apps for iPhone. Hopefully we'll see more web analytics apps for the Droid in the near future.

[Feb 2, 2010. Update: I've been through a round of troubleshooting with the GA for Android developers, so hopefully I will get to test out and review a working release of that app before too long. Also, I have unearthed an Omniture Android app. It was in the Android Market all along; ironically, it just didn't contain the word "analytics" in the description.]

Advancements in Mobile App Tracking [Or, the Mini-Me of Web 2.0]

When I got into mobile analytics a couple of years ago, I quickly realized that there were several very different ways to interact with (and therefore, to measure) the medium: there's the mobile web, and then there are mobile apps.

Mobile web - even though it's fraught with data collection problems - is easier to come to terms with as a web analyst because it's like the Mini-Me of the existing desktop (or "fixed") web. We know exactly what to expect from the mobile web, it just has a smaller-than-normal screen and lame browsing technology.

Continuing with the Austin Powers analogy, mobile apps are like the Mini-Me of Web 2.0. Since the "Web 2.0" buzzword can mean several things, in this context I am referring to rich internet applications. When RIAs first became popular on the fixed web, analysts had to shift our thinking away from measuring page views to measuring events. The same holds true with mobile apps, where we now have no tiny page views, just tiny events.

When I've written about measuring the mobile web in the past, I've always intentionally stepped around the issue of mobile apps because they seemed like a bit of an edge case. Well, those days are over. With the rise in popularity of the iPhone and similar devices, we see an increased awareness of and interest in mobile apps. You get your new phone and immediately you want to download apps. Back in September 2009 Apple announced that more than 2 billion apps have been downloaded from their app store. Wow.

As a web analyst, this is why I care:

Every time a new mobile app is born, there will come a point when the marketing managers and usability specialists and business executives want to know how customers are interacting with their mobile app. They will then turn to their resident web analyst and ask the classic, "So, how'd we do?" to which the web analyst will likely reply, "Uh ... we're not tracking the app."

The "we're not tracking the app" excuse won't fly much longer. There are many ways to track your mobile app, it's just a matter of realizing that you need to have a plan and convincing your development team to implement it. Unlike tracking a static page on the fixed web, it's not as simple as tacking on a snippet of javascript code. As stated earlier, it's more like tracking a Flash app on the fixed web - you have to use real brainpower to determine which events you consider significant, and then you need to embed the tracking within the app so that the appropriate events are logged as they occur.

In the past I've been involved in Semphonic client projects where we used the Omniture data insertion API to push mobile app activity to SiteCatalyst alongside related fixed web and mobile web activity. Omniture is definitely a viable solution for tracking mobile apps; within the past 6 months they've expanded the list of smartphone platforms whose apps they're able to measure. I have also recently learned about stand-alone mobile app tracking applications like Flurry and Pinch. I welcome your blog comments if you've used these products or others like them.

The big news, though, is that Google Analytics has entered the running as a mobile app measurement solution. Last month GA announced a set of new features, including the ability to measure iPhone and Android mobile apps. GA doesn't cover as many smartphone platforms as Omniture (yet), but then again it, uh, costs a lot less than Omniture. In another recent official GA post highlighting app tracking, they offer up a video clip case study:

The video interviewee says mobile app tracking has led to insight about popular features, which in turn has informed product development and usability initiatives. It's a useful story no matter which tracking tool you happen to use (or plan to use).

Lesson: the minute you get the idea to build a mobile app, take a breath and then think about how you're going to track it. You have a number of options. None are super-simple, but they're all manageable and will meet an important need down the road when your business executives want to know, quantifiably, how customers are using the app. Good luck!

[PS For those of you following my blog post series on functionalism, I will get back on track in my next post. I could not resist this tangent.]

Mobile Analytics Whitepaper: Written by Me, Available Free

I'm pleased to announce that - as a joint effort between Semphonic and our partner, Tealeaf - I have published a whitepaper on mobile analytics and customer experience management (CEM). You can get the whitepaper at Tealeaf's site or write to me directly ([email protected]) if you want to know more.

Working with Tealeaf has given me a new appreciation for the value of CEM, and I believe they've got a very unique offering when it comes to mobile web usage analysis. So have a read through the whitepaper. I welcome your feedback either privately or as comments on this blog.