June Dershewitz on Web Analytics

X Change 2008 News and Deals

Have you heard?  The X Change web analytics conference is coming up in San Francisco on August 17-19, 2008. At X Change you'll share the table with seasoned practitioners, vendors, consultants, bloggers and industry experts without feeling like you're subject to a formal speech or a sales pitch.  You'll all have an equal stake in the conversation, thanks to the small-group discussion "huddle" format.  You won't just be an attendee, you'll be an active participant.

If this sounds like something you'd enjoy, here are a few things you can - and should - do soon:

Get a room

X Change will be held at the Ritz-Carlton in San Francisco.  It's a classy place and the service is phenomenal.  Our discounted room rate of $329 USD per night is only available through June 30th.  If you want this rate you must reserve your room before the end of June; the discount code will be provided when you register for X Change.

Get a discount

Members of the Web Analytics Association are eligible for a 15% discount to X Change.  Use the code "XCWAA15" when you register for the conference.  If you are not a member of the WAA, I most certainly encourage you to join today.  Event discounts are among the many benefits of membership.

Get a free ride

Wish you could come to X Change but lack the funds?  Well, this year we are awarding a scholarship - including conference admission, a room at the Ritz and reasonable travel - to one talented individual, possibly you. Read the scholarship details and get started on your essay.  The conference will be a great place to network, and the scholarship will make an impressive addition to your resume.

If you've got any questions about X Change you can contact me directly at <june@semphonic.com>.  See you there!

Tracking Downloads with Page Tags, Just the Basics

I offer a useful tidbit for those of you who 1) rely on a page-tagging solution for web analytics, and 2) care about tracking downloads.  I've encountered this situation several times as a web analyst, so I thought I'd write up a basic summary of the issue. 

You will find this relevant if you are either a newbie web analyst or an experienced web analyst charged with educating your data consumers.  Here's what you need to know:

Page-tagging

Many popular web analytics tools available today – like Omniture and Google Analytics – rely solely on JavaScript tags for data collection.  Using this method, site owners place a tag on every page they wish to track; when a visitor accesses a tagged page, information is sent back to the web analytics tool.  That, in a nutshell, is page-tagging.

Before you read any further, confirm the method of data collection used on your site.  If you've got something other than page-tagging - such as log files, or network collection, or a hybrid solution - then you can stop reading now.  The issue I'm about to describe only matters if you use page-tagging alone for data collection.

Tracking downloads

Here's the problem with using page-tagging to track downloads: it's not possible to embed a JavaScript tag inside a downloadable file.  Instead, the tag tracks the very moment the visitor clicks a link to get a file.  This counts downloads initiated, not downloads completed. 

Measuring downloads initiated is not wrong, but it is slightly upstream from the spot we'd ideally like to track: successful completion of download.  Marking this slightly-upstream-but-still-valid action will necessarily inflate the number of real downloads, though, since it’s entirely possible to back out of the process before actually downloading the file 100%.  

What it all means

Do not expect accounting-level precision from tag-based download stats.  Take the metric at face value, and make sure everyone who uses the data understands how to interpret it.  If you find yourself needing to compare tag-based downloads with an overlapping source of business data, I invite you to read my data reconciliation how-to guide post.

When I encountered this issue with downloads most recently (last month, in fact), my client opted to put a statement like the following as a footnote in a widely-distributed report that includes download stats:

"Values in this report approximate the number of successfully completed [Product X] downloads. Since no status is returned to the server when a download completes, it's not possible to get an exact figure. Therefore, we count downloads initiated."

So, armed with education and footnotes, you and your fellow analysts should feel confident using the "download initiated" value as a valid marker of site success.

Extra credit

Here's one of my favorite pieces by early photographer Edweard Muybridge.   As progressive snapshots of a single activity, it seems appropriate to include it here.

Image credit, Digital Journalist.

eMetrics Recap in 100 Words

Highlights from eMetrics SF 2008:

Tom Davenport’s keynote was inspiring. He communicates what we’ve known all along – the importance of analytics - straight up to our executives.

Jakob Nielsen’s presentation was captivating. His videos of user testing sessions drew laughter; we can all relate.

Industry Insights Day was 100% worthwhile. Sharp minds; lively conversation.  Similar in spirit to X Change.

Zero newsworthy vendor announcements. Google Analytics unveils “almost” real-time processing. Ho-hum.

WAW was a smashing success. My super co-host Eric Peterson applauds the feat – and fête.

Our Twitter backchannel was a fun experiment.  Let's do it again, Clint.

That's all!
 

 

Speaking, Listening, Partying, Learning and Otherwise Engaged at eMetrics SF

The eMetrics Marketing Optimization Summit happens May 4-7 and I'm so looking forward it!  Although the conference is practically walking distance from home I still think of it as a Big Trip.  Here's what I'm excited about:

I get to speak.  James Gardner and I will do a modified reprise of the presentation we gave at eMetrics DC last fall, and we'll be sharing the hour with the inimitable Dylan Lewis and Mark Brooks.  The four of us have worked out a nicely-coordinated set of talks focused on career development and staffing in web analytics.  Read about our track.

I also get to listen.  With gusto I will attend as many sessions as I can.  After the conference I'll do a write-up on my blog.  If you just can't wait that long I may also post some juicy tidbits on Twitter (tsk, Twitter) so feel free to follow me.

And I am throwing a big party. Have you not heard?  Web Analytics Wednesday happens on May 6th and you're invited.  Muchas gracias to Eric T. Peterson, David Rogers, our fine sponsors (Coremetrics, SiteSpect, ForeSee, Tealeaf and eMetrics), our super volunteers and every single participant.  Sign up now if you haven't already. We already have more than 160 web analytics professionals signed up to attend!

If you're attending eMetrics and you'd like to meet, please introduce yourself - I still look like the picture on my blog, except I have a new short haircut.  If you are in the vicinity but you can't spring for an eMetrics ticket, don't despair!  There are 3 associated free activities during the week: Web Analytics Wednesday, the WAA Raucous Caucus, and the eMetrics Expo-Only pass.

Here's to eMetrics.  It's going to be a busy week but a lot of fun.

Web Analytics Data Reconciliation How-To Guide

I suspect that most experienced web analysts have done at least one data reconciliation project during the course of their tenure.  For something so common, however, it rarely gets discussed. 

Sure, it's not sexy like Angelina Jolie, but even Plain Jane likes a little attention now and then.

Data reconciliation is an important foundational activity because, when done well, it will inspire people to have confidence in the data that you share with them. Data quality will never be perfect, but it should be good enough for everyone to feel that they can make sound business decisions based on what's available.

Enough pep talk.  If you're on the brink of your first data reconciliation project, here's what to do:

1) Identify your two data sources

The need for data reconciliation arises when you have two separate systems that provide similar sets of data.  One of these sources - let's call it "Primary Source" - will necessarily be your standard web analytics application.  The other one - let's call it "Secondary Source" - can be one of several things, namely:

• An upstream system, like campaigns (banner, search, email)
• A downstream system, like commerce or downloads or form submissions
• A parallel system, such as when you migrate from one web analytics tool to another.  In this case I'd advise you to break your project into smaller chunks according to individual reports you care to reconcile.

2) Learn how your primary source gets collected

Read the documentation and talk to your internal tech team.  Be clear on the scope of the data you're collecting - ie exactly which pages are tagged, or exactly which log files are processed.  If you're using page tags, know whether the tag is placed at the top or the bottom of the page (this will affect when the tag fires, which in turn affects the level of data loss to some extent).  Make note of any special filters, transformations or business logic used here.

3) Learn how your secondary source gets collected

If your secondary source is a parallel web analytics system, repeat the process you followed in step 2, above. 

If it's an upstream system you're stuck with whatever documentation and lore you can glean regarding how that works. 

If it's a downstream system you'll need to identify the group within your business that owns that system, then grill them on how they do data collection and how they transform the data into the metric you're trying to reconcile.  There's a lot of variability here, especially if your downstream system is homegrown, so be sure to do a thorough investigation.  As in step 2, make note of any special filters, transformations or business logic used here.

4) Compare data sets

Pick a sensible date range and granularity level, then pull corresponding data from both sources.  A good default would be daily totals for a month.  If you're dealing with really high volume you may want to isolate a subset of your data based on some attribute that you can reliably pull from both sources, like a single URL (for downloads) or a single product (for commerce).

Now put your data sets side by side in Excel and calculate the delta.  Compare the trends over time and see if you can explain the differences.  Ask yourself, are you comfortable with the differences you see?  If not, consider fine-tuning the way you pull data from your primary and/or secondary source in order to account for those differences.

Reality check: you're never going to get a perfect match.  This is a good exercise, but know when to say when. Do not obsess!

5) Document and share your findings

This is the most important step.  Write a report about what you've done and what you've found.  Now go talk to people - give a verbal presentation of findings to your web analytics colleagues and your concerned data stakeholders. 

At this point you should be able to speak with confidence about the differences in the two data sources, and your goal should be to pass this confidence on to the people around you.  Save your report for future reference, as newcomers are likely to ask the questions you've already answered.

6) Plan to revisit if necessary

If reconciliation is part of tool migration, you are now done.  Good work.

If your secondary source is an upstream or downstream system, plan a periodic audit to make sure your findings are still valid.  If your systems are stable you can get away with doing this maybe once a year, but if you have any appreciable changes - like a major site redesign or a shopping cart overhaul - you may wish to do another quick round of reconciliation at that time.

Podcast Interview on the Aquent Talent Blog

Matthew Grant of the Aquent Talent Blog recently conducted a podcast interview with me; I invite you to listen to it here.  If you haven't got time for the whole show, skip straight to the highlight at 11:45, where I somehow manage to divert our conversation from web analytics to goats.

Aside from that one sidetrack we did actually talk a lot about web analytics, and, in particular, web analytics careers.  You'll find the podcast interesting if you're a newbie web analyst, and I'd also recommend it if you're thinking about taking contract work in web analytics (as I'd done prior to joining Semphonic).

I met Matthew earlier this year through his Aquent colleague, James Gardner.  James and I gave a joint presentation at eMetrics DC last fall on career management for web analysts, and we'll do a reprise at eMetrics SF next month.

Where to Put Integrated Data: 7 Helpful Questions

Web analytics data integration goes both ways.  When you marry clickstream data with other business data, you can put the combined result either inside or outside your web analytics application.  The trick is, if you can put it either place, how do you decide which place is best? 

Here are 7 questions to consider as you make your decision:

1. Is this a once-off or will you need an ongoing feed?  Say you're working on a deep-dive analysis project, or you're preparing a data set to use for data mining.  You're probably pulling activity from a discrete period of time.  If so, integrate outside your web analytics application, where there's less overhead for a one-time task.  If, on the other hand, you're going to want this integrated data to be available at moments notice for all eternity, you're best off integrating wherever you can most easily automate your feed, which brings me to my next point:


2. How much effort will it take to automate, in vs. out?  Call me lazy or call me practical, sometimes the right answer is the the easiest one (that's Occam's Razor, right?). The major commercial web analytics vendors have built-in integration tools, like Coremetrics Connect and Omniture Genesis.  If the data you need to integrate falls within the realm of what your web analytics application can handle, use the wizard and take a feed in.  If, on the other hand, you want to integrate custom data that's not wizard-able, take a feed out instead - but make sure you've got IT resources to help you automate the load into the destination system.


3. Which analysis tools do your data consumers prefer to use?  Maybe you've got a favorite data visualization application (like Tableau), or predictive modeling software, or another business intelligence tool that people at your company like to use.  Yes?  Then integrate your data in a place where it will be easy to get at using that tool, most likely outside your web analytics application.  If you plan to use Excel you have more of a choice, because most web analytics vendors have Excel plug-ins.  You could integrate within your web analytics application and then feed it to Excel, or, if your data set is small enough, you could integrate by VLOOKUP()-ing right there inside Excel.


4. Are your data consumers already active users of your web analytics application?  If so, you'd be doing them a favor by putting the integrated data where they're most likely to use it.  On the other hand, if they spend all day working with some other business data system, put it there instead.  It could be the factor that determines whether the integrated data ever gets adopted in practice by the people who are expected to use it. 


5. Will you need reporting components that web analytics applications handle especially well, like browser overlay and pathing?  This will depend on whether your web analytics application actually lets you display integrated data in browser overlay and pathing reports.  If so, and if you can imagine actually using these reporting components, try to integrate inside your web analytics application.  Although web analytics applications are not as robust, generally speaking, as other data analysis tools, they manage to do a good job of presenting clickstream-specific data. 


6. Are you hoping to integrate data that can actually be gathered at collection time?  Maybe the extra business data you want to integrate is something you'll be able to assign to a custom variable in your web analytics application at collection time.  If so, you'll be able to integrate without any after-the-fact joining.  If your integration data doesn't surface until further downstream, though, you can't use this approach.
   


7. Do you need to store your integrated data behind the corporate firewall?  This isn't so much a technical issue as a legal one.  If the data you want to integrate involves personally-identifiable information and you're using a hosted web analytics solution, go re-read your site's privacy policy.  Chances are you will need to store the integrated data behind your own corporate firewall.  If you host your own web analytics application on-site you may still be able to integrate inside it, otherwise you'll need to pull a feed out.

So, depending on your situation it's perfectly reasonable to join data in both directions - inside and outside your web analytics application.  Strive to find a solution that's practical, easy, legal, and most likely to make your data analysts happy.

Urchin Sticker Followup

My vintage Urchin sticker giveaway has ended.  Last week I mailed envelopes - literally all over the world - to 10 lucky recipients.  It was a fun way to meet my blog readers, so thanks for participating.

Here's a sticker I kept for myself, stuck to my home laptop:

WAA Board of Directors Election: I Need Your Vote!

The Web Analytics Association Board of Directors election is imminent and my name is on the ballot.  There are 17 great candidates running for 6 positions, so it will be a tough race.  Dear blog reader, I need your vote!

Polls will be open March 24 - April 7, 2008.  You must be a WAA member to participate.  If you're reading this and you're not a WAA member, you really oughtta sign up.

As part of the election application process each candidate had to answer 3 questions; here's how I responded:

What major contribution will you bring to Web Analytics Association and its membership?

I have spent most of my long career in web analytics as a practitioner, in the trenches, actually doing the work.  With that experience under my belt, I bring first-hand knowledge of the challenges we face and the values we hold as web analysts.  I have grown up with this field and now I want to help shape it.

My subject-level interests include web analytics career development, mentoring, education, community, local presence and providing demonstrable benefit to all members.

Why should members vote for you?

I live up to high professional standards in all work I do, and I intend to bring these standards with me to the WAA Board:

• I am 100% committed to active participation and follow-through.  Since Board members are volunteers, I feel that this is especially important.  We’re here because we want to be here, and we are as involved as we want to be.  I want to be involved.
• Ethical responsibility is very important to me and I will strive to do what’s right for our industry as a whole, with no preferential treatment of any strata, company or individual. 
• I am diplomatic, I enjoy bringing people together, and I am committed to the idea that we must work together to achieve the best for our field.
• I will listen to what you have to say and take it seriously.  I take pleasure in corresponding with all members of the web analytics community, and I will make sure that your input is given the attention it deserves.

Where do you think the Web Analytics Association should be in the next two years?

Our field is growing – there are more web analytics professionals every day.  We must work to ensure that the WAA member base continues to grow along with our field, and, at the same time, we must provide existing members with benefit that will encourage continued involvement.

Our field is also evolving – the scope of what we do now is broader than it has been in the past.  We must acknowledge this shift, forge connections with related associations, and remain open to new ideas.  It’s important for the WAA to continue to represent the most current scope of our profession, and think forward to where it’s headed yet even further down the road.

Speaking as someone who has paid WAA membership dues out-of-pocket for 3 years running, I aim to ensure that, 2 years from now, membership carries the clout and offers the value that will justify the expense for every one of us.

Vintage Urchin Sticker Giveaway

After reading Lars Johansson's entertaining post on Web Analytics Memorabilia I immediately went for a dig in my sticker pile. [Does everyone have a sticker pile, or is that just me?]

Story: Back in 2001 I worked for a first-generation web analytics vendor called WhiteCross Systems.  One day, while researching the competition, I found a vendor called Urchin.  They were giving away stickers on their web site, and I like stickers, so I sent away for some.  Two weeks later the stickers arrived.  I promptly tossed them in my sticker pile and forgot about them for 7 years.

Fast-forward to 2008.  Urchin has become Google Analytics.  My stickers have acquired retro-cool appeal.  I've been hoarding them long enough, so if you'd like a sticker, write to me and I'll mail you one.  I have 10 to give away.

[Update, March 19th: My stickers are all gone!  The giveaway has ended.]