I've been blogging for a few months now and it's been really fun, especially when it comes to choosing topics to write about. So far I've posted about career development, events I attend and host, data integration, user-generated content, ... heck, I've even posted a picture of my family's pet chicken.
Now, in an attempt to say more about what I actually do at Semphonic, I thought I'd tell you about one project I'm currently working on. Without giving out any identifying details about my client I'd like to share some tidbits that will be generally useful if you find yourself in a similar position.
So I'm doing a site redesign analysis project. My client has recently made some major structural, interface and content changes to their site, and I'm comparing visitor behavior (and outcomes and sentiment) before and after the launch of the new site.
Redesign analysis is something I've done numerous times throughout my career in web analytics. Simple fact: sites get redesigned over and over. Analyzing the impact of redesign is mostly about picking the right things to measure and then coming up with an interesting story to tell project stakeholders. Let it be clear that I'm talking about sweeping changes to a whole site that happen at the flip of a switch, not incremental changes made by way of testing.
Sometimes a redesign launches and I - as the measurement person - have looked back and said, "D'oh! I should have thought of that in advance!" In the remainder of this post I'll list out 3 ways I prepare for a site redesign measurement project. Setting up these things ahead of time will allow you to avoid some known pitfalls.
Redesign Prep #1: Take Screenshots
What to do:
Take "before" screenshots of all the major pages on your site and put them aside for safekeeping. Yes, you may be able to dig archived pages out of a content management system, but screenshots are a more reliable bet.Let me just say I absolutely love the screen capture utility called SnagIt, especially the scrolling window functionality. I recently discovered that SnagIt can also preserve links on a web page, which is quite useful.
Why to do it:
Because you'll need to know what the site looked like prior to redesign, and you may want to include these screenshots in your analysis presentation.Redesign Prep #2: Decide on Timing
What to do:
Take out your calendar, pick "pre" and "post" windows of time to analyze, then pick dates a little farther out from that when you'll be able to share your analysis findings.It's a bit of a balancing act. Invariably site owners want to know immediately whether or not their effort was a success, but from the analysis standpoint it's advisable to wait until a sufficient period of time has passed before you draw any conclusions about the impact of redesign.
In my current project I'm doing a 2-week quick check-in followed by a 1-month post final analysis - that way I'm able to give some immediate feedback while I wait for the data to roll in and then use the longer time period in my final presentation.
Why to do it:
Because being proactive rather than reactive about data analysis is simply good form, plus it gives redesign project stakeholders some dates to look forward to.Redesign Prep #3: Grab Data
What to do:
Figure out if there's any historical data you won't be able to get after the site launches. This varies depending on what tool you're using, but browser overlay and "Next Page" reports are the most commonly affected.What I mean is, you may only be able to view the browser overlay for the period of time when your page looks exactly like it does today. For dates in the past when the page looked different, your browser overlay report may be unintelligible.
Know what's fleeting, then go in and grab data for your "pre" time period(s) while the data is still available. If, like me, you choose to analyze two different windows of time - a short period and a long period - be sure to collect both of these snapshots. Be thorough.
Why to do it:
Because you don't want to be faced with gaps in data as you pull together your analysis.
So, how do you prepare for measuring redesign? Anything you'd like to add to what I've mentioned here? I welcome your comments.
Hi June,
Quite interesting. What do you make of people who say that two very different web site versions can't be compared (I'm not one of them)? Also, what's your take on "sweeping changes" versus incremental optimization? I understand that often company politics will decide on entirely redoing a site instead of trying to fix the current one, but isn't it risky?
Posted by: Jacques Warren | January 14, 2008 at 11:57 AM
Hi Jacques! Of course you ask the hard questions.
> What do you make of people who say that two very
> different web site versions can't be compared
I have always felt that it's possible to assess the effectiveness of web content based on its intended purpose (ie regardless of what color it is and where the navigation links are and what the copy says and all other superficial attributes). My move to Semphonic has not changed my approach, except now I refer to it as Functionalism. :-)
> Also, what's your take on "sweeping changes" versus
> incremental optimization?
I say, do whatever it takes in order to make forward progress. If, organizationally speaking, it's easier and faster to make a sweeping change, get out there and do it. It's always possible to follow up with refinements using incremental optimization techniques like MVT.
Posted by: June | January 14, 2008 at 09:02 PM
Hey June,
Definitely ask clients what questions they'll have about the new design BEFORE it launches.
Often times the questions they have require special tagging--onclick events, event tracking, etc.
If you don't ask, sometimes the opportunity to answer the question is delayed or lost forever.
-Alex
www.alexlcohen.com
Posted by: Alex | January 17, 2008 at 07:36 AM
Good point, Alex! It's important to make sure that redesign won't break data collection. I've heard plenty of sob stories.
Posted by: June | January 17, 2008 at 08:35 AM