Real-World Challenges of Tracking Mobile with Page Tags

[I offer a more-formal-than-not article based on recent real-world experience doing mobile analytics for a Semphonic client.  You will be interested if you are a traditional web analyst who soon faces the added responsibility of measuring mobile.

For what's bound to be an engaging discussion about mobile analytics, come to the X Change Conference and attend Bob Schukai's 2 huddles on the topic.  Bob advised me as I wrote the following article and I was really impressed with his depth of knowledge about the mobile space.]

Real-World Challenges of Tracking Mobile with Page Tags

As the mobile web grows in popularity, so does the need for mobile measurement.  When put to the test, say, with a basic mobile campaign, we find that mobile measurement has not yet reached the level of maturity that we have come to expect from traditional web analytics.

If we treat mobile measurement like traditional web measurement we face challenges right away, starting with data collection.

Many popular web analytics tools available today – like Omniture and Google Analytics – rely on JavaScript tags for data collection.  Using this method, site owners place a tag on every page they wish to track; when a visitor accesses a tagged page, event-level data is sent back to the web analytics tool. 

Tracking mobile web in this manner is problematic, though, because mobile browsers do not reliably support JavaScript.  Some tag-based web analytics tools provide a fallback tracking method when JavaScript is not present – namely, a hard-coded 1x1 pixel image request that collects a subset of the standard event-level data.

An increasing number of mobile-specific tracking solutions are surfacing on the market; they offer arguably more robust data collection methods than traditional web analytics tools.  [My colleague Phil Kemelor has written a nice series on the mobile analytics vendor landscape, if you're interested.] However, despite known deficiencies, site owners may still opt to use a traditional web analytics tool for mobile measurement simply because it’s already an accepted business standard for tracking existing web content. 

In this article we will outline some of the challenges faced when using a tag-based web analytics tool adapted – by way of an image request – to measure mobile. 

An example

Consider two very similar online campaigns aimed at promoting the download of an application: the first campaign goes out as an email blast on traditional web, and the second campaign goes out as an SMS blast on mobile.  Beyond the difference in screen size, these two campaigns look a lot alike: a message, a click, a download.  But, when it comes to measuring the effectiveness of each campaign, mobile brings a whole new set of challenges.

Using this example – a pair of simple campaigns – we will illustrate common mobile measurement issues concerning campaign clickthroughs and downloadable files.

How do we track campaign clickthroughs?

In our campaigns, site-side tracking begins the moment someone clicks through from the message they receive to the landing page on the site.  This tracking process is well-defined on traditional web, but less so on mobile.

Campaign clickthroughs are typically tracked using a code appended to the query string of each URL.  When a visitor clicks through to the site, the campaign code gets collected by the JavaScript tag on the landing page and is then passed on to the web analytics tool.  Without the use of JavaScript, a bit more effort is required.

On mobile, and with image requests for data collection, we cannot easily intercept campaign code query parameters on URLs.  As a workaround, unique landing pages can be set up to track traffic from each campaign.  This approach, although it works, does add a layer of complexity to the campaign creation process beyond what we would normally do to track campaigns on traditional web sites.

How do we track downloads?

The measure of success in our campaigns is the download of a file.  Once again, established tracking techniques on traditional web must be re-evaluated on mobile.

Traditional tag-based web analytics tools track downloads at the moment the visitor clicks on a link to get a file.  This counts downloads initiated, not downloads completed, but it is a concession we’re willing to make in return for the convenience of tag-based tracking.  [For more on this somewhat-sticky subject see my recent post on downloads and page tags.]  However, since link click tracking requires the use of JavaScript, even this limited functionality is lacking on mobile.

On mobile, and with image requests for data collection, we must make do by treating the page just prior to download as a signifier of the download itself.  This method will necessarily inflate the number of downloads, though, since it’s entirely possible to back out of the process before actually requesting the file.  In this case, clear communication to data consumers is recommended in order to set expectations about data quality.

After all that, just the basics

The tracking techniques described above provide the absolute bare minimum we need in order to measure mobile campaign effectiveness, namely page views of the campaign landing page and downloads of the file.  Tracking becomes more problematic if we hope to get accurate visit and unique visitor counts; mobile browsers do not reliably set cookies, and we are left with very weak rules for visitor identification.

The lesson learned, in comparing very similar activities across traditional web and mobile, is that mobile requires somewhat more effort just to get the most basic tracking in place.  While we need not reinvent the wheel as we move into this new medium, mobile does force us to reconsider some of our fundamental assumptions about web measurement.

Comments

[Jacques Warren]

Hi June,

Finally, a concrete and informed post about mobile analytics in our blogosphere!!

Shall we announce the return of the server log files??

It seems that "hybrid" companies such as WebTrends could play the mobile analytics card, while the market is not entirely dominated by smartphones (which executes javascript and accept cookies I understand).

[June Dershewitz]

Thanks for your feedback, Jacques! I know I'll see *you* in the mobile analytics huddles at X Change.

It is sort of funny that data collection methods come into question again as web analysts begin to measure mobile. A few years ago we saw the shift in preference toward page tags; now - for at least some customers - we may see a shift *away* from page tags.

The pure page tagging approach is clean and convenient, sure, but it doesn't always give us what we need.

[dean collins]

Hi June,

Where's the link love?

How come you didn't mention that Amethon Mobile Analytics are the only vendor who uses Wireline Capture for our mobile analytics solution.

Because this methodology is the only mobile analytics solution that can see 100% of the packets traveling from the handset to the mobile web content server we can tell not only when a file begins to download but also if the file download is abandoned half way.

But you already know this.....

For those that dont - check out http://www.amethon.com/Content_Common/pg-Mobile-Analytics.seo

Cheers,
Dean Collins
www.Amethon.com

[June Dershewitz]

Hi Dean, thanks for reading! As weak as page tagging is, I think that some web marketers are willing to try it out anyway because it's a tracking solution they've already got in their repertoire of tools.

In the case of campaign tracking it would be great to see mobile campaigns listed alongside traditional web campaigns in the reporting interface we analysts already use every day. Obviously we're not there yet.

In the meantime I think that web marketers will turn to separate measurement solutions - such as Amethon's fine product - that rely on alternate methods of data collection. Certainly we could overcome the download tracking deficiency I describe here.

[Andy Bovingdon]

Great summary June, you make many valid points. For most marketers the idea of adding a simple image tag to pages is much easier than installing and maintaining additional hardware. I just wanted to mention a few tricks we can play on mobile that helps alleviate some of the problems you mention.

With campaign clickthroughs we find benefits in using similar URL redirection tricks that many of the ad companies use. This reliably tracks the action of the consumer clicking the ad and simply redirects to a single consistent landing page. In fact you can then use image tags to detect the actual page view. If you have a reliable ID for each user you can directly tie the two events together to measure each campaign and compare against natural browse-ins.

Not all image tag technologies are equal – while each use the same basic mechanism, some are more advanced than traditional desktop solutions. For example the Bango mobile web analytics product adds our unique user ID technology on top. This uses a number of steps to allocate a persistent identity to each visitor allowing you to easily track repeat visits across many months. Like the PC this will use cookies where possible, but it also uses handset configuration data and secure information available to Bango from a number of carriers – we call these steps our fingerprinting process. You can find out more at http://bango.com/analytics.

One thing to be warned of is blindly using an existing web analytics solution on mobile. While some work but simply don’t tell you enough, others actually break your site on many handsets. Quite often the mobile site owner does not realise this problem early enough because they don’t have a sufficient range of handsets to test against.

Regards,

Andy Bovingdon
http://bango.com

[Scott Zakrajsek]

I agree completely with Jaques's post above, this is an excellent post on mobile analytics, and in plain english no less.

We launched our mobile site a few months back and track it with page tags stuffed into image requests as mentioned above. It has been quite the learning experience for myself as well as our mobile vendor.

We began with simple page view tracking which led to phase 2, tracking transactional and cart data (orders, sales, abandonment rates, all that good stuff). Now there is an appetite to survey our mobile customers which means we'll need to collect their email addresses upon login (as I do not think we can capture it in the URL string). Page tracking in this media is a cumbersome process at times, but our mobile traffic is very small now and does not require a dedicated mobile analytics vendor. However, this could very well change in the future.

I guess the part I struggle with most is that we've recently received some great support from senior leadership on our web data, and its difficult for me to tell them why our mobile data isn't as "clean". But you probably know all about that...

http://www.webanalyticsassociation.org/en/art/?473 (#4)

[June Dershewitz]

@Andy: Now that I know what hoops we must jump through to track campaigns on mobile I can appreciate the existence of a solution such as the one your company provides.

I take interest in your comment about web analytics solutions "breaking" mobile content on some handsets. How troublesome! The ability to test our tracking on mobile is severely limited (where's WASP for mobile, Stephane?) and I know we lack visibility regarding what works and what doesn't.

@Scott: Thanks for sharing your experience. I think your company's rationale ("our effort is not a big enough deal to merit a dedicated mobile analytics vendor") is probably pretty common ... at least until marketers demand more than page views. :^/

And I know *exactly* what you mean about having to repeatedly explain why mobile data quality sucks even more than regular web data quality. (Glad you found that article!)

[Andy Bovingdon]

The testing aspect is an interesting one June. At Bango we used to have a large testing lab full of the latest handsets. This has now been totally replaced by Device Anywhere from Mobile Complete (http://www.deviceanywhere.com/). It provides remote access to real handsets on a range of networks worldwide. Makes it very simple to check your mobile site against a selection of top handsets.

We then use Bango Analytics (http://bango.com/analytics) to monitor top handset traffic and re-test the sites on those devices to ensure optimal experience. We also check handsets hitting error pages and test these devices also to see why they may have failed.

Regards,

Andy
http://bango.com

[Steve Jackson]

Hi June,

Unfortunately I won't be able to make Xchange but thanks for sharing. I work with a lot of telco companies and we've run into this problem a number of times. The point about mobile not being big enough to justify installation of another tool is the main issue we've found as well.

I think the main reason Omniture or GA for instance haven't done anything about the problem is the trend in mobile browsing and the rate of adoption. All (smart)phones execute Javascript and allow cookies. As the phones get better and the older models are phased out it means Omniture for instance don't have an incentive in investing heavily in mobile tracking. Why bother investing when in 1-2 years all phones being used for browsing will be tracked to a certain extent anyway?

The problems you mention (downloads/campaigns) are minor in comparison to the costs of adding an additional tool to an enterprises suite of tracking tools in my opinion.

The real challenge with mobile technology is how will we start tracking offline actions and this is something had I been going to Xchange I would have raised. When advertisers start using GSM to know your location and push adverts based on where you are physically we have a different set of challenges. Vouchers/Bar codes are one way to track the success of such campaigns but advertisers will need more information than simply who buys and who doesn't.

In my view this is something we should start thinking about sooner rather than later.

Br
Steve.

[Micah Fisher-Kirshner]

Perfect timing! I was just going to start searching for information on mobile analytics this week and read your blog over the weekend. Thank you for putting this information out there and making life easier for us all.

[June Dershewitz]

@Steve: Thanks for pointing out that this is indeed a temporary situation. Eventually the majority of our mobile devices will have JavaScript and cookies, at which time we'll get the same level of tracking that we do with traditional web sites.

In the meantime ... well, we either make do with our existing analytics system's limited functionality, we accept the additional expense and get a mobile-specific solution, or (my least favorite) we don't measure at all.

Regarding location-based tracking - ooh analysts are salivating over that! Vendors continue to make progress when it comes to data integration, so I'm hopeful that, by the time we're ready for it, it won't be too cumbersome to connect the dots.

@Micah: Glad you found my article! Come to SF WAW on July 23rd and I'll gripe in even more detail. Uh, that's an offer, not a threat.

[Leevi Kokko]

One thing which can also severely limit the mobile analytics data quality is that many mobile network operators actually tamper the traffic data, ripping HTTP headers and blocking cookies at their gateways. So, to add another layer of complexity to the challenge, it's not only the handset scripting and cookie capabilities that matter, but also the network which is being used to access service.

Thanks for the great post, June!